]>
git.sesse.net Git - plocate/log
Steinar H. Gunderson [Sat, 12 Jun 2021 12:27:21 +0000 (14:27 +0200)]
Release plocate 1.1.8.
Steinar H. Gunderson [Sat, 5 Jun 2021 17:31:12 +0000 (19:31 +0200)]
Fix a printing issue when outputting to a block-buffered pipe from a forked child (ie., multiple databases).
Steinar H. Gunderson [Wed, 7 Apr 2021 20:04:49 +0000 (22:04 +0200)]
Fix a typo in try_complete_pread().
Pointed out by Taj Khattra.
Steinar H. Gunderson [Wed, 7 Apr 2021 20:04:40 +0000 (22:04 +0200)]
Bump version number.
Steinar H. Gunderson [Wed, 7 Apr 2021 07:34:46 +0000 (09:34 +0200)]
Release plocate 1.1.7.
Steinar H. Gunderson [Wed, 7 Apr 2021 07:33:33 +0000 (09:33 +0200)]
Move timer settings from .service to .timer file.
Patch from Vladimir Lomov.
Steinar H. Gunderson [Wed, 7 Apr 2021 07:33:10 +0000 (09:33 +0200)]
Bump version number.
Steinar H. Gunderson [Fri, 2 Apr 2021 11:09:10 +0000 (13:09 +0200)]
Release plocate 1.1.6.
Steinar H. Gunderson [Fri, 2 Apr 2021 11:12:49 +0000 (13:12 +0200)]
Randomize updatedb systemd unit start time.
Suggested by Calum McConnell.
Steinar H. Gunderson [Sat, 27 Mar 2021 23:19:49 +0000 (00:19 +0100)]
Support searching multiple databases in plocate.
We support multiple -d arguments, single -d arguments with multiple
databases (colon-separated), and LOCATE_PATH. The two latter should
be compatible with mlocate, although we support escaping colons in
file paths and I believe mlocate does not.
This is slightly trickier than one might expect due to security
considerations; we're not robust against malicious input, so we need to
make sure that if we process an attacker-supplied database, the process
has already dropped privileges and cannot subvert a privileged reader.
Steinar H. Gunderson [Sun, 21 Mar 2021 14:48:55 +0000 (15:48 +0100)]
Make updatedb understand DT_UNKNOWN.
Some filesystems don't know from getdents() whether an entry is a file
or a directory without a stat(). I had assumed this was only an issue
for obscure operating systems, so I removed it (mlocate's updatedb
supported it), but evidently older versions of XFS has this issue, too,
so add back checking.
Reported by Marcel Partap.
Steinar H. Gunderson [Sun, 21 Mar 2021 12:11:45 +0000 (13:11 +0100)]
Give a slightly better error message on short read.
Steinar H. Gunderson [Sun, 21 Mar 2021 12:11:35 +0000 (13:11 +0100)]
Bump version number.
Steinar H. Gunderson [Wed, 24 Feb 2021 18:14:11 +0000 (19:14 +0100)]
Release plocate 1.1.5.
Steinar H. Gunderson [Fri, 19 Feb 2021 17:54:35 +0000 (18:54 +0100)]
Reduce the amount of lseek() syscalls; evidently, ftell() does not cache the position.
Steinar H. Gunderson [Fri, 19 Feb 2021 17:42:12 +0000 (18:42 +0100)]
Support filesystems that do not support O_TMPFILE, even on Linux.
Evidently, overlayfs does not support O_TMPFILE. Reported by Daniel Lewart.
Steinar H. Gunderson [Fri, 19 Feb 2021 17:38:33 +0000 (18:38 +0100)]
Bump version number.
Steinar H. Gunderson [Sun, 14 Feb 2021 09:56:16 +0000 (10:56 +0100)]
Release plocate 1.1.4.
Steinar H. Gunderson [Sat, 13 Feb 2021 10:06:44 +0000 (11:06 +0100)]
Call strlen() a bit less. Helps ~2% CPU.
Steinar H. Gunderson [Sat, 13 Feb 2021 00:58:31 +0000 (01:58 +0100)]
Microoptimizations to docid counting.
Steinar H. Gunderson [Sat, 13 Feb 2021 00:41:04 +0000 (01:41 +0100)]
Take an assert off the hot path.
Seemingly, Meson doesn't include -DNDEBUG by default, and this assert
is pretty hot, to take it out to be sure.
Steinar H. Gunderson [Sat, 13 Feb 2021 00:30:27 +0000 (01:30 +0100)]
Fix a typo.
Steinar H. Gunderson [Sat, 13 Feb 2021 00:24:58 +0000 (01:24 +0100)]
Switch to histogram-based estimation of PFOR varbyte sizes.
This makes plocate.db ~0.1% larger, but reduces CPU usage by something
like 3%, so it's an OK tradeoff.
Steinar H. Gunderson [Fri, 12 Feb 2021 20:15:24 +0000 (21:15 +0100)]
Optimize trigram reading somewhat.
Helps 1–2% CPU.
Steinar H. Gunderson [Thu, 11 Feb 2021 09:09:38 +0000 (10:09 +0100)]
Replace std::string with std::vector<unsigned char> when encoding posting lists.
This is a ~0.5–1% win since we never use the short-string optimization,
and the vector fits better in cache.
Steinar H. Gunderson [Thu, 11 Feb 2021 08:45:30 +0000 (09:45 +0100)]
Streamline the add_docid() path by removing a redundant comparison. Saves ~1% CPU.
Steinar H. Gunderson [Thu, 11 Feb 2021 08:37:31 +0000 (09:37 +0100)]
Remove an unused member. Saves ~0.5% CPU due to better cache efficiency.
Steinar H. Gunderson [Thu, 11 Feb 2021 08:35:17 +0000 (09:35 +0100)]
Inline EncodingCorpus::get_pl_builder(). Saves ~2% CPU.
Steinar H. Gunderson [Thu, 11 Feb 2021 08:32:16 +0000 (09:32 +0100)]
Rename Corpus to EncodingCorpus, and make it private.
We don't really want two different classes named Corpus; that's pretty
confusing, even though they don't live in the same binary.
Steinar H. Gunderson [Wed, 10 Feb 2021 20:39:37 +0000 (21:39 +0100)]
Install CACHEDIR.tag in /var/lib/plocate.
This helps certain tools understand that everything in the directory
is autogenerated. Suggested by Marco d'Itri.
Jakub Wilk [Wed, 10 Feb 2021 08:13:29 +0000 (09:13 +0100)]
updatedb.8: fix typo.
Jakub Wilk [Wed, 10 Feb 2021 08:13:28 +0000 (09:13 +0100)]
plocate.1: remove spurious space after "(".
Steinar H. Gunderson [Wed, 10 Feb 2021 09:37:44 +0000 (10:37 +0100)]
Bump version number.
Steinar H. Gunderson [Sat, 19 Dec 2020 00:17:59 +0000 (01:17 +0100)]
Release plocate 1.1.3.
Steinar H. Gunderson [Sat, 12 Dec 2020 11:56:46 +0000 (12:56 +0100)]
Compile on systems without O_TMPFILE.
This is a last-resort solution; we don't do unlink-on-signal or similar,
so if updatedb or plocate-build is aborted on such platforms, there
will be an orphan temporary file.
Steinar H. Gunderson [Sat, 12 Dec 2020 11:54:31 +0000 (12:54 +0100)]
Compile on platforms without O_NOATIME.
Steinar H. Gunderson [Sat, 12 Dec 2020 11:43:28 +0000 (12:43 +0100)]
Fix a Clang warning.
Steinar H. Gunderson [Sat, 12 Dec 2020 11:53:45 +0000 (12:53 +0100)]
Bump version number.
Steinar H. Gunderson [Wed, 9 Dec 2020 23:40:03 +0000 (00:40 +0100)]
Release plocate 1.1.2.
Steinar H. Gunderson [Wed, 9 Dec 2020 23:37:41 +0000 (00:37 +0100)]
Increase hard file limit if possible.
Fixes issues where updatedb was run with a too-low open file limit
(both soft and hard), but by root, so we could just raise both limits.
Reported by Gregor Herrmann.
Steinar H. Gunderson [Wed, 9 Dec 2020 23:30:04 +0000 (00:30 +0100)]
Make it possible to rename updatedb (e.g. to updatedb.plocate).
Steinar H. Gunderson [Wed, 9 Dec 2020 23:24:42 +0000 (00:24 +0100)]
Fix a typo in plocate(1).
Steinar H. Gunderson [Wed, 9 Dec 2020 23:23:51 +0000 (00:23 +0100)]
Make updatedb(8) stop referring to mlocate (except in one deliberate place).
Steinar H. Gunderson [Mon, 7 Dec 2020 09:22:52 +0000 (10:22 +0100)]
Turn off ProtectSystem=full.
It turns out this works by bind mounts, which we ignore by default,
causing /usr to be ignored in the database.
Steinar H. Gunderson [Mon, 7 Dec 2020 09:22:39 +0000 (10:22 +0100)]
Fix an issue where the database was too small to create a proper zstd dictionary.
Steinar H. Gunderson [Sat, 5 Dec 2020 09:51:52 +0000 (10:51 +0100)]
Release plocate 1.1.1.
Steinar H. Gunderson [Sat, 5 Dec 2020 09:49:55 +0000 (10:49 +0100)]
Fix an issue where the database could be built with the wrong check_visibility flag.
The check_visibility flag would never be set in the header, and thus be set to some
random variable instead of what the user wanted.
Steinar H. Gunderson [Wed, 2 Dec 2020 22:39:49 +0000 (23:39 +0100)]
Release plocate 1.1.0.
Steinar H. Gunderson [Wed, 2 Dec 2020 22:51:55 +0000 (23:51 +0100)]
Add missing man page .in files.
Steinar H. Gunderson [Tue, 1 Dec 2020 23:34:16 +0000 (00:34 +0100)]
Import the updatedb and updatedb.conf man pages from mlocate.
Steinar H. Gunderson [Sun, 29 Nov 2020 18:30:03 +0000 (19:30 +0100)]
Fix installation when DESTDIR is set.
Steinar H. Gunderson [Sun, 29 Nov 2020 14:07:37 +0000 (15:07 +0100)]
Reuse the locategroup option for the setgid.
Steinar H. Gunderson [Sat, 28 Nov 2020 22:21:06 +0000 (23:21 +0100)]
Small README tweak.
Steinar H. Gunderson [Sat, 28 Nov 2020 22:16:36 +0000 (23:16 +0100)]
In the README, make the group before we install.
Steinar H. Gunderson [Sat, 28 Nov 2020 22:14:45 +0000 (23:14 +0100)]
mlocate -> plocate in some help.
Steinar H. Gunderson [Sat, 28 Nov 2020 22:09:13 +0000 (23:09 +0100)]
setgid to plocate, not mlocate.
Steinar H. Gunderson [Sat, 28 Nov 2020 22:08:23 +0000 (23:08 +0100)]
Add a systemd service and timer for updatedb, obsoleting the cron entry.
Steinar H. Gunderson [Sat, 28 Nov 2020 20:59:21 +0000 (21:59 +0100)]
Fix verbose printing when updatedb reads directories from an old database.
Steinar H. Gunderson [Sat, 28 Nov 2020 20:56:54 +0000 (21:56 +0100)]
Fix writing of the configuration block.
Steinar H. Gunderson [Sat, 28 Nov 2020 20:46:00 +0000 (21:46 +0100)]
Create /var/lib/plocate on install.
Steinar H. Gunderson [Sat, 28 Nov 2020 20:05:56 +0000 (21:05 +0100)]
Put the database file in sharedstatedir.
Steinar H. Gunderson [Sat, 28 Nov 2020 20:05:27 +0000 (21:05 +0100)]
Use join_paths() in Meson.
Steinar H. Gunderson [Sat, 28 Nov 2020 19:57:03 +0000 (20:57 +0100)]
Move to our own directory and group.
Steinar H. Gunderson [Sat, 28 Nov 2020 19:55:25 +0000 (20:55 +0100)]
Update README.
Steinar H. Gunderson [Sat, 28 Nov 2020 17:17:23 +0000 (18:17 +0100)]
Honor the “require visibility” flag (in the negative).
Steinar H. Gunderson [Wed, 25 Nov 2020 00:02:25 +0000 (01:02 +0100)]
Run clang-format.
Steinar H. Gunderson [Sat, 21 Nov 2020 17:23:20 +0000 (18:23 +0100)]
Add a native updatedb.
This incorporates some code from mlocate's updatedb, and thus is compatible
with /etc/updatedb.conf, and supports all the pruning options from it.
All the code has been heavily modified, e.g. the gnulib dependency has been
removed and replaced with STL code (kicking 10k+ lines of code), the bind
mount code has been fixed (it was all broken since the switch from /etc/mtab
to /proc/self/mountinfo) and everything has been reformatted. Like with mlocate,
plocate's updatedb is merging, ie., it can skip readdir() on unchanged
directories. (The logic here is also copied pretty verbatim from mlocate.)
updatedb reads plocate's native format; there's a new max_version 2 that
contains directory timestamps (without it, updatedb will fall back to a full
scan). The timestamps increase the database size by only about 1%, which is a
good tradeoff when we're getting rid of the entire mlocate database.
We liberally use modern features to simplify the implementation; in particular,
openat() to avoid race conditions, instead of mlocate's complicated chdir() dance.
Unfortunately, the combination of the slightly strange storage order from mlocate,
and openat(), means we can need to keep up a bunch of file descriptors open,
but they are not an expensive resource these days, and we try to bump the
limit ourselves if we are allowed to. We also use O_TMPFILE, to make sure we
never leave a half-finished file lying around (mlocate's updatedb tries to
catch signals instead). All of this may hinder portability, so we might ease up
on the requirements later. We don't use io_uring for updatedb at this point.
plocate-build does not write the needed timestamps, so the first upgrade from
mlocate to native plocate requires a full rescan.
NOTE: The format is _not_ frozen yet, and won't be until actual release.
Steinar H. Gunderson [Sun, 22 Nov 2020 20:28:57 +0000 (21:28 +0100)]
Move complete_pread() into a separate file, to avoid the io_uring false dependency.
Steinar H. Gunderson [Sat, 21 Nov 2020 14:34:59 +0000 (15:34 +0100)]
Make DatabaseBuilder write the file atomically.
By opening with O_TMPFILE, we guarantee we'll never be leaving
an unfinished file visible on the filesystem. The move across the
old one isn't atomic, but the window of failure is very small now.
Steinar H. Gunderson [Tue, 10 Nov 2020 18:01:48 +0000 (19:01 +0100)]
Remove unfinished debug code.
Steinar H. Gunderson [Tue, 10 Nov 2020 00:09:31 +0000 (01:09 +0100)]
Split DatabaseBuilder into its own compilation unit.
Steinar H. Gunderson [Mon, 9 Nov 2020 23:19:58 +0000 (00:19 +0100)]
When reading mlocate.db, properly skip the configuration block.
This could cause some entries to be skipped until we regained sync,
especially in the root directory.
Steinar H. Gunderson [Mon, 9 Nov 2020 22:50:20 +0000 (23:50 +0100)]
Encapsulate some database-building logic into a class.
Steinar H. Gunderson [Sat, 7 Nov 2020 10:22:52 +0000 (11:22 +0100)]
Escape file names with backticks in them.
Steinar H. Gunderson [Sat, 7 Nov 2020 10:16:46 +0000 (11:16 +0100)]
Bump version number.
Steinar H. Gunderson [Sat, 31 Oct 2020 22:04:39 +0000 (23:04 +0100)]
Release plocate 1.0.7.
Steinar H. Gunderson [Sat, 31 Oct 2020 21:27:41 +0000 (22:27 +0100)]
Fix an infinite loop when encountering invalid UTF-8 in file names.
Bug report and patch by Leah Neukirchen.
Leah Neukirchen [Sat, 31 Oct 2020 18:52:34 +0000 (19:52 +0100)]
Fix two typos in manpages.
Steinar H. Gunderson [Sat, 31 Oct 2020 14:13:38 +0000 (15:13 +0100)]
Run clang-format.
Steinar H. Gunderson [Sat, 31 Oct 2020 11:25:36 +0000 (12:25 +0100)]
Work around brokenness in FreeBSD mbtowc().
The manpage claims the return value should be 0 on a null byte,
just like on Linux, but in practice, it returns -1, so we need to
check for end-of-string manually.
Steinar H. Gunderson [Sat, 31 Oct 2020 11:24:34 +0000 (12:24 +0100)]
Check for endian.h before including it. Fixes compilation on FreeBSD.
Steinar H. Gunderson [Sat, 31 Oct 2020 10:57:29 +0000 (11:57 +0100)]
Add missing <endian.h> include.
Seemingly improves musl compatibility. Taken from the Void Linux
packaging repository.
Steinar H. Gunderson [Fri, 30 Oct 2020 23:44:14 +0000 (00:44 +0100)]
Bump version number.
Steinar H. Gunderson [Thu, 29 Oct 2020 23:06:10 +0000 (00:06 +0100)]
Release plocate 1.0.6.
Steinar H. Gunderson [Thu, 29 Oct 2020 22:42:01 +0000 (23:42 +0100)]
Escape unprintable characters when outputting filenames to a terminal.
Filenames are generally untrusted, and can contain any kind of cruft.
In particular, there have been terminals (hopefully not in wide use anymore!)
that will do insanity like running specific commands when seeing a
specific escape sequence. More prosaically, embedded newlines can
make for confusing output.
Thus, escape any nonprintable characters in a shell-parseable way,
much the same way GNU ls does these days. Also escape quotes, backslashes
and the likes to make sure nothing unescaped looks like it's escaped.
This doesn't mean it's safe to take whatever and parse it uncritically
(we don't escape $, for instance), but it's generally good enough.
Escaping is disabled when doing zero-terminated output, or when printing
to a pipe or file.
Steinar H. Gunderson [Tue, 20 Oct 2020 16:55:37 +0000 (18:55 +0200)]
Fix a crash when we have a too few blocks to train a dictionary.
Steinar H. Gunderson [Tue, 20 Oct 2020 16:53:58 +0000 (18:53 +0200)]
Support building databases from plaintext files.
This was already possible by uncommenting some code, but has now
given a switch and also being made more robust.
Steinar H. Gunderson [Sat, 17 Oct 2020 13:03:39 +0000 (15:03 +0200)]
Add an alternative for __builtin_clz.
Speed isn't critical here, and this was ostensibly the last GCC-ism.
Steinar H. Gunderson [Sat, 17 Oct 2020 12:37:25 +0000 (14:37 +0200)]
Remove some unneeded __attribute__((unused)).
Steinar H. Gunderson [Sat, 17 Oct 2020 12:33:12 +0000 (14:33 +0200)]
Fix the function multiversioning Meson test.
The old one was seemingly too lenient, and would have false positives.
Steinar H. Gunderson [Sat, 17 Oct 2020 12:33:05 +0000 (14:33 +0200)]
Bump version number.
Steinar H. Gunderson [Sat, 17 Oct 2020 09:39:54 +0000 (11:39 +0200)]
Release plocate 1.0.5.
Steinar H. Gunderson [Sat, 17 Oct 2020 09:32:17 +0000 (11:32 +0200)]
Fix the -r short option.
Steinar H. Gunderson [Sat, 17 Oct 2020 09:10:41 +0000 (11:10 +0200)]
Support compiling on x86 platforms without working function multiversioning.
Steinar H. Gunderson [Sat, 17 Oct 2020 07:55:45 +0000 (09:55 +0200)]
clang-format.
Steinar H. Gunderson [Sat, 17 Oct 2020 07:55:18 +0000 (09:55 +0200)]
Add the missing end timing if linear scan and --debug is used together.
Steinar H. Gunderson [Sat, 17 Oct 2020 07:47:13 +0000 (09:47 +0200)]
Fix some inconsistencies in the man page.
Steinar H. Gunderson [Sat, 17 Oct 2020 07:46:55 +0000 (09:46 +0200)]
Implement the -b (--basename) option.
Steinar H. Gunderson [Fri, 16 Oct 2020 08:02:28 +0000 (10:02 +0200)]
Fix a wrong IWYU include.
Steinar H. Gunderson [Fri, 16 Oct 2020 08:01:28 +0000 (10:01 +0200)]
Fix detection of -latomic (it doesn't come from pkg-config).