]>
git.sesse.net Git - plocate/log
Steinar H. Gunderson [Tue, 5 Oct 2021 21:05:55 +0000 (23:05 +0200)]
Release plocate 1.1.12.
Steinar H. Gunderson [Tue, 5 Oct 2021 21:02:09 +0000 (23:02 +0200)]
Implement the -N/--literal option.
This matches the GNU coreutils option to turn off the automatic
quoting of special characters when printing to a tty.
Steinar H. Gunderson [Thu, 23 Sep 2021 17:54:51 +0000 (19:54 +0200)]
Support kernels that don't know about O_TMPFILE.
Seemingly, this hit WSL1, making it impossible to run updatedb there.
Steinar H. Gunderson [Thu, 16 Sep 2021 17:54:46 +0000 (19:54 +0200)]
updatedb is in section 8, not 1.
Steinar H. Gunderson [Thu, 16 Sep 2021 17:02:42 +0000 (19:02 +0200)]
Update the plocate-build man page to inform about the existence of the native updatedb.
Arsen Arsenović [Mon, 23 Aug 2021 22:16:34 +0000 (00:16 +0200)]
fix maybe-uninitialized warning in parse_trigrams
Steinar H. Gunderson [Sat, 4 Sep 2021 22:57:34 +0000 (00:57 +0200)]
Bump version number.
Steinar H. Gunderson [Sat, 4 Sep 2021 22:50:01 +0000 (00:50 +0200)]
Release plocate 1.1.11.
Steinar H. Gunderson [Sat, 4 Sep 2021 22:47:14 +0000 (00:47 +0200)]
Fix a crash on --existing and --regex together.
--regex (and other linear scans) would launch multiple threads,
but IOUringEngine isn't thread-safe. Since --existing is obscure,
and --regex is obscure, we solve this by simply making synchronous
stat() calls.
Reported by Michael Arndt.
Steinar H. Gunderson [Mon, 23 Aug 2021 22:31:32 +0000 (00:31 +0200)]
Bump version number.
Steinar H. Gunderson [Mon, 23 Aug 2021 20:02:36 +0000 (22:02 +0200)]
Release plocate 1.1.10.
Steinar H. Gunderson [Mon, 23 Aug 2021 18:35:55 +0000 (20:35 +0200)]
Use a more C++-standard way of suppressing the unused variable warning in submit_stat().
Steinar H. Gunderson [Mon, 23 Aug 2021 18:29:23 +0000 (20:29 +0200)]
Fix a warning with -Wvla (MB_CUR_MAX is not a compile-time constant).
Arsen Arsenović [Mon, 23 Aug 2021 15:52:48 +0000 (17:52 +0200)]
fix unused parameter warns in io_uring_engine.cpp
Steinar H. Gunderson [Mon, 23 Aug 2021 16:13:08 +0000 (18:13 +0200)]
Unbreak compilation without systemd.
Steinar H. Gunderson [Mon, 23 Aug 2021 15:54:57 +0000 (17:54 +0200)]
Add missing unistd.h #include.
Adapted from the Void Linux repository, for musl compatibility.
Steinar H. Gunderson [Mon, 23 Aug 2021 15:53:34 +0000 (17:53 +0200)]
Replace canonicalize_file_name() with realpath().
canonicalize_file_name() is a GNU extension, while realpath() is POSIX.
Seemingly needed for musl. Adapted from the Void Linux repository.
Arsen Arsenović [Mon, 23 Aug 2021 08:07:01 +0000 (10:07 +0200)]
plocate: fix compilation without libiouring
Steinar H. Gunderson [Sat, 31 Jul 2021 13:37:54 +0000 (15:37 +0200)]
Unbreak the benchmark program.
Steinar H. Gunderson [Wed, 18 Aug 2021 20:16:02 +0000 (22:16 +0200)]
Release plocate 1.1.9.
Steinar H. Gunderson [Tue, 27 Jul 2021 20:16:01 +0000 (22:16 +0200)]
Run clang-format.
Steinar H. Gunderson [Tue, 27 Jul 2021 14:19:01 +0000 (16:19 +0200)]
Support the -e (--existing) option from mlocate.
This can be useful if recently having deleted some files,
and not wanting to do a database rebuild. Note that we don't
support the --nofollow option (--follow is the default),
since it's not clear what it would be useful for, and the mlocate
source code says it “looks like a historical accident”.
Steinar H. Gunderson [Tue, 27 Jul 2021 13:45:27 +0000 (15:45 +0200)]
Send the result value of the stat to the callback.
Steinar H. Gunderson [Mon, 21 Jun 2021 19:21:53 +0000 (21:21 +0200)]
Make it possible to set systemunitdir explicitly.
This allows a distributor to hard-code the systemd unit directory
instead of relying on systemd-dev. Based on a patch by
Zbigniew Jędrzejewski-Szmek.
Steinar H. Gunderson [Mon, 21 Jun 2021 18:57:44 +0000 (20:57 +0200)]
Bump version number.
Steinar H. Gunderson [Sat, 12 Jun 2021 12:27:21 +0000 (14:27 +0200)]
Release plocate 1.1.8.
Steinar H. Gunderson [Sat, 5 Jun 2021 17:31:12 +0000 (19:31 +0200)]
Fix a printing issue when outputting to a block-buffered pipe from a forked child (ie., multiple databases).
Steinar H. Gunderson [Wed, 7 Apr 2021 20:04:49 +0000 (22:04 +0200)]
Fix a typo in try_complete_pread().
Pointed out by Taj Khattra.
Steinar H. Gunderson [Wed, 7 Apr 2021 20:04:40 +0000 (22:04 +0200)]
Bump version number.
Steinar H. Gunderson [Wed, 7 Apr 2021 07:34:46 +0000 (09:34 +0200)]
Release plocate 1.1.7.
Steinar H. Gunderson [Wed, 7 Apr 2021 07:33:33 +0000 (09:33 +0200)]
Move timer settings from .service to .timer file.
Patch from Vladimir Lomov.
Steinar H. Gunderson [Wed, 7 Apr 2021 07:33:10 +0000 (09:33 +0200)]
Bump version number.
Steinar H. Gunderson [Fri, 2 Apr 2021 11:09:10 +0000 (13:09 +0200)]
Release plocate 1.1.6.
Steinar H. Gunderson [Fri, 2 Apr 2021 11:12:49 +0000 (13:12 +0200)]
Randomize updatedb systemd unit start time.
Suggested by Calum McConnell.
Steinar H. Gunderson [Sat, 27 Mar 2021 23:19:49 +0000 (00:19 +0100)]
Support searching multiple databases in plocate.
We support multiple -d arguments, single -d arguments with multiple
databases (colon-separated), and LOCATE_PATH. The two latter should
be compatible with mlocate, although we support escaping colons in
file paths and I believe mlocate does not.
This is slightly trickier than one might expect due to security
considerations; we're not robust against malicious input, so we need to
make sure that if we process an attacker-supplied database, the process
has already dropped privileges and cannot subvert a privileged reader.
Steinar H. Gunderson [Sun, 21 Mar 2021 14:48:55 +0000 (15:48 +0100)]
Make updatedb understand DT_UNKNOWN.
Some filesystems don't know from getdents() whether an entry is a file
or a directory without a stat(). I had assumed this was only an issue
for obscure operating systems, so I removed it (mlocate's updatedb
supported it), but evidently older versions of XFS has this issue, too,
so add back checking.
Reported by Marcel Partap.
Steinar H. Gunderson [Sun, 21 Mar 2021 12:11:45 +0000 (13:11 +0100)]
Give a slightly better error message on short read.
Steinar H. Gunderson [Sun, 21 Mar 2021 12:11:35 +0000 (13:11 +0100)]
Bump version number.
Steinar H. Gunderson [Wed, 24 Feb 2021 18:14:11 +0000 (19:14 +0100)]
Release plocate 1.1.5.
Steinar H. Gunderson [Fri, 19 Feb 2021 17:54:35 +0000 (18:54 +0100)]
Reduce the amount of lseek() syscalls; evidently, ftell() does not cache the position.
Steinar H. Gunderson [Fri, 19 Feb 2021 17:42:12 +0000 (18:42 +0100)]
Support filesystems that do not support O_TMPFILE, even on Linux.
Evidently, overlayfs does not support O_TMPFILE. Reported by Daniel Lewart.
Steinar H. Gunderson [Fri, 19 Feb 2021 17:38:33 +0000 (18:38 +0100)]
Bump version number.
Steinar H. Gunderson [Sun, 14 Feb 2021 09:56:16 +0000 (10:56 +0100)]
Release plocate 1.1.4.
Steinar H. Gunderson [Sat, 13 Feb 2021 10:06:44 +0000 (11:06 +0100)]
Call strlen() a bit less. Helps ~2% CPU.
Steinar H. Gunderson [Sat, 13 Feb 2021 00:58:31 +0000 (01:58 +0100)]
Microoptimizations to docid counting.
Steinar H. Gunderson [Sat, 13 Feb 2021 00:41:04 +0000 (01:41 +0100)]
Take an assert off the hot path.
Seemingly, Meson doesn't include -DNDEBUG by default, and this assert
is pretty hot, to take it out to be sure.
Steinar H. Gunderson [Sat, 13 Feb 2021 00:30:27 +0000 (01:30 +0100)]
Fix a typo.
Steinar H. Gunderson [Sat, 13 Feb 2021 00:24:58 +0000 (01:24 +0100)]
Switch to histogram-based estimation of PFOR varbyte sizes.
This makes plocate.db ~0.1% larger, but reduces CPU usage by something
like 3%, so it's an OK tradeoff.
Steinar H. Gunderson [Fri, 12 Feb 2021 20:15:24 +0000 (21:15 +0100)]
Optimize trigram reading somewhat.
Helps 1–2% CPU.
Steinar H. Gunderson [Thu, 11 Feb 2021 09:09:38 +0000 (10:09 +0100)]
Replace std::string with std::vector<unsigned char> when encoding posting lists.
This is a ~0.5–1% win since we never use the short-string optimization,
and the vector fits better in cache.
Steinar H. Gunderson [Thu, 11 Feb 2021 08:45:30 +0000 (09:45 +0100)]
Streamline the add_docid() path by removing a redundant comparison. Saves ~1% CPU.
Steinar H. Gunderson [Thu, 11 Feb 2021 08:37:31 +0000 (09:37 +0100)]
Remove an unused member. Saves ~0.5% CPU due to better cache efficiency.
Steinar H. Gunderson [Thu, 11 Feb 2021 08:35:17 +0000 (09:35 +0100)]
Inline EncodingCorpus::get_pl_builder(). Saves ~2% CPU.
Steinar H. Gunderson [Thu, 11 Feb 2021 08:32:16 +0000 (09:32 +0100)]
Rename Corpus to EncodingCorpus, and make it private.
We don't really want two different classes named Corpus; that's pretty
confusing, even though they don't live in the same binary.
Steinar H. Gunderson [Wed, 10 Feb 2021 20:39:37 +0000 (21:39 +0100)]
Install CACHEDIR.tag in /var/lib/plocate.
This helps certain tools understand that everything in the directory
is autogenerated. Suggested by Marco d'Itri.
Jakub Wilk [Wed, 10 Feb 2021 08:13:29 +0000 (09:13 +0100)]
updatedb.8: fix typo.
Jakub Wilk [Wed, 10 Feb 2021 08:13:28 +0000 (09:13 +0100)]
plocate.1: remove spurious space after "(".
Steinar H. Gunderson [Wed, 10 Feb 2021 09:37:44 +0000 (10:37 +0100)]
Bump version number.
Steinar H. Gunderson [Sat, 19 Dec 2020 00:17:59 +0000 (01:17 +0100)]
Release plocate 1.1.3.
Steinar H. Gunderson [Sat, 12 Dec 2020 11:56:46 +0000 (12:56 +0100)]
Compile on systems without O_TMPFILE.
This is a last-resort solution; we don't do unlink-on-signal or similar,
so if updatedb or plocate-build is aborted on such platforms, there
will be an orphan temporary file.
Steinar H. Gunderson [Sat, 12 Dec 2020 11:54:31 +0000 (12:54 +0100)]
Compile on platforms without O_NOATIME.
Steinar H. Gunderson [Sat, 12 Dec 2020 11:43:28 +0000 (12:43 +0100)]
Fix a Clang warning.
Steinar H. Gunderson [Sat, 12 Dec 2020 11:53:45 +0000 (12:53 +0100)]
Bump version number.
Steinar H. Gunderson [Wed, 9 Dec 2020 23:40:03 +0000 (00:40 +0100)]
Release plocate 1.1.2.
Steinar H. Gunderson [Wed, 9 Dec 2020 23:37:41 +0000 (00:37 +0100)]
Increase hard file limit if possible.
Fixes issues where updatedb was run with a too-low open file limit
(both soft and hard), but by root, so we could just raise both limits.
Reported by Gregor Herrmann.
Steinar H. Gunderson [Wed, 9 Dec 2020 23:30:04 +0000 (00:30 +0100)]
Make it possible to rename updatedb (e.g. to updatedb.plocate).
Steinar H. Gunderson [Wed, 9 Dec 2020 23:24:42 +0000 (00:24 +0100)]
Fix a typo in plocate(1).
Steinar H. Gunderson [Wed, 9 Dec 2020 23:23:51 +0000 (00:23 +0100)]
Make updatedb(8) stop referring to mlocate (except in one deliberate place).
Steinar H. Gunderson [Mon, 7 Dec 2020 09:22:52 +0000 (10:22 +0100)]
Turn off ProtectSystem=full.
It turns out this works by bind mounts, which we ignore by default,
causing /usr to be ignored in the database.
Steinar H. Gunderson [Mon, 7 Dec 2020 09:22:39 +0000 (10:22 +0100)]
Fix an issue where the database was too small to create a proper zstd dictionary.
Steinar H. Gunderson [Sat, 5 Dec 2020 09:51:52 +0000 (10:51 +0100)]
Release plocate 1.1.1.
Steinar H. Gunderson [Sat, 5 Dec 2020 09:49:55 +0000 (10:49 +0100)]
Fix an issue where the database could be built with the wrong check_visibility flag.
The check_visibility flag would never be set in the header, and thus be set to some
random variable instead of what the user wanted.
Steinar H. Gunderson [Wed, 2 Dec 2020 22:39:49 +0000 (23:39 +0100)]
Release plocate 1.1.0.
Steinar H. Gunderson [Wed, 2 Dec 2020 22:51:55 +0000 (23:51 +0100)]
Add missing man page .in files.
Steinar H. Gunderson [Tue, 1 Dec 2020 23:34:16 +0000 (00:34 +0100)]
Import the updatedb and updatedb.conf man pages from mlocate.
Steinar H. Gunderson [Sun, 29 Nov 2020 18:30:03 +0000 (19:30 +0100)]
Fix installation when DESTDIR is set.
Steinar H. Gunderson [Sun, 29 Nov 2020 14:07:37 +0000 (15:07 +0100)]
Reuse the locategroup option for the setgid.
Steinar H. Gunderson [Sat, 28 Nov 2020 22:21:06 +0000 (23:21 +0100)]
Small README tweak.
Steinar H. Gunderson [Sat, 28 Nov 2020 22:16:36 +0000 (23:16 +0100)]
In the README, make the group before we install.
Steinar H. Gunderson [Sat, 28 Nov 2020 22:14:45 +0000 (23:14 +0100)]
mlocate -> plocate in some help.
Steinar H. Gunderson [Sat, 28 Nov 2020 22:09:13 +0000 (23:09 +0100)]
setgid to plocate, not mlocate.
Steinar H. Gunderson [Sat, 28 Nov 2020 22:08:23 +0000 (23:08 +0100)]
Add a systemd service and timer for updatedb, obsoleting the cron entry.
Steinar H. Gunderson [Sat, 28 Nov 2020 20:59:21 +0000 (21:59 +0100)]
Fix verbose printing when updatedb reads directories from an old database.
Steinar H. Gunderson [Sat, 28 Nov 2020 20:56:54 +0000 (21:56 +0100)]
Fix writing of the configuration block.
Steinar H. Gunderson [Sat, 28 Nov 2020 20:46:00 +0000 (21:46 +0100)]
Create /var/lib/plocate on install.
Steinar H. Gunderson [Sat, 28 Nov 2020 20:05:56 +0000 (21:05 +0100)]
Put the database file in sharedstatedir.
Steinar H. Gunderson [Sat, 28 Nov 2020 20:05:27 +0000 (21:05 +0100)]
Use join_paths() in Meson.
Steinar H. Gunderson [Sat, 28 Nov 2020 19:57:03 +0000 (20:57 +0100)]
Move to our own directory and group.
Steinar H. Gunderson [Sat, 28 Nov 2020 19:55:25 +0000 (20:55 +0100)]
Update README.
Steinar H. Gunderson [Sat, 28 Nov 2020 17:17:23 +0000 (18:17 +0100)]
Honor the “require visibility” flag (in the negative).
Steinar H. Gunderson [Wed, 25 Nov 2020 00:02:25 +0000 (01:02 +0100)]
Run clang-format.
Steinar H. Gunderson [Sat, 21 Nov 2020 17:23:20 +0000 (18:23 +0100)]
Add a native updatedb.
This incorporates some code from mlocate's updatedb, and thus is compatible
with /etc/updatedb.conf, and supports all the pruning options from it.
All the code has been heavily modified, e.g. the gnulib dependency has been
removed and replaced with STL code (kicking 10k+ lines of code), the bind
mount code has been fixed (it was all broken since the switch from /etc/mtab
to /proc/self/mountinfo) and everything has been reformatted. Like with mlocate,
plocate's updatedb is merging, ie., it can skip readdir() on unchanged
directories. (The logic here is also copied pretty verbatim from mlocate.)
updatedb reads plocate's native format; there's a new max_version 2 that
contains directory timestamps (without it, updatedb will fall back to a full
scan). The timestamps increase the database size by only about 1%, which is a
good tradeoff when we're getting rid of the entire mlocate database.
We liberally use modern features to simplify the implementation; in particular,
openat() to avoid race conditions, instead of mlocate's complicated chdir() dance.
Unfortunately, the combination of the slightly strange storage order from mlocate,
and openat(), means we can need to keep up a bunch of file descriptors open,
but they are not an expensive resource these days, and we try to bump the
limit ourselves if we are allowed to. We also use O_TMPFILE, to make sure we
never leave a half-finished file lying around (mlocate's updatedb tries to
catch signals instead). All of this may hinder portability, so we might ease up
on the requirements later. We don't use io_uring for updatedb at this point.
plocate-build does not write the needed timestamps, so the first upgrade from
mlocate to native plocate requires a full rescan.
NOTE: The format is _not_ frozen yet, and won't be until actual release.
Steinar H. Gunderson [Sun, 22 Nov 2020 20:28:57 +0000 (21:28 +0100)]
Move complete_pread() into a separate file, to avoid the io_uring false dependency.
Steinar H. Gunderson [Sat, 21 Nov 2020 14:34:59 +0000 (15:34 +0100)]
Make DatabaseBuilder write the file atomically.
By opening with O_TMPFILE, we guarantee we'll never be leaving
an unfinished file visible on the filesystem. The move across the
old one isn't atomic, but the window of failure is very small now.
Steinar H. Gunderson [Tue, 10 Nov 2020 18:01:48 +0000 (19:01 +0100)]
Remove unfinished debug code.
Steinar H. Gunderson [Tue, 10 Nov 2020 00:09:31 +0000 (01:09 +0100)]
Split DatabaseBuilder into its own compilation unit.
Steinar H. Gunderson [Mon, 9 Nov 2020 23:19:58 +0000 (00:19 +0100)]
When reading mlocate.db, properly skip the configuration block.
This could cause some entries to be skipped until we regained sync,
especially in the root directory.
Steinar H. Gunderson [Mon, 9 Nov 2020 22:50:20 +0000 (23:50 +0100)]
Encapsulate some database-building logic into a class.
Steinar H. Gunderson [Sat, 7 Nov 2020 10:22:52 +0000 (11:22 +0100)]
Escape file names with backticks in them.
Steinar H. Gunderson [Sat, 7 Nov 2020 10:16:46 +0000 (11:16 +0100)]
Bump version number.