Brian Foster [Mon, 27 Nov 2023 18:31:10 +0000 (13:31 -0500)]
bcachefs-tools: don't skip stale superblock wipe in force mode
Even though the blkid wipe has been fixed to detect and wipe old
superblocks, we still incorrectly skip the wipe when in force mode.
Update the force logic in open_for_format() to bypass the user
request and otherwise proceed with the wipe.
Signed-off-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
This fixes the following build failure on Debian bookworm:
error: failed to run custom build command for `clang-sys v1.6.1`
Caused by:
process didn't exit successfully: `/home/minoru/src/bcachefs-tools/rust-src/target/release/build/clang-sys-df95f6d1266be773/build-script-build` (exit status: 101)
--- stdout
cargo:warning=could not execute `llvm-config` one or more times, if the LLVM_CONFIG_PATH environment variable is set to a full path to valid `llvm-config` executable it will be used to try to find an instance of `libclang` on your system: "couldn't execute `llvm-config --prefix` (path=llvm-config) (error: No such file or directory (os error 2))"
--- stderr
thread 'main' panicked at /home/minoru/.cargo/registry/src/index.crates.io-6f17d22bba15001f/clang-sys-1.6.1/build/dynamic.rs:206:45:
called `Result::unwrap()` on an `Err` value: "couldn't find any valid shared libraries matching: ['libclang.so', 'libclang-*.so'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Daniel Hill [Mon, 20 Nov 2023 12:43:52 +0000 (01:43 +1300)]
improve kmalloc performance
Reading from /proc/meminfo is really slow
We don't want to start swapping to disk.
Deceptively, memory available goes up when we start to swap to disk making
performance even worse.
To mitigate this:
1. replace reading from meminfo with proper system calls.
2. attempt to lock allocations in physical memory space.
3. check our own allocated memory instead of available memory.
4. still check available memory in the off chance we're trying to play
nice with other apps.
Signed-off-by: Daniel Hill <daniel@gluo.nz> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
When playing around with comparing some fstests results on different
filesystems, I noticed that a 'bcachefs format' of a previously
btrfs-formatted device still continued to mount as btrfs. The reason
for this is that the blkid wipe invoked via open_for_format() is not
working correctly. blkid_do_wipe() depends on the "SBMAGIC[_OFFSET]"
values to do any work, and the associated superblock magic flag is
not enabled on the probe.
Set the probe flags to explicitly enable the values the bcachefs
code depends on in the probe. This includes the type, label and
superblock magic information.
There are also a couple quirks in the libblkid code that might be
worth noting. One is that the superblock enablement and flag setting
functions appear hardcoded to return zero, so we just combine the
error checks. Second, while blkid_do_wipe() can return an error, it
actually doesn't in the scenario being addressed here because it
doesn't seem to distinguish between the values being absent because
nothing was found by the probe or because the values weren't enabled
in the first place. Regardless, add an error check here in the event
the wipe does explicitly fail for some unexpected reason.
Signed-off-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
prior to this patch, on certain platforms (ie: armv7l), compilation fails due
to atomic64_sub_return_release not being defined here. Ensure that the atomics
header is pulled in, and ensure that it is available in all cases, regardless
of whether ATOMIC64_SPINLOCK is defined.
Signed-off-by: Nicholas Sielicki <linux@opensource.nslick.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
libc proper treats mount flags as an unsigned long, which is usually u64,
except when it isn't. When preparing mount flags, use the libc::c_ulong type
instead of u64 to allow for this.
This fixes compiling this file under armv7l.
Signed-off-by: Nicholas Sielicki <linux@opensource.nslick.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Tue, 7 Nov 2023 01:47:48 +0000 (20:47 -0500)]
Avoiding picking different bucket sizes for different devices
Erasure coding can't create stripes across devices with different bucket
sizes - therefore, format shouldn't pick different bucket sizes for
different devices.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Tue, 7 Nov 2023 00:01:15 +0000 (19:01 -0500)]
cmd_format: Check for device options after device arguments
It's a common user error to specify device specific options at the end
of a format command, and then not have them apply to any devices - add a
check for this.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Brian Foster [Tue, 19 Sep 2023 14:26:11 +0000 (10:26 -0400)]
bcachefs-tools: fix endian problems between bit spinlocks and futexes
bcachefs format on a big-endian (s390x) machine crashes down in the
rhashtable code imported from the kernel. The reason this occurs
lies within the rht_lock() -> bit_spin_lock() code, the latter of
which casts bitmaps down to 32-bits to satisfy the requirements of
the futex interface.
The specific problem here is that a 64 -> 32 bit cast doesn't refer
to the lower 8 bytes on a big endian machine, which means setting
bit number 0 in the 32-bit map actually corresponds to bit 32 in the
64-bit map. The rhashtable code specifically uses bit zero of the
bucket pointer for exclusion and uses native bitops elsewhere (i.e.
__rht_ptr()) to identify NULL pointers. If bit 32 of the pointer is
set by the locking code instead of bit 0, an otherwise NULL pointer
looks like a non-NULL value and results in a segfault.
The bit spinlock code imported by the kernel is originally intended
to work with unsigned long. The kernel code is able to throttle the
cpu directly when under lock contention, while the userspace
implementation relies on the futex primitives to emulate reliable
blocking. Since the futex interface introduces the 32-bit
requirement, isolate the associated userspace hack to that
particular code.
Restore the native bitmap behavior of the bit spinlock code to
address the rhashtable problem described above. Since this is not
compatible with the futex interface, create a futex wrapper
specifically to convert the native bitmap type to a 32-bit virtual
address and mask value for the purposes of waiting/waking when under
lock contention.
Signed-off-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Brian Foster [Fri, 15 Sep 2023 13:40:25 +0000 (09:40 -0400)]
libbcachefs: convert sb features mask to little endian
bch_sb.features is an array of __le64. Convert the native endian format
of the features mask appropriately. This causes a bcachefs format to
produce an unmountable fs when run from a big endian system.
Signed-off-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
The comman line help claims that `bcachefs mount <DEV>` without a
mount point will do a dry-run mount - all the steps required to
mount the fs, but without actually doing the final real mount.
Make the code actually do this, rather than complain that you
haven't supplied a mountpoint if you don't provide a mountpoint
Signed-off-by: Christopher James Halse Rogers <raof@ubuntu.com>
Colin Gillespie [Sat, 12 Aug 2023 03:50:37 +0000 (13:50 +1000)]
cmd_mount: use the correct keyring key type
Mount is checking for and adding encryption keys using the logon key
type instead of the user key type. This was causing it to not be able to
unlock volumes on its own, and ask for a passphrase on already unlocked
volumes.
Signed-off-by: Colin Gillespie <colin@cgillespie.xyz>
projects / bcachefs-tools-debian / log