std::filesystem::path is wrong for mergerfs's hot path:
- `branch->path / fusepath_` heap-allocates per FUSE op (3 allocs each
on libstdc++; the SSO buffer is 15 bytes and any joined path exceeds it).
- The implicit `operator string_type()` adds another std::string when
passing to `fs::*` wrappers that take `const std::string&`.
- Per-branch loops rebuild fullpath from scratch every iteration.
The new fs::relpath is a fixed 8192-byte (2 * PATH_MAX) inline buffer,
NUL-terminated, never heap-allocates. The rel data is anchored at offset
REL_OFFSET = 4096; the 4096 bytes before the anchor form the "prefix
area" so a long base/prefix can be written there in place via
set_prefix() without disturbing the rel. The remaining ~4096 bytes
after the rel are available for trailing append (filenames in
readdir-style loops, etc.).
set_prefix's cost is O(prefix size) -- only the actually-used bytes
are touched -- so the buffer size is a stack-footprint question, not a
hot-path cost. FUSE worker threads use the pthread default stack
(~8 MB on Linux); a 4-relpath frame in fuse_link/fuse_rename is 32 KB,
~0.4% of the stack budget.
The per-branch hot loop:
fs::relpath fullpath = fusepath_; // one copy of the rel
for (auto branch : branches) {
fullpath.set_prefix(branch->path); // writes only [start, 4096)
fs::syscall(fullpath);
}
set_prefix only rewrites the prefix area; the rel stays anchored. A
cached _rel_size means no recomputation across iterations.
Microbenchmark (8 branches, 30-char fusepath, -O3, 4 M path builds):
variant ns/build allocs/build
-------------------------------------- -------- ------------
origin/master (std::filesystem::path) 158 3.00
fs::relpath assign_concat (single-shot) 9.1 0
fs::relpath set_prefix (loop-optimized) 4.5 0
35x faster, zero allocations in the hot path.
Contract: paths flow through fs::relpath in canonical form -- prefix
without trailing '/', rel without leading '/'. No sanitization;
violations yield double slashes. Mergerfs satisfies this by
construction:
- libfuse's try_get_path() walks leaf-to-root via prepend(), then
calls consolidate_to_rel(strip=true) which drops the leading '/'
that prepend produces.
- branch paths are realpath-canonicalized at config load so they
never have a trailing slash.
Mergerfs has exactly one path-manipulation type: a relative fusepath.
Branch paths and other absolute paths are stored as std::string -- they
are spliced into the prefix area of an fs::relpath at syscall time but
are never themselves mutated as path objects.
Major changes:
- vendored/libfuse/{include/fs_path.hpp,lib/fs_path.cpp}: new
fs::relpath class (8 KB inline, REL anchor at 4096, set_prefix /
clear_prefix / consolidate_to_rel, _rel_size cache, self-aliasing-
safe assign_concat, sticky ENAMETOOLONG via errnum()).
- 30+ FUSE-op call sites converted from `fullpath = branch / fusepath`
to the `set_prefix` loop pattern (fuse_unlink, fuse_chmod,
fuse_chown, fuse_listxattr, fuse_truncate, fuse_utimens,
fuse_setxattr, fuse_rmdir, fuse_removexattr, fuse_link's
preserve-path and create-path loops, fuse_rename's create-path
and preserve-path loops, fuse_statfs FULL mode, fs::findallfiles,
fs::findonfs).
- fs::* syscall wrappers that previously only took
`const std::string&` got `const char*` overloads so fs::relpath's
implicit `operator const char*()` lands cleanly (fs::link,
fs::lutimens, fs::umount_lazy, fs::open_dir_ro, fs::readlink,
fs::lremovexattr, fs::lstatvfs, fs::info, fs::statvfs_cache).
- fs::exists gained a (fs::relpath&, basepath, ...) overload that
calls set_prefix internally so per-branch policy fan-out loops
share a single fullpath, rewriting only the prefix area on each
iteration.
- All policies (epall, epff, eplfs, eplus, epmfs, eppfrd, ff, lup,
msplfs, msplus, mspmfs, msppfrd, newest) refactored to hoist the
fs::relpath fullpath construction out of the per-branch loop and
use the set_prefix-based fs::exists overload.
- fs::inode::ReaddirCalc::_fusepath migrated from std::string to
fs::relpath with the same resize/append API.
- std::filesystem removed entirely from the project: branches.cpp
and option_parser.cpp's exists/is_directory/equivalent calls now
use direct ::stat(); mergerfs_fsck.cpp's recursive_directory_iterator
was replaced with an opendir-based walk.
Tests (tests/tests.cpp): 27 new fs::relpath test cases covering
construction, set_prefix in/out of REL anchor, prefix()/rel()
accessors, clear_prefix, concat canonical form, self-aliasing
(`p /= "x"`), operator/=, resize zero-fill, swap, copy/move,
equality, implicit conversions, lexically_relative, remove_filename,
_rel_size invariant across every mutator, capacity bounds.
Full suite: 213/213 pass.
- Add mergerfs_mount() context manager in posix_parity.py that creates
temporary branch dirs in tests/.test_tmp/, mounts an ephemeral mergerfs
instance, and cleans up after use.
- Add find_mergerfs(), find_fusermount(), mount_mergerfs(), unmount_mergerfs()
helpers.
- Update run-tests to invoke each TEST_* script without a mountpoint arg.
- Update README.md to document the new `python3 tests/run-tests` invocation.
- Fix 44 tests to use mergerfs_mount() instead of expecting a pre-mounted path.
- Add tempfile import to 10 tests that were missing it.
- Fix TEST_mount_lifecycle to use mergerfs_mount() context manager.
- Fix TEST_posix_tmpfile missing join import.
- Fix TEST_cfg_xattr_modes error message framing (no "(known bug)").
- Fix TEST_posix_copy_file_range: loop-copy full size, compare st_size and
SHA256 hash; add NativeTmp for space management; route native temp to
tests/.test_tmp/ to avoid ENOSPC on /tmp.
- Fix TEST_posix_getattr_fgetattr: st_cmp skips st_size for dirs; add
compare_ino param for safe inode comparison when inodecalc=passthrough.
- Fix TEST_unlink_rename, TEST_use_* tests: namespace cleanup, consistent
variable naming, None check for cmpfunc.
- Fix TEST_readlink_semantics: remove type annotations to support older
Python; wrap in mergerfs_mount() try/except.
- Add flush=True to run-tests print so test names show immediately.
- Make the test runner fail on failed tests and report explicit skips
- Strengthen assertions for O_DIRECT, LUP policy, EXDEV fallback, and xattrs
- Fix cleanup and file descriptor handling across test scripts
Antonio S. Musumeci