Reinstate the splice read/write/move paths removed in 84592a9f, now
gated behind runtime options with negotiated pipe sizing and fallback
logging.
- Transport: msgbuf-piped splice receive, fd-referenced splice reply
(vmsplice + splice), splice-move pipe-bounce, and restored
fuse_buf_copy/fuse_buf_flags plumbing for copy fallbacks
- Negotiation: new pipe-max-size option (auto-raised at INIT when
possible; the sysctl read/write is factored into src/pipe_max_size.cpp
so fuse_init.cpp and option_parser.cpp share one implementation);
fuse-msg-size clamped to what the pipe can hold; splice is disabled
at INIT with a warning if the sysctl can't fit even the minimum,
retracting the FUSE_CAP_SPLICE_* bits already advertised to the
kernel in that case
- Read replies and splice-move are gated at 128 KiB (below that the
syscalls cost more than the copy they avoid); every fallback path
degrades to the copy path and logs once per process via SysLog
- msgbuf pipe cache is process-global and mutex-guarded (mutex_lockguard)
so pipes can move between read and worker threads under
process-thread mode
- API simplified after review: collapsed the three per-axis options
(splice-read/write/move) into a single splice boolean (default on);
no-splice-read/write/move and the bare splice-read/write/move
spellings stay registered as accepted no-ops so existing mount
options don't fail the mount, but they do NOT alter the splice
knob - upgraders with no-splice-* in their config get splice
enabled and must set splice=false explicitly (see
deprecated_options.md)
- Fixed several correctness issues found in parallel review: a
dropped write_buf registration that silently corrupted writes, a
dirty-pipe reuse bug in msgbuf_free, an iov double-send in
fuse_reply_data's splice-move fallback, and pipe-capacity math that
assumed a 4096-byte page size
- Modernized the transport/negotiation code to this codebase's C++
style: RAII (unique_ptr/DEFER) instead of malloc/free and manual fd
close(), std::from_chars/to_chars instead of stoul/snprintf,
_ReplyPipe's pipe ops as methods instead of free functions closing
over a thread_local global, and vendored/libfuse/lib/buffer.cpp (a
near-verbatim port of upstream's buffer.c) brought in line with the
rest of the fork
- Documented splice and pipe-max-size options (mkdocs, FAQ,
deprecated-options list); added tools/bench-splice.sh
- Benchmarked on tmpfs: seq_write +21%, seq_read +52%, small_write
+5%, small_read +12%; validated with a 1h soak (~271 GiB
hash-verified I/O, zero integrity failures), a live-mount round-trip
of I/O above the splice threshold, and the existing unit/smoke
suites at their pre-existing baseline
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Store cloned read-worker fds on fuse_session and close them there.
- Set up clone fallback through session helpers used by the loop.
- Carry per-request fds through receive, process, and reply paths.
Tests show significant performance increases in highly concurrent
workloads on a high core count system. Upto 15% better on reads and
27% on writes.
- Expose the ioprio enabled flag as an atomic for relaxed inline checks
- Inline SetFrom construction so disabled read/write paths avoid a function call
- Move priority application into a slow helper used only when ioprio is enabled
- Guard fuse_cfg_t log file and filepath shared pointers with a shared mutex
- Replace atomic shared pointer loads and stores with shared and unique locks
- Replace atomic shared pointer loads/stores with shared/unique locking.
- Store the configured readdir mode directly as std::string.
- Copy the active ReadDir implementation under lock before invoking it.
- Use relaxed increment under the map bucket lock when duplicating file handles.
- Use acq_rel decrement when releasing and checking for final references.
- Load moved ref_count with relaxed ordering during map insertion.
- Explain that the mutex is per readdir call.
- Document that only dirent deduplication and list append are serialized.
- Clarify that per-branch opendir and getdents64 work runs outside the critical section.
- Replace the per-file Mutex with std::shared_mutex for shared write access.
- Run direct and cached writes under a shared lock while the fd is stable.
- Retry failed writes under a unique lock before move-on-ENOSPC relocation.
- Add helpers that release path locks and update node state under f.lock.
- Update getattr and setattr stat-cache invalidation while preserving ESTALE handling.
- Rework create, open, tmpfile, and set_path_info to mutate lookup/open state under the shared lock.
- Document the lock/unlock pair as an acquire/release barrier.
- Explain that releasedir waits for in-flight readdir calls before destroy/free.
- Warn against removing the synchronization despite it not being a critical section.
Important changes:
- Switch mergerfs and vendored libfuse Makefiles to -std=c++20.
- Select the latest available GCC/G++ or toolset in container builds.
- Verify compiler C++20 support during package installation.
- Fix dynamic SysLog formatting for fmt C++20 checks.
- Add mergerfs_mount() context manager in posix_parity.py that creates
temporary branch dirs in tests/.test_tmp/, mounts an ephemeral mergerfs
instance, and cleans up after use.
- Add find_mergerfs(), find_fusermount(), mount_mergerfs(), unmount_mergerfs()
helpers.
- Update run-tests to invoke each TEST_* script without a mountpoint arg.
- Update README.md to document the new `python3 tests/run-tests` invocation.
- Fix 44 tests to use mergerfs_mount() instead of expecting a pre-mounted path.
- Add tempfile import to 10 tests that were missing it.
- Fix TEST_mount_lifecycle to use mergerfs_mount() context manager.
- Fix TEST_posix_tmpfile missing join import.
- Fix TEST_cfg_xattr_modes error message framing (no "(known bug)").
- Fix TEST_posix_copy_file_range: loop-copy full size, compare st_size and
SHA256 hash; add NativeTmp for space management; route native temp to
tests/.test_tmp/ to avoid ENOSPC on /tmp.
- Fix TEST_posix_getattr_fgetattr: st_cmp skips st_size for dirs; add
compare_ino param for safe inode comparison when inodecalc=passthrough.
- Fix TEST_unlink_rename, TEST_use_* tests: namespace cleanup, consistent
variable naming, None check for cmpfunc.
- Fix TEST_readlink_semantics: remove type annotations to support older
Python; wrap in mergerfs_mount() try/except.
- Add flush=True to run-tests print so test names show immediately.
- Fix typo in static.installer: /pkg/ -> /pkgs/
- Add missing mkdir -p /build before makeself
- Mount pkgs directory via podman build -v so static.installer can access
previously built static tarballs without relying on bind mounts relative
to the build context
- Register qemu-user-static binfmt handlers for cross-compilation
- Make --git-repo optional when using --setup
- Reserve a zero stat CRC to represent an invalid cache fingerprint.
- Include device, size, inode, mtime, and ctime data in the fingerprint.
- Recompute the fingerprint during auto-cache open before keeping cache.
- Make the test runner fail on failed tests and report explicit skips
- Strengthen assertions for O_DIRECT, LUP policy, EXDEV fallback, and xattrs
- Fix cleanup and file descriptor handling across test scripts
No functional change: libfuse/helper.cpp (fuse_daemonize,
fuse_helper_opt_proc, fuse_parse_cmdline) and mergerfs
fuse_init.cpp (_want helpers) reformatted to match the project's
GNU-style brace placement and aligned parameter lists.
- fuse_opt: reformat process_opt_param to project style; return the
rv from sscanf through a named local so the error branch prints
the failing option cleanly.
- cpu.cpp: when parsing /sys/devices/system/cpu/*/topology/core_id,
bail out cleanly on stream read failure rather than recording an
uninitialized core_id.
- fusermount: free(copy) on the success path of parsing the mount
option (previously leaked on every mount), and replace the fixed-
buffer sprintf into procfile with snprintf bounded by the buffer
size.
- maintenance_thread: widen the iteration count from int to u64
so a long-lived daemon doesn't wrap after ~68 years of ticks
(real effect: silence signed-overflow warnings under UBSan).
- ThreadPool::destroy used to iterate _threads directly while another
thread could be adjusting the same vector. Snapshot the vector
under the mutex and iterate the local copy for pthread_cancel /
pthread_join.
- ObjPool::construct now returns NULL when the pooled allocator
returns NULL instead of invoking placement-new on a null
pointer. In destroy, compute to_node(obj) before running the
destructor - the destructor may scrub bytes the to_node helper
reads.
- In the async/sync workers in fuse_loop, msgbuf_alloc can return
NULL under memory pressure. Fail the worker cleanly with a
diagnostic rather than dereferencing. Also introduce a small
_nproc() helper that treats hardware_concurrency()==0 as 1, so
the downstream min/max arithmetic doesn't produce a zero thread
count.
The signal handler reads the global session pointer from an async-
signal context while another thread installs/removes the session
via set/remove_signal_handlers. Use std::atomic<fuse_session*> with
acquire/release ordering so the handler either sees NULL or a fully
published session, never a torn pointer.
Collapse get_node/get_node_nocheck into a single function that
returns NULL when the node is absent rather than abort()ing.
Callers that previously assumed non-NULL now check the return value
and either reply ESTALE/ENOMEM or fall through gracefully; a few
places where the node is load-bearing (e.g. unhash during remove)
use fatal::abort with a descriptive message.
Switch mutex_lock / mutex_unlock pairs to mutex_lockguard so early
returns on error paths no longer leak the lock, and add NULL-safety
around fuse_dh in readdir/readdir_plus, fuse_dirents_init error
returns, the getattr path when both name and fh are empty, and the
msgbuf allocation in fuse_lib_read.
Pull the new xstrdup/xmalloc wrappers from xalloc.hpp for the root
node name so the common-case OOM path terminates with a diagnostic
instead of a silent corrupted node.
Also guard fuse_session_destroy against a NULL session - called
from error-recovery paths.
The low-level init/destroy callbacks were passed a userdata pointer
that mergerfs never consumed - the caller threaded it through
fuse_lowlevel_new only to hand it back unchanged. Remove the
parameter from the ops struct, from fuse_lowlevel_new(_common),
and from the internal fuse_ll struct; update fuse_lib_init and
fuse_lib_destroy accordingly.
Also drop fuse_reply_ioctl_iov: no caller in-tree invokes it and
the public API is defined by this vendored libfuse fork.
