A log-structuredfilesystem for S3.
ZeroFS serves S3-compatible buckets as POSIX filesystems over NFS and 9P, and as raw block devices over NBD. All three servers run in one userspace process. Writes go out as immutable segments, compressed and encrypted before upload. Warm reads come from local cache.
or via Docker · GitHub Action · other platforms
Files over NFS and 9P, block devices over NBD.
The three servers share one process and one bucket. Clients connect with the NFS and 9P
support already in the kernel, ZeroFS' own client or nbd-client.
NFS
macOS, Linux, Windows, and the BSDs mount it with the NFS clients they ship. NFS COMMIT semantics let fsync return before data reaches S3; use a 9P mount where fsync durability matters.
# mounts from any major OS
mount -t nfs 127.0.0.1:/ /mnt/zerofs
9P
fsync returns only after data reaches stable storage. The bundled FUSE client, zerofs mount,
mounts without root and reconnects on its own; it is the recommended Linux mount.
# bundled FUSE client, no root
zerofs mount 127.0.0.1:5564 /mnt/zerofs
NBD
Device files in a .nbd directory attach as raw block devices; ZFS pools and VM boot disks
run on them. FLUSH and FUA replies return only after data is durable, so write barriers hold.
# attach a block device
nbd-client 127.0.0.1 10809 /dev/nbd0 -N vol1
How ZeroFS is tested.
POSIX semantics
8,662 pjdfstest cases: permissions, ownership, links, rename behavior. A few cases per protocol are excluded where NFS or 9P can't express the semantics (or where we decided to make tradeoffs).
pjdfstest workflow →xfstests
A standard filesystems test suite.
xfstests workflow →Kernel builds
The Linux kernel compiles with make -j$(nproc) on NFS, 9P, and FUSE mounts. A parallel build
keeps many processes writing the same tree at once.
stress-ng
file-handling stressors run concurrently against live NFS, 9P, and FUSE mounts: dentry churn, deep and wide directories, rename, metamix, utime, and more.
stress-ng workflow →ZFS pool and scrub
CI creates a ZFS pool on ZeroFS block devices, extracts the Linux kernel source onto it, and runs a scrub.
zfs-test workflow →Jepsen local-fs and HA
The local-fs suite drives a 9P mount with random operation histories checked against a reference model, with a crash mode. The HA suite kills and pauses a leader/standby pair over a local S3 server.
jepsen workflows →A ZFS mirror across three S3 regions.
Each instance exposes one region's bucket as a block device; to ZFS they are plain disks.
When a region is unreachable, the pool degrades and stays online. ZFS resilvers the member when it returns.
The storage engine.
Immutable segments
File data is split into 32 KiB extents and packed into immutable segment objects of up to 256 MiB.
Encryption
XChaCha20-Poly1305 on every extent before upload. The data key is wrapped by a key derived from the password with Argon2id.
Compression
zstd or lz4, applied before encryption. The codec can change at any time without migration; existing data's codec is detected on read.
Caching
Memory and disk tiers hold recently read and written data.
TRIM
TRIM is supported and eventually translates to a reclamation of physical S3 objects.
Checkpoints
Named point-in-time checkpoints. Any checkpoint opens read-only.
Read replicas
One writer, any number of read-only instances against the same bucket. Replicas follow the writer's changes and return EROFS on write.
fsync and failover
Over 9P and NBD, a successful fsync means data is committed to the object store. An optional standby holds acked-but-unflushed writes and takes over in seconds.
Terminal session recordings.
The Linux kernel compiles in 16 seconds on ZFS over a ZeroFS NBD volume.
One config section adds a web console.
The file manager speaks 9P over WebSocket. Drag-and-drop uploads work, including entire folders.