Make S3 yourprimary storage.
ZeroFS serves S3-compatible buckets as POSIX filesystems over NFS and 9P, or as raw block devices over NBD. Data is compressed and encrypted before upload. Warm reads come from local cache in microseconds.
or via Docker · GitHub Action
The test suites run in public CI.
CI runs pjdfstest, xfstests, kernel builds, stress-ng, and ZFS scrubs on every change. The first three run separately over NFS, 9P, and the FUSE client. Each card links to its workflow.
POSIX semantics
The pjdfstest suite runs on every change, once per protocol: permissions, ownership, links, rename behavior. The exclude lists, a few cases per protocol, are published in the repository.
pjdfstest workflow →xfstests
The kernel filesystem test suite runs over NFS, 9P, and FUSE in separate workflows. These are the tests ext4 and XFS themselves are validated against.
xfstests workflow →ZFS as the end-to-end test
CI builds a ZFS pool on ZeroFS block devices, extracts the Linux kernel source tree onto it, and runs a full scrub. The scrub reports no checksum errors.
zfs-test workflow →Kernel builds
CI compiles the Linux kernel on NFS, 9P, and FUSE mounts with make -j$(nproc). Parallel compilation is the stress test: many processes writing the same tree at once.
stress-ng
Filesystem stressors run against live mounts in CI: access, chdir, chmod, chown, and the rest of the file-handling set, all at once.
stress-ng workflow →Self-hosting
The Rust toolchain builds ZeroFS on a filesystem that ZeroFS itself is serving. This one is a recorded session rather than a CI job.
watch the recording →Files over NFS and 9P, block devices over NBD.
All three servers run in one userspace process against the same bucket. Clients mount it with the NFS and 9P support already in Linux, the NFS clients other systems ship, or nbd-client for block devices.
NFS
macOS, Linux, Windows, and the BSDs mount it over their own NFS support, with nothing extra installed on the client. The server stays in userspace.
# mounts from any major OS
mount -t nfs 127.0.0.1:/ /mnt/zerofs
9P
9P follows POSIX more closely than NFS, and fsync returns only after data reaches stable storage. The bundled FUSE client mounts without root and reconnects on its own.
# bundled FUSE client, no root
zerofs mount 127.0.0.1:5564 /mnt/zerofs
NBD
Raw block devices stored in the bucket hold ext4 filesystems, ZFS pools, or VM boot disks. New devices are picked up at runtime, with no server restart.
# attach a block device
nbd-client 127.0.0.1 10809 /dev/nbd0 -N vol1
A ZFS mirror across three S3 regions.
Each ZeroFS instance exposes one S3 region as a block device. To ZFS they are plain disks, so a mirror that spans continents is set up like any other pool.
If a region becomes unreachable, the pool degrades and the data stays available from the other two.
Eight properties of the storage engine.
Always encrypted
Every block is encrypted with XChaCha20-Poly1305 before upload. The data key is wrapped with a key derived from your password using Argon2id. There is no unencrypted mode.
Compression
Data is compressed with zstd or lz4 before encryption. The codec can change at any time without migration, since the codec of existing data is detected on read.
Caching
Configurable memory and disk caches hold recently used blocks. Warm reads return in microseconds. A raw S3 round trip takes 50 to 300 milliseconds.
Checkpoints
Named checkpoints capture the filesystem at a point in time. Any of them can be opened read-only with a single flag at startup.
Read replicas
One instance writes while read-only instances serve the same bucket. Replicas pick up the writer's changes automatically and return EROFS on writes.
TRIM support
Discard from any filesystem or zpool deletes the corresponding chunks from the LSM tree. Compaction then reclaims the space in S3, so freed blocks come off the bill.
Standalone compactor
Compaction can run as a separate process on its own machine. A spot instance in the bucket's region avoids egress fees and keeps the load off the serving path.
Separate WAL store
With the WAL enabled, fsync latency is the latency of the WAL's backing store. The WAL can live on local NVMe or S3 Express One-Zone while data stays in the main bucket.
One config section adds a web console.
Enable [servers.webui] and the same process serves a console: a file manager running 9P over WebSocket, a dashboard streaming stats over gRPC-web, and a terminal that boots a Linux VM rooted on the mount.
Drag-and-drop uploads work for files and entire folders, written to the bucket over 9P.
Four recorded terminal sessions.
These are asciinema recordings of real terminal sessions, rendered as text rather than video. Playback caps idle pauses at two seconds and changes nothing else.
The Linux kernel compiles in 16 seconds on ZFS over a ZeroFS NBD volume.