Separate WAL Object Store

With the WAL enabled, every fsync writes to the WAL, so fsync latency equals the latency of the WAL's backing store. You can point the WAL at a separate, lower-latency store.

Overview

By default the WAL goes to the same object store as everything else. If fsync latency matters for your workload, you can configure a separate WAL store, such as local NVMe, S3 Express One Zone, or a nearby S3-compatible service.

Configuration

Add a [wal] section to your configuration file:

[wal]
url = "file:///mnt/nvme/zerofs-wal"

The [wal] section supports its own credential blocks ([wal.aws], [wal.azure], [wal.gcp]), independent from the main storage credentials:

[wal]
url = "s3://my-wal-bucket/zerofs-wal"

[wal.aws]
access_key_id = "${WAL_AWS_KEY}"
secret_access_key = "${WAL_AWS_SECRET}"
endpoint = "https://s3-endpoint.example.com"

The [wal] section also accepts an optional storage_class (see Storage Class), independent from the one under [storage].

If no [wal] section is present, the WAL is written to the main object store.

Important Constraints

Whether you use a separate WAL store is decided at filesystem creation time. The underlying storage engine records this in its manifest, so you cannot add or remove a separate WAL store on an existing filesystem.

You can move the WAL to a different location by updating the [wal] URL and credentials, but you must manually migrate the WAL files from the old store to the new one before starting ZeroFS.