Cloudflare R2

Index snapshots: StorageContext over fsspec

LlamaIndex persistence has a quietly great property: StorageContext writes through fsspec, so "persist to disk" and "persist to a bucket" are the same code path with a different filesystem handle. This platform's snapshot lane hands it an s3fs filesystem pointed at R2: the index snapshots to object storage and restores from it — same API, no custom serialization.

Like every optional capability in the service, the lane is opt-in (R2_STORE=1, its own requirements file): s3fs drags in aiobotocore, which is dead weight the production runtime doesn't carry unless the lane is actually in use.

Why R2

The platform already runs on Cloudflare (Workers, D1, Queues) — but the deciding feature is zero egress fees. The audio library is served straight from a public R2 bucket: 40+ minute generated audio guides are exactly the kind of asset where egress pricing on other clouds quietly becomes the bill.

What lives there

• Index snapshots — the StorageContext lane above; a fresh environment restores retrieval state without re-ingesting.
• Audio artifacts — every chapter the TTS pipeline produces, published under a public domain after passing its quality gates.
• Backups — a scheduled cron lane snapshots platform data to the bucket.

The pattern

Compute is disposable here — Workers, a Render service, batch lanes that run and exit. R2 is the durable layer they all agree on: artifacts are written once, content-addressed where it matters, and any environment can be reconstructed from the bucket plus the repo. Object storage as the source of recovery, not just a file dump.

Numbers

• Zero egress fees — the deciding feature for serving 40+ minute audio guides from a public bucket.
• Index snapshot/restore is one code path — StorageContext over fsspec, disk and bucket interchangeable.
• A daily cron backs up the production Qdrant collection — the retrieval index's only copy — via the serving ops lane.
• The s3fs lane is opt-in (R2_STORE=1, own requirements file): aiobotocore never rides to production unused.