diff --git a/docs/architecture/caching.md b/docs/architecture/caching.md index ed93aae6..7e2ba448 100644 --- a/docs/architecture/caching.md +++ b/docs/architecture/caching.md @@ -1,8 +1,13 @@ # Caching Architecture -OxiCloud uses **moka** (a lock-free, concurrent cache) for write-behind caching that delivers sub-millisecond hot reads. +OxiCloud runs two independent cache layers with different jobs: -## Cache Layers +1. **In-memory metadata caches** — [moka](https://docs.rs/moka) instances that sit in front of PostgreSQL and other hot-path lookups. Sub-millisecond hits, bounded by entry count, TTL-evicted. Cover file metadata, directory listings, blob hashes, audio metadata, small thumbnails, on-the-fly image transcodes. +2. **On-disk blob cache** — an LRU cache of blob **bytes** on local SSD, only meaningful when the storage backend is remote (S3, Azure). Turns remote fetches into local reads for hot content; bounded by a disk-budget in bytes, LRU-evicted. Off by default. + +The two layers are orthogonal — the moka caches shave query round-trips regardless of backend; the disk-blob cache shaves network round-trips when the backend is elsewhere. + +## Layer 1 — In-memory metadata caches (moka) | Cache | TTL | Max Entries | Purpose | |---|---|---|---| @@ -13,21 +18,63 @@ OxiCloud uses **moka** (a lock-free, concurrent cache) for write-behind caching | Blob hash | 30 s TTI | 5 000 | BLAKE3 hashes for dedup lookups | | Audio metadata | — | 2 000 | ID3 tags and duration | -## How It Works +### How it works 1. **Read path:** check cache → if hit, return immediately (sub-ms); if miss, query PostgreSQL, populate cache, return 2. **Write path:** update PostgreSQL → invalidate relevant cache entries 3. **TTL expiry:** entries are evicted after their time-to-live, ensuring eventual consistency -## Why moka? +### Why moka? - **Lock-free** — no mutex contention under concurrent access - **Bounded memory** — max entries prevent unbounded growth - **TTL + TTI** — supports both time-to-live and time-to-idle eviction - **Async-ready** — works natively with Tokio +## Layer 2 — On-disk blob cache + +A local-SSD LRU cache of blob bytes, sitting between OxiCloud and remote storage backends (S3, Azure, or any other `BlobStorageBackend`). Every blob read probes the local cache first; misses fetch from the remote backend and populate the cache. Writes go to the remote backend AND the local cache simultaneously, so a just-uploaded blob is immediately hot for its own re-reads. + +Structurally: the bytes live on disk, one `.blob` file per hash, sharded by hash prefix under a configurable directory (default `{root}/.blob-cache//.blob`). The in-process index is a `moka::sync::Cache` with a byte-weigher — same crate as Layer 1, but weighing by content size not entry count, and only tracking file existence, not payload. + +### When it earns its keep + +Turn on for any deployment where the backend is not on the same box: + +- S3 (AWS, DigitalOcean Spaces, Cloudflare R2, MinIO on another host, …) +- Azure Blob Storage +- Any future network-attached backend + +Local backends (`LocalFilesystem`) don't need it — they're already on the same box. Enabling it there just doubles disk usage for zero latency win. + +**Thumbnails are the strongest reason to turn this on.** OxiCloud stores thumbnails as blobs alongside primary content (via `content_derived_blobs`, tracked in `storage.blobs` like any other blob) — the sidecar-on-disk layout is gone. On a remote backend this means every thumbnail render is a network fetch: a photos grid with 100 thumbnails is 100 S3 requests, per user, per visit. With Layer 2 on, that cost is paid once per thumbnail hash; every subsequent grid render is local-disk reads. + +Concrete impact for the photos / file-listing hot paths: + +- **Cold render** (all thumbnails uncached): one remote fetch per thumbnail, latency dominated by the backend's per-request round-trip (S3 typically 30-80 ms per object, more at distance). +- **Warm render** (thumbnails cached): local `open()` + read, sub-millisecond per file. +- **Hit rate in practice**: high — thumbnails are small (typically 5-30 KB per size variant), users re-visit the same folders repeatedly, and the LRU pattern strongly favours recency. + +Rule of thumb: if your backend is remote AND you have any user-facing photo grid or file browser, Layer 2 is worth the disk budget. On S3 backends it's the difference between a snappy gallery and a spinner-per-tile browsing experience. + +### Sizing guidance + +The cache is LRU on a disk-budget basis. A working set larger than the cache size will still work but re-fetch cold blobs from the remote — no correctness cost, just latency. Rough sizing: + +- **Home / personal cloud** — 5-10 GB is plenty; the working set for a household of active users is small. +- **Small team / SMB** — 50-100 GB for a hot photo library or shared document store. +- **Large deployment** — size against your top-decile access pattern; the cache doesn't need to cover the whole store. + +The default budget is 50 GB (only applied if the cache is enabled). Adjust to what your local SSD can spare. + +### Interaction with the moka layer + +Independent. A file-metadata hit in Layer 1 tells you the row exists and has a `blob_hash` — but reading the actual bytes still goes through Layer 2 (or straight to the remote backend if disabled). A hit in Layer 2 short-circuits the network fetch; a miss populates it for the next read. + ## Configuration +### In-memory metadata caches (Layer 1) + Cache parameters are currently hardcoded in `src/common/config.rs`. Key defaults: ```rust @@ -35,3 +82,15 @@ file_cache_ttl_ms: 60_000, // 1 minute directory_cache_ttl_ms: 120_000, // 2 minutes max_cache_entries: 10_000, ``` + +### On-disk blob cache (Layer 2) + +Environment-tunable — off by default; enable per deployment when the backend is remote: + +| Env var | Default | Purpose | +|---|---|---| +| `OXICLOUD_STORAGE_CACHE_ENABLED` | `false` | Master switch. Set `true` to wrap the blob backend with the cache decorator. | +| `OXICLOUD_STORAGE_CACHE_MAX_SIZE` | `53687091200` (50 GB) | Disk-budget in bytes. LRU eviction fires when the cache exceeds this size. | +| `OXICLOUD_STORAGE_CACHE_PATH` | `{root}/.blob-cache` | Where the cache files live. Point at a fast SSD; can be a separate volume from the primary storage root. | + +Restart the server after changing any of these — the cache is instantiated once at boot around the configured blob backend.