cjw/Oxicloud
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

perf(thumbnail): cache attached-blob lookups on the request path

Browse Source 
Every thumbnail request paid an uncached storage.file_attached_blobs
point query before it could answer — including 304 revalidations and
RAM thumbnail hits, where the ETag path (thumbnail_content_id) probes
the row every time and tier 2b probes it again with the same key. A
photos grid revalidating 60 thumbnails per visit meant 60+ point
queries per browse, repeated on every visit.

find_attached_blob now reads through a process-local moka cache in
DedupService, keyed by the row's (file_id, kind, variant) PK, holding
positive and negative entries (most files have no attached preview, so
the negative side carries the win). Two rules keep it honest:

- DB faults are surfaced as Err and never cached — a transient outage
  cannot freeze "no attached blob" into a negative entry (a read
  failure is never proof that data is absent). The public signature is
  unchanged; the SQL body moved to find_attached_blob_uncached.
- Writes invalidate eagerly: store_attached_blob and the Inserted arm
  of store_attached_blob_if_absent on success, and deletions via
  ThumbnailRefreshHook::on_file_deleted, which all three production
  delete paths (single file, folder cascade, trash clear) fire after
  the DELETE commits. The 60s TTL bounds only what the process cannot
  see (bare SQL, copy_file_satellites races).

The Nextcloud preview endpoint rides the same lookup and benefits
identically. Five in-memory contract tests pin the cache behaviour,
including the fault-not-cached rule.

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
SZCJW
2026-09-20 00:28:00 +08:00
parent 68e21f4bef
commit d33d1932b6
4 changed files with 340 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/caching.md
+25
View File
@@ -16,8 +16,33 @@ The two layers are orthogonal — the moka caches shave query round-trips regard
| Thumbnail cache | configurable | 1 000 | Generated WebP/AVIF thumbnails |
| Image transcode | configurable | 500 | On-the-fly image transcoding results |
| Blob hash | 30 s TTI | 5 000 | BLAKE3 hashes for dedup lookups |
| Attached blob | 60 s TTL | 50 000 | `file_attached_blobs` row lookups on the thumbnail hot path (ETag + tier-2b, also the Nextcloud preview endpoint) |
| Audio metadata | — | 2 000 | ID3 tags and duration |
### The attached-blob cache
Every thumbnail request pays a `storage.file_attached_blobs` point query
before it can even answer "304 Not Modified" — the ETag names the attached
blob's hash. A photos grid revalidating 60 thumbnails per visit means
60+ point queries per browse. The cache sits in `DedupService` in front of
that lookup (`find_attached_blob`), keyed by the row's `(file_id, kind,
variant)` primary key, and caches **both directions**: `Some(row)` and
`None` (most files have no attached preview, so the negative side is where
most of the win is).
Two rules keep it honest:
- **DB faults are never cached.** The uncached lookup surfaces errors as
`Err`; only a genuine `Ok(None)` fills a negative entry. A transient
outage must not freeze "no attached blob" into place for a full TTL —
a read failure is never proof that data is absent.
- **TTL is the bound, not the invalidation strategy.** Writes invalidate
eagerly — `store_attached_blob` / `store_attached_blob_if_absent` on
success, deletions via `ThumbnailRefreshHook::on_file_deleted` (which
all three production delete paths fire). The 60 s TTL only bounds what
the process cannot see: bare SQL, the `copy_file_satellites` race
window, a hypothetical second instance.
### How it works
1. **Read path:** check cache → if hit, return immediately (sub-ms); if miss, query PostgreSQL, populate cache, return