docs(plan): ETag moves to the derived hash at the read-order flip
The ETag shipped in fe9c4f49 is keyed on (source_hash, size, format), which is one term short: a thumbnail is a function of those PLUS the renderer. Change the encoder or a quality setting and identical inputs produce different bytes under an unchanged ETag — the same staleness class the commit fixed, one level down. It bites when an already-cached thumbnail is re-rendered after a renderer change. Keying on the derived blob's own hash removes the term entirely: the ETag IS the hash of the bytes, so any output change invalidates by construction. It is self-consistent for free, because store_derived_blob is ON CONFLICT DO NOTHING — a re-render never displaces the stored row, so the ETag always equals what the derived tier will serve. No renderer version constant to remember to bump. Records why it cannot land yet. The derived tier is read LAST by design, so an ETag naming the derived hash would describe a tier the response probably did not come from; sidecar and derived agree at creation but diverge if a sidecar is re-rendered while the derived row stays pinned by DO NOTHING. An ETag that lies about the body is worse than one that is merely coarse. Also the tier is WebP-only (variant is the size, with no format term) and empty for anything predating this work until derived_import backfills. So it lands at step 10 with the flip, keeping today's form as the fallback for ungenerated variants and formats the tier does not hold. The LEFT JOIN already planned for the read path returns the derived hash in the same query, so it costs no extra round-trip.
This commit is contained in:
@@ -949,6 +949,48 @@ the request. Read order:
|
|||||||
backend stack, which is where the disk cache lives.
|
backend stack, which is where the disk cache lives.
|
||||||
4. Generate only if step 2 found no row.
|
4. Generate only if step 2 found no row.
|
||||||
|
|
||||||
|
### HTTP ETag — move to the derived hash when the order flips
|
||||||
|
|
||||||
|
Shipped ahead of this plan (2026-08-24): the thumbnail ETag is
|
||||||
|
`"thumb-{source_hash}-{size}-{format}"` on both the REST and the
|
||||||
|
NextCloud preview endpoint. It replaced a `file_id`-keyed ETag that,
|
||||||
|
combined with `Cache-Control: immutable`, meant replacing a file's
|
||||||
|
content never invalidated the client's copy — `file_id` survives the
|
||||||
|
replacement, so the ETag did too, for a year.
|
||||||
|
|
||||||
|
**That key is still one term short.** A thumbnail is a function of
|
||||||
|
`(source bytes, size, format, RENDERER)`. Change the encoder, a quality
|
||||||
|
setting, or EXIF-rotation handling, and identical inputs produce
|
||||||
|
different bytes under an unchanged ETag — the same staleness class, one
|
||||||
|
level down. It bites when an already-cached thumbnail is re-rendered
|
||||||
|
after a renderer change (sidecar evicted, regenerated on miss).
|
||||||
|
|
||||||
|
**The fix is to key on the derived blob's own hash** — the ETag then
|
||||||
|
*is* the hash of the bytes served, so any change in output invalidates
|
||||||
|
by construction, with no version constant to remember to bump. It is
|
||||||
|
self-consistent for free: `store_derived_blob` is
|
||||||
|
`ON CONFLICT DO NOTHING`, so a re-render never displaces the stored
|
||||||
|
row, and the ETag therefore always equals what the derived tier will
|
||||||
|
serve.
|
||||||
|
|
||||||
|
**It must land with the read-order flip, not before.** Today the
|
||||||
|
derived tier is deliberately read LAST, so an ETag naming the derived
|
||||||
|
hash would describe a tier the response probably did not come from.
|
||||||
|
The two agree at creation — `render_and_persist_all_webp` writes both
|
||||||
|
from the same bytes — but diverge if a sidecar is re-rendered while the
|
||||||
|
derived row stays pinned by `DO NOTHING`. Sidecar is served, ETag
|
||||||
|
describes the other one: an ETag that lies about the body is worse than
|
||||||
|
one that is merely coarse. Two further reasons it has to wait: the
|
||||||
|
derived tier is WebP-only (`variant = size.dir_name()`, no format in
|
||||||
|
the key), so non-WebP clients have no row to key on; and nothing
|
||||||
|
predating this work has a row until `derived_import` backfills.
|
||||||
|
|
||||||
|
So at step 10, alongside the flip: ETag becomes the derived hash, with
|
||||||
|
the current `source_hash` form kept as the fallback for a variant not
|
||||||
|
yet generated and for formats the derived tier does not hold. The
|
||||||
|
`LEFT JOIN` in step 2 above already returns the derived hash in the
|
||||||
|
same query, so the ETag costs no extra round-trip.
|
||||||
|
|
||||||
**The disk cache is `CachedBlobBackend`, reused unchanged.** No
|
**The disk cache is `CachedBlobBackend`, reused unchanged.** No
|
||||||
thumbnail-specific cache, no second root path. Routing derived
|
thumbnail-specific cache, no second root path. Routing derived
|
||||||
blobs through the same stack gets, for free:
|
blobs through the same stack gets, for free:
|
||||||
@@ -1171,7 +1213,11 @@ hardcoded SQL). New sources bolt on independently.
|
|||||||
declare its version semantics.
|
declare its version semantics.
|
||||||
10. **`derived_import` job + the dual-read fallback** — see the
|
10. **`derived_import` job + the dual-read fallback** — see the
|
||||||
migration section. Phase 3 (deleting the fallback and the sidecar
|
migration section. Phase 3 (deleting the fallback and the sidecar
|
||||||
dirs) is a separate later release, gated on an empty tail.
|
dirs) is a separate later release, gated on an empty tail. **The
|
||||||
|
HTTP ETag moves to the derived hash here**, with the read-order
|
||||||
|
flip and not before — see "HTTP ETag" under *Read path and
|
||||||
|
caching* for why keying it earlier would make the ETag describe a
|
||||||
|
tier the response did not come from.
|
||||||
11. **`DedupService` → `BlobHandler` rename** — decided, mechanical,
|
11. **`DedupService` → `BlobHandler` rename** — decided, mechanical,
|
||||||
34 files. Standalone commit, `src/AGENTS.md` updated with it. Can
|
34 files. Standalone commit, `src/AGENTS.md` updated with it. Can
|
||||||
land at any point; last is easiest, since every earlier slice
|
land at any point; last is easiest, since every earlier slice
|
||||||
|
|||||||
Reference in New Issue
Block a user