feat(storage): derived variant encodes the output format
`content_derived_blobs.variant` held the size alone, so one source could
hold exactly one artifact per size regardless of codec. That surfaced
when the read order flipped in 10c: a JPEG request matched the WebP row
and would have been served the wrong codec — hidden previously because
the .jpg sidecar won first. The flip had to be gated to WebP, which
meant JPEG clients could never leave the sidecar, which meant the
sidecar could never be deleted.
It blocks transcodes harder: those are multi-format by nature, so two
output codecs of one source collide on the primary key without a format
term.
The axis goes inside the string rather than into a fourth PK column,
per the column's own rule — "new axes go inside this string, never into
new columns". Shape is {size}.{ext}: preview.webp, icon.jpg, later
720p.webp.
The backfill is deterministic, not a guess: store_derived_blob has only
ever written "image/webp" for thumbnails. content_type is checked anyway
rather than assumed — a row that fails the assumption is left alone and
counted in a warning, because the read path then simply misses it and
falls back to the sidecar, whereas guessing a codec would serve wrong
bytes. Idempotent via NOT LIKE '%.%', so a re-apply cannot produce
preview.webp.webp; verified on a scratch PG by applying it twice.
One helper builds the string, because it is a primary-key component:
a writer and reader that disagree do not fail loudly, they just never
find each other's rows and the derived tier silently looks empty. It
lives on the service's ThumbnailSize, not the port's — they are distinct
types, which the compiler pointed out after I put it on the wrong one.
The WebP gate on the read path is now removed: each codec has its own
row, so JPEG can finally reach the derived tier — the prerequisite for
deleting the sidecar for those clients.
file_attached_blobs keeps a bare size: store_external_thumbnail
re-encodes everything to JPEG, so it is single-format by construction
and a format term would cost a migration for nothing.
This commit is contained in:
@@ -159,7 +159,18 @@ impl RecoverableJobHandler for ThumbDerivedImport {
|
||||
let mut already = 0u64;
|
||||
let mut failed = 0u64;
|
||||
let mut since_checkpoint = 0usize;
|
||||
let variant_of = |s: ThumbnailSize| s.dir_name().to_string();
|
||||
// Sidecars under `.thumbnails/` are `{hash}.webp` — the filter that
|
||||
// built this list requires the extension — so the imported rows are
|
||||
// WebP, and the variant must say so since migration
|
||||
// `20261022000000`. Writing the bare size here would produce rows the
|
||||
// read path can never match.
|
||||
let variant_of = |s: ThumbnailSize| {
|
||||
format!(
|
||||
"{}.{}",
|
||||
s.dir_name(),
|
||||
crate::application::ports::thumbnail_ports::ThumbnailFormat::Webp.ext()
|
||||
)
|
||||
};
|
||||
|
||||
for size in ThumbnailSize::all() {
|
||||
let dir_name = variant_of(*size);
|
||||
|
||||
@@ -59,6 +59,22 @@ impl ThumbnailSize {
|
||||
}
|
||||
}
|
||||
|
||||
/// The `content_derived_blobs.variant` value for this size and format.
|
||||
///
|
||||
/// One place builds the string, because it is a primary-key component: a
|
||||
/// writer and a reader that disagree do not fail loudly, they simply
|
||||
/// never find each other's rows — the read falls back to the sidecar and
|
||||
/// the derived tier silently looks empty.
|
||||
///
|
||||
/// The format term is what lets one source hold both codecs at a size.
|
||||
/// Without it a JPEG request matched the WebP row and would be served the
|
||||
/// wrong codec, which is why the step-10c read flip had to be gated to
|
||||
/// WebP and why JPEG clients could never leave the sidecar. See migration
|
||||
/// `20261022000000`.
|
||||
pub fn derived_variant(&self, format: ThumbnailFormat) -> String {
|
||||
format!("{}.{}", self.dir_name(), format.ext())
|
||||
}
|
||||
|
||||
/// Get all thumbnail sizes
|
||||
pub fn all() -> &'static [ThumbnailSize] {
|
||||
&[
|
||||
@@ -310,7 +326,7 @@ impl ThumbnailService {
|
||||
.store_derived_blob(
|
||||
blob_hash,
|
||||
"thumbnail",
|
||||
size.dir_name(),
|
||||
&size.derived_variant(format),
|
||||
format.mime(),
|
||||
bytes.clone(),
|
||||
)
|
||||
@@ -806,17 +822,15 @@ impl ThumbnailService {
|
||||
// draining, most content has a sidecar and no row, and terminating
|
||||
// here would return "no thumbnail" for all of it.
|
||||
//
|
||||
// **WebP only.** `store_derived_blob` writes `image/webp` and keys
|
||||
// `variant` on the size alone, with no format term, so a JPEG request
|
||||
// would match the WebP row and be served the wrong codec — a
|
||||
// regression the old ordering hid, because the `.jpg` sidecar won
|
||||
// first. Until `variant` encodes format, JPEG clients stay on the
|
||||
// sidecar, and the sidecar therefore cannot be deleted for them. See
|
||||
// docs/plan/derived-blobs.md.
|
||||
if format == ThumbnailFormat::Webp
|
||||
&& let Some(dedup) = dedup
|
||||
// All formats, since migration `20261022000000` put the output format
|
||||
// inside `variant`. Before that, `variant` was the size alone, so a
|
||||
// JPEG request matched the WebP row and would have been served the
|
||||
// wrong codec — the flip had to be gated to WebP, which meant JPEG
|
||||
// clients could never leave the sidecar and the sidecar could never
|
||||
// be deleted. Now each codec has its own row.
|
||||
if let Some(dedup) = dedup
|
||||
&& let Some(derived) = dedup
|
||||
.find_derived_blob(hash, "thumbnail", size.dir_name())
|
||||
.find_derived_blob(hash, "thumbnail", &size.derived_variant(format))
|
||||
.await
|
||||
&& let Some(bytes) =
|
||||
Self::read_blob_to_bytes(dedup, &derived.blob_hash, file_id, size).await
|
||||
|
||||
Reference in New Issue
Block a user