feat(thumbnails): stop writing sidecars (step 10d2)
The write paths now persist only to the blob tiers. Until this, dual-write
meant any render or upload recreated .thumbnails/ seconds after the import
job removed it, so step 10e's gate — "the directory no longer exists" —
could never hold.
Rendered thumbnails: the fs::write in persist_rendered is gone. Safe
because the read flip landed first, so nothing depended on that write to
be found, and a failed derived store now costs a re-render rather than
data — regenerable by definition. Existing sidecars are untouched and stay
readable through the fallback until the import drains them.
Uploaded previews needed a change first, and the order was not optional.
upload_thumbnail_impl logged and still returned 201 when store_attached_blob
failed — safe only while ext-{file_id}.jpg was a second copy. These bytes
have NO server-side render path, so removing the sidecar while the store
stayed best-effort would lose a user's upload behind a success response.
The PUT is now fatal, and drops the RAM entry too, or the cache would keep
serving a preview that was never persisted and vanishes on eviction,
contradicting the error the client just received. Only then does the ext-
write go.
thumb_import_check.sh had to change with it: its premise was "upload, then
delete the row, and what remains on disk is legacy state", which no longer
holds now that nothing writes sidecars. It lays them down itself, with the
bytes the API just served, at the exact paths the pre-10d2 code used. The
reconstruction stays faithful — same bytes, same paths — it just no longer
depends on current code to produce a shape current code has stopped
producing. Both sidecars get identical bytes, which is realistic rather
than a shortcut: they dedup to one blob while keeping separate mappings,
which is the property the keying split exists to preserve.
This commit is contained in:
@@ -45,6 +45,7 @@ set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
COMPOSE_FILE="$REPO_ROOT/tests/common/docker-compose.test.yml"
|
||||
STORAGE_PATH="${OXICLOUD_STORAGE_PATH:-$REPO_ROOT/tests/api/storage}"
|
||||
|
||||
# shellcheck source=test.env
|
||||
source "$SCRIPT_DIR/test.env"
|
||||
@@ -131,6 +132,31 @@ curl -sf -X PUT -H "$AUTH" -H "Content-Type: image/png" \
|
||||
UPLOADED_THUMB=$(mktemp)
|
||||
curl -sf -H "$AUTH" "$base_url/api/files/$FILE_ID/thumbnail/preview" -o "$UPLOADED_THUMB"
|
||||
|
||||
# ── 1b. Lay down the sidecars the server no longer writes ────────────────
|
||||
#
|
||||
# Since step 10d2 the write paths persist ONLY to the blob tiers, so an
|
||||
# upload no longer leaves anything under .thumbnails/ — which is the point,
|
||||
# but it removes the source this test used to manufacture legacy state from.
|
||||
#
|
||||
# So write them here, with the bytes the API just served, at the exact paths
|
||||
# the pre-10d2 code used: `{size}/{blob_hash}.jpg` for the rendered
|
||||
# thumbnail and `{size}/ext-{file_id}.jpg` for the upload. Requests omit
|
||||
# `Accept`, so both negotiate JPEG.
|
||||
#
|
||||
# This keeps the reconstruction faithful rather than approximate: same
|
||||
# bytes, same paths, same filenames a pre-migration install holds. What it
|
||||
# no longer does is rely on the current code to produce them — which it
|
||||
# cannot, and should not.
|
||||
SIDECAR_DIR="$STORAGE_PATH/.thumbnails/preview"
|
||||
mkdir -p "$SIDECAR_DIR"
|
||||
cp "$UPLOADED_THUMB" "$SIDECAR_DIR/ext-$FILE_ID.jpg"
|
||||
cp "$UPLOADED_THUMB" "$SIDECAR_DIR/$BLOB_HASH.jpg"
|
||||
# Identical bytes in both, which is realistic rather than a shortcut: they
|
||||
# dedup to one blob, so the derived and attached rows end up referencing the
|
||||
# same content while keeping separate mappings — exactly the property the
|
||||
# keying split exists to preserve.
|
||||
log "legacy sidecars written to $SIDECAR_DIR"
|
||||
|
||||
DERIVED_BEFORE=$(sql "SELECT count(*) FROM storage.content_derived_blobs WHERE source_hash='$BLOB_HASH';")
|
||||
ATTACHED_BEFORE=$(sql "SELECT count(*) FROM storage.file_attached_blobs WHERE file_id='$FILE_ID';")
|
||||
[[ "$DERIVED_BEFORE" -ge 1 ]] || fail "expected a content_derived_blobs row before stripping"
|
||||
|
||||
Reference in New Issue
Block a user