feat(thumbnails): uploaded previews survive a copy
Completes step 9. The PUT wrote `ext-{file_id}.jpg` and nothing else —
keyed by file id, on local disk. No copy path duplicates it and no other
instance can see it, so a copied file lost the preview its owner
uploaded. Silently: the server falls back to rendering one from the
source, or to 204 for a PDF, which has no render path at all. A
user-supplied preview is not derivable from the content, so once lost it
is gone.
The PUT now also records a storage.file_attached_blobs row, which
copy_file_satellites already duplicates, so both copy paths carry it.
Best-effort: the sidecar has already succeeded by then and the user can
see their thumbnail, so failing the request would report an error for an
operation that visibly worked.
Read path consults attachments ahead of every content-derived tier: an
uploaded preview is an explicit choice about THIS file and must beat
anything rendered from its content. Cached under the per-file key — a
content key would leak those bytes to every other file sharing the
content, which is the poisoning the file-keyed table exists to prevent.
store_attached_blob is ON CONFLICT DO UPDATE, unlike its derived twin:
re-uploading a preview is a deliberate replacement, where a re-derived
thumbnail is the same bytes again. The superseded blob's reference is
released, or it would be pinned forever with nothing pointing at it.
Deletion goes through a trigger, not a hook. file_id is ON DELETE
CASCADE, and on_file_deleted fires AFTER delete_file — by then the
cascade has run and there is nothing left to enumerate. This matters
most for folder deletion, where PG cascades folders to files to
attachments and Rust never sees the rows at all. storage.decrement_blob_ref
keys off OLD.blob_hash and is otherwise table-agnostic, so it is reused
verbatim rather than transcribed into a second trigger that can drift.
DELETE only: a replacement updates in place and is handled in Rust, so
adding UPDATE would double-decrement.
Extracted read_blob_to_bytes, shared by the attached and derived tiers —
the only difference between them is which table produced the hash.
tests/api/attached_thumbnail_copy.hurl guards it. The file is red and
the uploaded thumbnail is green, so a render could never produce the
uploaded bytes; the pre-upload render is captured first and required to
change, which stops three identical renders from satisfying the
byte-equality. Then both copy paths must serve the upload, and after the
original is purged and GC runs, both copies must still serve it — each
holds its own reference, because the rows are duplicated rather than
shared.
This commit is contained in:
@@ -576,6 +576,106 @@ impl DedupService {
|
||||
/// `docs/plan/derived-blobs.md`.
|
||||
///
|
||||
/// Returns the derived blob hash.
|
||||
/// Attach user-supplied bytes to a FILE — the file-keyed twin of
|
||||
/// [`Self::store_derived_blob`].
|
||||
///
|
||||
/// Same storage path (the bytes are still content-addressed and still
|
||||
/// deduplicated), different mapping: the row is keyed by `file_id`, so
|
||||
/// two files holding identical attached bytes get two rows and two
|
||||
/// references. Sharing the mapping is what must not happen — a
|
||||
/// content-keyed client preview would let one user's upload be served
|
||||
/// for another user's file.
|
||||
///
|
||||
/// `ON CONFLICT … DO UPDATE`, unlike the derived twin: re-uploading a
|
||||
/// preview for the same `(file_id, kind, variant)` is a deliberate
|
||||
/// replacement, whereas a re-derived thumbnail is the same bytes again.
|
||||
/// The reference held by the row being replaced is released.
|
||||
pub async fn store_attached_blob(
|
||||
&self,
|
||||
file_id: &str,
|
||||
kind: &str,
|
||||
variant: &str,
|
||||
content_type: &str,
|
||||
bytes: Bytes,
|
||||
uploaded_by: uuid::Uuid,
|
||||
) -> Result<String, DomainError> {
|
||||
let stored = self
|
||||
.store_from_stream(
|
||||
stream::once(async move { Ok::<Bytes, std::io::Error>(bytes) }),
|
||||
Some(content_type.to_string()),
|
||||
)
|
||||
.await?;
|
||||
let attached_hash = stored.hash().to_string();
|
||||
|
||||
// `previous` is the hash this row pointed at before, when it existed
|
||||
// and differed — the reference to release once the row no longer
|
||||
// holds it.
|
||||
let previous: Option<(Option<String>,)> = sqlx::query_as(
|
||||
"INSERT INTO storage.file_attached_blobs
|
||||
(file_id, kind, variant, blob_hash, content_type, uploaded_by)
|
||||
VALUES ($1::uuid, $2, $3, $4, $5, $6)
|
||||
ON CONFLICT (file_id, kind, variant) DO UPDATE
|
||||
SET blob_hash = EXCLUDED.blob_hash,
|
||||
content_type = EXCLUDED.content_type,
|
||||
uploaded_by = EXCLUDED.uploaded_by,
|
||||
created_at = now()
|
||||
RETURNING NULLIF(storage.file_attached_blobs.blob_hash, EXCLUDED.blob_hash)",
|
||||
)
|
||||
.bind(file_id)
|
||||
.bind(kind)
|
||||
.bind(variant)
|
||||
.bind(&attached_hash)
|
||||
.bind(content_type)
|
||||
.bind(uploaded_by)
|
||||
.fetch_optional(self.pool.as_ref())
|
||||
.await
|
||||
.map_err(|e| DomainError::internal_error("Dedup", format!("record attached blob: {e}")))?;
|
||||
|
||||
// A replaced row's old blob loses its only reference from here. Not
|
||||
// releasing it would pin those bytes forever — nothing else points at
|
||||
// a superseded preview.
|
||||
if let Some((Some(old_hash),)) = previous
|
||||
&& old_hash != attached_hash
|
||||
&& let Err(e) = self.remove_reference(&old_hash).await
|
||||
{
|
||||
tracing::warn!(
|
||||
target: "oxicloud::dedup",
|
||||
error = %e,
|
||||
"failed to release replaced attached-blob reference for {}",
|
||||
&old_hash[..old_hash.len().min(12)],
|
||||
);
|
||||
}
|
||||
|
||||
Ok(attached_hash)
|
||||
}
|
||||
|
||||
/// Look up bytes attached to a file. File-keyed counterpart of
|
||||
/// [`Self::find_derived_blob`].
|
||||
pub async fn find_attached_blob(
|
||||
&self,
|
||||
file_id: &str,
|
||||
kind: &str,
|
||||
variant: &str,
|
||||
) -> Option<crate::application::ports::dedup_ports::DerivedBlobRef> {
|
||||
sqlx::query_as::<_, (String, String)>(
|
||||
"SELECT blob_hash, content_type FROM storage.file_attached_blobs
|
||||
WHERE file_id = $1::uuid AND kind = $2 AND variant = $3",
|
||||
)
|
||||
.bind(file_id)
|
||||
.bind(kind)
|
||||
.bind(variant)
|
||||
.fetch_optional(self.pool.as_ref())
|
||||
.await
|
||||
.ok()
|
||||
.flatten()
|
||||
.map(|(blob_hash, content_type)| {
|
||||
crate::application::ports::dedup_ports::DerivedBlobRef {
|
||||
blob_hash,
|
||||
content_type,
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
pub async fn store_derived_blob(
|
||||
&self,
|
||||
source_hash: &str,
|
||||
|
||||
Reference in New Issue
Block a user