Defer CDC chunk reclamation in manifest dereference to GC
remove_manifest_reference unlinked a chunk's backing file right after the row-delete committed — the same TOCTOU the GC grace window was added to close: a concurrent upload of identical content can re-reference (pin) the chunk in the gap between commit and unlink, after which the deferred unlink strands a referenced chunk with no bytes. Route physical chunk reclamation through the single grace-protected path: on last reference, delete the manifest and decrement its chunks (stamping orphaned_at on the ones that reach 0), but leave the chunk rows and files for garbage_collect() to reclaim once orphaned past the grace window. The manifest deletion and its blob-keyed thumbnail hook stay eager. remove_legacy_reference and cleanup_if_orphaned's legacy path are left as eager deletes on purpose: a legacy whole-file hash can never be re-created by an ingest (uploads are always CDC now), so there is no writer to race — the existing "row gone ⇒ no resurrection" reasoning holds for them. Adds an integration test asserting a CDC manifest dereference leaves chunks orphaned-but-present, then reclaimed by a post-grace GC. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0172rsVwzTwD216R9HXT2aU4
This commit is contained in:
@@ -112,7 +112,10 @@ pub trait DedupPort: Send + Sync + 'static {
|
||||
|
||||
/// Remove a reference from a blob.
|
||||
///
|
||||
/// Returns `true` if the blob was deleted (ref_count reached 0).
|
||||
/// Returns `true` if the last reference was removed (the content is now
|
||||
/// unreferenced). For CDC content the now-orphaned chunks are reclaimed
|
||||
/// later by garbage collection rather than unlinked inline; legacy
|
||||
/// whole-file blobs are still freed eagerly.
|
||||
async fn remove_reference(&self, hash: &str) -> Result<bool, DomainError>;
|
||||
|
||||
/// Calculate BLAKE3 hash of a file (streaming).
|
||||
|
||||
Reference in New Issue
Block a user