Silent data loss on an ordinary UI folder copy.
The function bumped only storage.blobs:
UPDATE storage.blobs b SET ref_count = ref_count + hc.cnt
FROM (...) hc WHERE b.hash = hc.blob_hash;
but a CDC file's blob_hash names a MANIFEST, not a chunk. For any
multi-chunk file that predicate matches zero rows, so the copy took no
reference at all. Delete the original afterwards and remove_reference
walks the manifest to 0, dedup_gc reaps the manifest and every chunk
behind it, and the copy is unreadable.
Single-chunk files escaped by accident: their whole-file hash equals
their lone chunk's hash, so the UPDATE did match — bumping the wrong
counter, which surfaces as a manifest under-count plus a blob
over-count rather than as loss. That asymmetry is why the bug survived:
small files, which dominate most test corpora, look fine.
Reproduced through the UI on a 5 MiB / 18-chunk file:
chunk_manifests.ref_count stayed at 1 while two storage.files rows
referenced it, and manifests_consistency reported
manifest_refcount_mismatch with delta 1, reap_risk true.
The fix mirrors DedupService::add_reference — manifest first, blobs only
as fallback, with a NOT EXISTS guard so a single-chunk file is not
counted at both levels (which would turn the under-count into an
over-count). orphaned_at is cleared on the blobs branch, as
add_reference does when resurrecting a blob inside its GC grace window.
Only the reference-counting block changes; the rest of the function is
20260902000001 verbatim.
Existing drift is deliberately NOT repaired here — a schema migration
cannot know which counter is authoritative. manifests_consistency
reports it; repair belongs with the recovery framework.
NOT executed against a database: the test instance was down and the dev
instance is read-only by convention. A parse error would fail at boot,
before any data is touched. Verify by re-running the reproduction — a
folder copy of a >1 MiB file should now leave ref_count at 2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Edouard Vanbelle
4b05eec8ad