docs(plan): add step 12 — document the two satellite tables

The tables exist and carry COMMENT ON text, but nothing explains the
pair together, and the relationship is the part that matters: they hold
the same kind of artifact under two different keys, and the keying
difference IS the security boundary. Content keying shares one
derivation across identical content, which is exactly what must not
happen for user-supplied bytes — one user's uploaded preview would be
served for every file whose content matches. A single table with a kind
column would not prevent that; the split does.

Today that reasoning lives only in scattered prose across this plan and
two job doc-comments, so someone adding a third artifact type has
nothing to read.

Scopes the page: column-by-column structure including the parts that
mislead (nullable blob_hash meaning a negative verdict, uploaded_by
NOT NULL with no FK), worked examples that make the rule checkable,
lifecycle and which consistency job covers which failure, and the
NULL-handling trap that has already caused two bugs — comparison
against NULL silently excludes negative rows, which is correct for
refcounts and wrong for dangling checks.

Lands in docs/architecture/ beside backend-storage.md, which documents
the blob layer underneath.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Edouard Vanbelle
2026-08-30 22:42:34 +02:00
parent ba6fe49c71
commit 620800ba32
+59
View File
@@ -1567,6 +1567,65 @@ hardcoded SQL). New sources bolt on independently.
land at any point; last is easiest, since every earlier slice
would otherwise rebase across it.
12. **Document the two satellite tables** — `storage.content_derived_blobs`
and `storage.file_attached_blobs`, with worked examples. **Not
started.**
The tables exist and each carries `COMMENT ON` text, but nothing
explains the pair *together*, and the thing that matters about them
is the relationship: they hold the same kind of artifact under two
different keys, and **the keying difference is the security
boundary**. Someone adding a third artifact type has to get that
right, and today the only place it is written down is prose
scattered through this plan and two job doc-comments.
What the page needs:
* **Column-by-column structure** of both tables, including the
parts that are easy to get wrong: `blob_hash` is nullable on
`content_derived_blobs` (a NULL is a NEGATIVE row — the
derivation was attempted and is known not to be worth storing),
paired with `content_type` by a CHECK; `uploaded_by` on
`file_attached_blobs` is `NOT NULL` with no FK, so provenance
survives a user deletion, and imported rows carry the nil-UUID
sentinel rather than a fabricated owner.
* **Why two tables and not one with a discriminator.**
`content_derived_blobs` is keyed by the BLAKE3 of the SOURCE, so
identical content shares one derivation — which is exactly what
must NOT happen for user-supplied bytes, or one user's uploaded
preview would be served for every file with matching content.
That is the poisoning vector; the split is what prevents it, and
a `kind` column on a single table would not.
* **Worked examples**, which is what turns the rule into something
checkable:
- the same image uploaded twice → one `content_derived_blobs` row,
two files, one thumbnail blob;
- a PDF with a client-uploaded preview → one `file_attached_blobs`
row per file, duplicated on copy, never shared;
- a screenshot WebP cannot shrink → a negative row with NULL
`blob_hash`, and what a reader is expected to conclude from it;
- the same content transcoded and thumbnailed → two rows, same
`source_hash`, different `kind`.
* **Lifecycle**: which rows hold a reference (positive
`blob_hash` bumps `chunk_manifests.ref_count`; a negative row
holds none), what reaps them (`purge_derived_blobs` on source
reap; the decrement trigger on `file_attached_blobs`), and which
consistency job covers which failure — the point being that a
satellite row whose SOURCE is gone breaks no refcount invariant,
which is why `satellites_consistency` had to exist at all.
* **The NULL-handling trap**, worth a paragraph because it has
already bitten twice: SQL comparison against NULL is NULL, so
`EXISTS (… WHERE blob_hash = h)` silently excludes negative rows.
That is correct for refcounts (a negative row holds none) and
WRONG for dangling checks (it reported every negative row as
`data_loss`). Anything joining on `blob_hash` must decide which
of the two it wants.
Home: `docs/architecture/`, alongside `backend-storage.md`, which
already documents the blob layer these sit on top of. Operator-
facing rather than end-user, so schema and SQL are appropriate
here in a way they are not in `docs/guide/`.
Tracked separately, **not** part of this plan: the
`storage.copy_folder_tree` refcount bug (see the copy section). It is
a production data-loss bug on a path this plan doesn't otherwise