feat(consistency): satellites_consistency covers both tables, and the sweep covers every job
Extends the derived check to `file_attached_blobs` and renames it, since the two tables are one concept — the content-keyed and file-keyed halves of "things attached to a Blob" — and `storage.copy_file_satellites` already established the vocabulary. The attached half is the one that cannot be recovered. `attached_dangling_blob` is data_loss with `recoverable: false`: those bytes were user-supplied and have no server-side render path, so nothing can regenerate them. Its derived twin carries `recoverable: true`, because a derived artifact is a pure function of its source and re-rendering restores it. Same finding shape, materially different stakes, and the detail says which. No orphan-mapping check on the attached side, deliberately: `file_id` is ON DELETE CASCADE, so a row cannot outlive its file. The database enforces what the derived table cannot, since a content hash has no row to point a foreign key at — which is exactly why only that half could rot. One job walking two tables needs a phase in the cursor, or an attached checkpoint would be replayed against the derived table and silently re-scan or skip. Two things the sweep was missing, found while checking whether every consistency job is actually exercised: drives_consistency and folders_consistency were registered but never run by any test. Now included; the list is exhaustive by intent. An unknown job was a warning-and-skip. That protected feature-gated builds at the cost of something worse: this list said `derived_consistency` for one commit after the rename and would have dropped that coverage without a word, leaving the suite green over a check that no longer ran. It fails now.
This commit is contained in:
@@ -225,13 +225,22 @@ jsonpath "$.outcome.count" exists
|
||||
# Step 4c — Trigger `consistency_batch`. Coordinator (plain
|
||||
# JobHandler) — snapshots the registry, filters names
|
||||
# ending `_consistency`, sequentially triggers each.
|
||||
# `outcome.count` = number of children dispatched (6 as
|
||||
# of the refcount_cascade fix: drives + folders +
|
||||
# files + blobs + manifests + backend). `extra.per_check`
|
||||
# carries a per-child outcome
|
||||
# map. Batch itself always returns ok — child failures
|
||||
# live inside per_check. `?deep=true` propagates as
|
||||
# `extra.deep`.
|
||||
# `extra.per_check` carries a per-child outcome map. Batch
|
||||
# itself always returns ok — child failures live inside
|
||||
# per_check. `?deep=true` propagates as `extra.deep`.
|
||||
#
|
||||
# NO assertion on `outcome.count`. The batch auto-discovers
|
||||
# tenants via `.ends_with("_consistency")`, so a hardcoded
|
||||
# total breaks every time one is added — it broke on
|
||||
# `manifests_consistency` and again on
|
||||
# `satellites_consistency`, each time asserting arithmetic
|
||||
# rather than behaviour. Per the house rule: `contains` per
|
||||
# item, never a total.
|
||||
#
|
||||
# What matters is that every child SUCCEEDED, which
|
||||
# `err == 0` states directly and without a magic number, plus
|
||||
# a named check per tenant below so a job silently dropping
|
||||
# out of the batch is still caught.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
POST {{base_url}}/api/admin/jobs/consistency_batch/trigger?deep=true
|
||||
Authorization: Bearer {{admin_token}}
|
||||
@@ -240,9 +249,9 @@ HTTP 200
|
||||
[Asserts]
|
||||
jsonpath "$.ok" == true
|
||||
jsonpath "$.outcome.outcome" == "ok"
|
||||
jsonpath "$.outcome.count" == 6
|
||||
jsonpath "$.outcome.extra.deep" == true
|
||||
jsonpath "$.outcome.extra.ok" == 6
|
||||
# Zero failures, whatever the tenant count happens to be. `ok` is not
|
||||
# asserted against a number for the same reason `count` is not.
|
||||
jsonpath "$.outcome.extra.err" == 0
|
||||
# per_check is keyed by child job name. `manifests_consistency` was added
|
||||
# by the refcount_cascade fix — see docs/plan/derived-blobs.md and
|
||||
@@ -255,6 +264,11 @@ jsonpath "$.outcome.extra.per_check.files_consistency.outcome" == "ok"
|
||||
jsonpath "$.outcome.extra.per_check.blobs_consistency.outcome" == "ok"
|
||||
jsonpath "$.outcome.extra.per_check.manifests_consistency.outcome" == "ok"
|
||||
jsonpath "$.outcome.extra.per_check.backend_consistency.outcome" == "ok"
|
||||
# Finds satellite mappings whose Blob is gone — the one class every
|
||||
# refcount-based check above reports as healthy, because the row holds a
|
||||
# valid reference with an exactly correct count while pinning an artifact
|
||||
# that can never be reclaimed.
|
||||
jsonpath "$.outcome.extra.per_check.satellites_consistency.outcome" == "ok"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -527,24 +527,37 @@ fi
|
||||
# Zero findings is the assertion. These jobs are read-only, so a finding
|
||||
# here is a real invariant violation, not a repair opportunity.
|
||||
|
||||
# EVERY registered consistency tenant. Keep this list exhaustive: two of
|
||||
# these (drives, folders) were missing until 2026-08-28 and had never run
|
||||
# under test at all.
|
||||
CONSISTENCY_JOBS=(
|
||||
files_consistency
|
||||
folders_consistency
|
||||
drives_consistency
|
||||
blobs_consistency
|
||||
manifests_consistency
|
||||
backend_consistency
|
||||
# Catches what the others structurally cannot: a derived mapping whose
|
||||
# Catches what the others structurally cannot: a satellite mapping whose
|
||||
# source Blob is gone looks healthy to every refcount-based check — valid
|
||||
# reference, correct count, bytes present — while pinning its artifact
|
||||
# forever. That leak reached this suite as three unreclaimable blobs and
|
||||
# took four runs to identify.
|
||||
derived_consistency
|
||||
satellites_consistency
|
||||
)
|
||||
|
||||
CONSISTENCY_FAILED=0
|
||||
for job in "${CONSISTENCY_JOBS[@]}"; do
|
||||
# FAIL on an unknown job rather than warn-and-skip.
|
||||
#
|
||||
# The warning was there so a feature-gated build would not break, but the
|
||||
# cost is worse than the case it protects: renaming a job (or a typo)
|
||||
# silently removes it from the sweep, and the suite goes on reporting
|
||||
# green over a check that no longer runs. This list said
|
||||
# `derived_consistency` for exactly one commit after the rename and would
|
||||
# have skipped it without comment.
|
||||
TRIGGER=$(curl -sf -X POST -H "$AUTH" "$base_url/api/admin/jobs/$job/trigger") \
|
||||
|| { log "WARNING: $job not registered in this build — skipped"; continue; }
|
||||
[[ -z "$TRIGGER" ]] && { log "WARNING: $job returned an empty body — skipped"; continue; }
|
||||
|| fail "$job could not be triggered — renamed, unregistered, or a typo in CONSISTENCY_JOBS"
|
||||
[[ -z "$TRIGGER" ]] && fail "$job returned an empty body"
|
||||
|
||||
# The trigger is synchronous for these tenants, but the run row is what
|
||||
# carries the findings, so read it back rather than trusting the
|
||||
|
||||
Reference in New Issue
Block a user