Edouard Vanbelle
2b52d233f0
`AzureBlobBackend` inherited the trait's `operation_not_supported`
default for `list_blob_hashes`, so every `backend_consistency` run on
Azure fell back to a per-row probe. That fallback walks `storage.blobs`
asking "are these bytes there", which structurally cannot find orphans:
bytes no row claims are invisible to anything starting from the
database, because you need a hash to ask about one and discovering
unknown hashes IS enumeration. Azure had half the coverage of local and
S3, in the direction that wastes space.
## Enumeration
The obstacle was the cursor contract. The caller advances ONE cursor
across both sides of the merge-join, feeding the same value to the
backend and to `WHERE hash > $1`, so the cursor IS a blob hash. S3
satisfies that with `StartAfter`. Azure has no equivalent on this SDK:
REST 2023-05-03 added `startFrom`, but `azure_storage_blobs` 0.21 never
sends it — `ListBlobs` exposes only prefix, delimiter, max_results and
an opaque marker that cannot be derived from a hash.
Resume rides on `prefix` instead. Names are `{hash[0..2]}/{hash}.blob`,
which partitions the container into 256 shards that are themselves in
hash order, so walking 00/…ff/ yields exactly the global order the
merge-join needs and a cursor names the shard to restart in.
Re-listing on resume is bounded by shard width rather than by the whole
container — the cost a client-side skip over a flat listing would pay on
every page. `marker` pages within one call and never escapes as the
cursor, the same treatment the S3 impl gives its continuation token.
One asymmetry against S3, deliberate: constraining to `{2-hex}/` means
foreign files outside that shape never reach `unknowns`. Safe in the
direction that matters — an orphan is a blob we wrote and stopped
referencing, so it always has the canonical name — and it buys O(N)
enumeration instead of O(N²/limit).
`hash_from_blob_name` mirrors S3's parser, shard-equals-prefix check
included: without it a mis-sharded name would round-trip to a
`blob_name` we never wrote, reporting a live blob no read path can find.
Tested for round-trip, for nine non-canonical shapes, and for the
ordering premise the merge-join rests on.
## Removing the fallback
With Azure enumerating, nothing shipped answers
`operation_not_supported`. The fallback's other stated justification —
mid-migration — never applied: it named a `MigrationBlobBackend` that
does not exist, and `SwappableBlobBackend::list_blob_hashes` forwards to
whatever is currently active, as do the Encrypted, Cached and Retry
wrappers.
What still reached it was a transient failure (auth blip, throttle,
network) relabelled as a capability limit, on a run that then read as
clean while having silently lost orphan coverage. So it was not merely
dead, it produced the wrong outcome — the only one it could. It also had
zero test coverage across 227 lines.
Now any `Err` from `list_blob_hashes` fails the run. That is louder than
an anomaly on a green run, which was the fallback's own goal. The trait
default still returns `operation_not_supported`, so a genuinely
unenumerable backend would fail every run — detectable, not silent, and
the point at which to bring the fallback back with tests.
## Also here
A doc note on `get_blob_range_stream` explaining why the Azurite
migration hang is not worked around: `azure_core` 0.21's
`Range::as_headers` attaches `x-ms-range-get-content-crc64` to any range
under 4 MiB with no opt-out, Azurite 500s on it, and `azure_core`
retries a deterministic error forever. The reachable path is
`backend_migration` → `EncryptedBlobBackend::head_check` →
`get_blob_range_stream(hash, 0, HEADER_SIZE)` — a pre-write probe on the
TARGET, so it fires on the first blob while `migration_readonly` refuses
writes app-wide. Working around it would trade production read
amplification for emulator support; the fix is the official SDK, where
`range_get_content_crc64` is an explicit field.
`RUSTSEC-2026-0275` is ignored on the same reasoning — `azure_core` 0.21
logs the `authorization` header at debug, the advisory's "upgrade to
>=0.22.0" names a version that does not exist, and the real remedy is
that same migration. Reachable only via an explicit
`RUST_LOG=…,azure_core=debug`; the entry says not to run that against a
real account.
`docs/plan/jobs-handling-recoverable-error.md` covers the other half —
a bounded retry should have turned that hang into a Paused run with a
reason, whatever the SDK does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
157 lines
8.6 KiB
TOML
157 lines
8.6 KiB
TOML
[advisories]
|
|
ignore = [
|
|
# paste unmaintained — transitive via azure_core 0.21.0 (latest available).
|
|
# No direct security impact; no upgrade path exists.
|
|
# keep warning "RUSTSEC-2024-0436",
|
|
|
|
# rand 0.7.3 unsound — transitive via http-types → azure_core 0.21.0.
|
|
# Only exploitable with a custom logger using rand::rng(); not applicable here.
|
|
"RUSTSEC-2026-0097",
|
|
|
|
# RUSTSEC-2023-0071 (Marvin Attack): rsa crate is a transitive dependency from jsonwebtoken.
|
|
# Not affected: This application uses HS256 for internal JWT signing and only performs
|
|
# RSA public key verification (not private key operations) for OIDC/OAuth2 tokens.
|
|
"RUSTSEC-2023-0071",
|
|
|
|
# rustls-webpki 0.101.7 — three advisories, all transitive via AWS SDK → rustls 0.21.12.
|
|
# aws-sdk-s3 1.x (latest) has not yet migrated to rustls 0.23.x; no upgrade path available.
|
|
# Severity: low — exploitation requires either a rogue CA or a MitM on AWS S3 infrastructure.
|
|
# RUSTSEC-2026-0104: DoS via panic in CRL parsing (only if CRLs are processed).
|
|
# RUSTSEC-2026-0098: URI name constraints ignored (requires CA misissuance).
|
|
# RUSTSEC-2026-0099: Wildcard name constraint bypass (requires CA misissuance).
|
|
"RUSTSEC-2026-0104",
|
|
"RUSTSEC-2026-0098",
|
|
"RUSTSEC-2026-0099",
|
|
|
|
# h2 0.3.27 — RUSTSEC-2026-0258 "unbounded empty DATA frames"
|
|
# (GHSA-q83h-524g-xf6h). Transitive via aws-smithy-http-client 1.1.12
|
|
# → hyper 0.14.32 → h2 0.3.27. The patched line is 0.4.16+, but hyper
|
|
# 0.14's `h2 = "0.3"` requirement pins us to the 0.3.x branch which
|
|
# will not receive a backport — real fix requires aws-smithy-http-client
|
|
# to migrate to hyper 1.x (which our other h2 copy — 0.4.16, already
|
|
# bumped — is on). The 0.4.x copy is fixed via `cargo update`; this
|
|
# ignore covers only the 0.3.x chain.
|
|
#
|
|
# Severity: low (advisory's own classification). Attack is empty-DATA-
|
|
# frame flooding by a malicious HTTP/2 peer → memory pressure or panic.
|
|
# In this codebase h2 0.3.x runs strictly on the CLIENT side of AWS
|
|
# SDK requests to S3 endpoints. Exploitation requires either
|
|
# compromising AWS S3 (out-of-scope) or MitM with a valid TLS cert
|
|
# for the configured S3 host (bigger problem than the DoS). No
|
|
# data-integrity or auth impact; panic path contained by
|
|
# request-level unwind.
|
|
#
|
|
# Un-ignore trigger: aws-smithy-http-client releases a version that
|
|
# switches to hyper 1.x (checkable with `cargo tree -i h2@0.3` — the
|
|
# command returns no rows once the chain is gone). Track upstream at
|
|
# https://github.com/smithy-lang/smithy-rs/issues (search "hyper 1").
|
|
"RUSTSEC-2026-0258",
|
|
|
|
# instant unmaintained — transitive via azure_core 0.21.0 (latest available).
|
|
# No direct security impact; no upgrade path exists.
|
|
"RUSTSEC-2024-0384",
|
|
|
|
# azure_core 0.21.0 writes the `authorization` header value to logs —
|
|
# `policies/transport.rs` does `debug!("…{request:#?}")` over the whole
|
|
# request. For a SharedKey entry that value is the per-request HMAC
|
|
# signature; for a SAS entry it is the token. Severity 6.5 (medium).
|
|
#
|
|
# The advisory says "upgrade to >=0.22.0". That version does not exist:
|
|
# `azure_core` jumped 0.21 → 1.x, and `azure_storage_blobs` never left
|
|
# 0.21.0 before being archived. So the stated remedy IS the official-SDK
|
|
# migration, tracked separately alongside the quick-xml pair above.
|
|
#
|
|
# Not reachable at our log levels: the line is `debug!` on the
|
|
# `azure_core::policies::transport` target, and the default filter is
|
|
# `info`. It fires only if an operator explicitly asks for
|
|
# `RUST_LOG=…,azure_core=debug`, which is not hypothetical — that is the
|
|
# invocation used to diagnose the Azurite migration hang. **Do not run
|
|
# `azure_core=debug` against a real Azure account**; it prints request
|
|
# signatures to the terminal. Against Azurite it only exposes the
|
|
# published dev key's signatures.
|
|
#
|
|
# A subscriber-level directive pinning that target off was prototyped
|
|
# and rejected 2026-09-02 — not worth carrying a filter hack for a
|
|
# dependency being replaced.
|
|
#
|
|
# Un-ignore trigger: the azure_storage_blob 1.x migration lands
|
|
# (`cargo tree -i azure_core@0.21` returns no rows).
|
|
"RUSTSEC-2026-0275",
|
|
|
|
# quick-xml 0.31.0 — transitive via azure_core 0.21.0 (unofficial SDK,
|
|
# now archived). Our direct dep is already on 0.41.0; the 0.31 copy is
|
|
# only reachable through the azure_storage_blobs chain, which parses
|
|
# XML responses received from Azure Storage over TLS. Neither CVE is
|
|
# exploitable without attacker-controlled XML, so the vector requires
|
|
# MitM of the TLS channel to Azure (or a compromised storage
|
|
# endpoint). Real fix is migrating to the official azure_core 1.0 /
|
|
# azure_storage_blob 1.0 SDK — tracked separately.
|
|
# RUSTSEC-2026-0195: unbounded ns-declaration allocation → mem-DoS
|
|
# RUSTSEC-2026-0194: quadratic dup-attribute check → CPU-DoS
|
|
"RUSTSEC-2026-0195",
|
|
"RUSTSEC-2026-0194",
|
|
|
|
# wasmtime 43.0.2 — "Stores can mix up type indices between engines"
|
|
# (GHSA-hgjw-h833-99q9). Transitive via extism 1.30.0 (latest published;
|
|
# extism `main` still pins wasmtime 43, no upgrade path). The advisory
|
|
# has no patched 43.x — fix requires wasmtime >=46.0.2 or >=47.0.3, and
|
|
# forcing that via [patch.crates-io] would break extism (three major
|
|
# wasmtime API bumps between 43 and 46). Real fix waits on extism
|
|
# upstream to migrate.
|
|
#
|
|
# Runtime exposure is zero in default deployments:
|
|
# - `plugins` is an OPT-IN build feature; default builds and the CI
|
|
# release binary don't link wasmtime at all.
|
|
# - Runtime activation additionally requires OXICLOUD_ENABLE_PLUGINS=true.
|
|
# - Plugin binaries are ADMIN-SUPPLIED, not attacker input.
|
|
# - The advisory's attack pattern is multi-Engine Store-sharing;
|
|
# OxiCloud's plugin runtime creates one fresh Plugin per invocation
|
|
# with its own Store (see infrastructure/services/plugins/runtime.rs).
|
|
"RUSTSEC-2026-0222",
|
|
|
|
# wasmtime 43.0.2 — "Filesystem sandbox escape when paths or symlinks
|
|
# contain trailing slashes" (RUSTSEC-2026-0269, 8.8 high). Same crate,
|
|
# same chain and same absent upgrade path as RUSTSEC-2026-0222 above:
|
|
# extism 1.30.0 is the latest published and pins wasmtime 43, while the
|
|
# advisory's fixed releases are >=24.0.13 <25, >=36.0.14 <37,
|
|
# >=46.0.3 <47, >=47.0.4 — none in the 43.x line, so there is no
|
|
# version satisfying extism's requirement that carries the fix.
|
|
#
|
|
# NOT REACHABLE, and for a stronger reason than the build-feature
|
|
# gating: this is a WASI filesystem sandbox escape, and OxiCloud's
|
|
# plugin runtime gives plugins no filesystem to escape from.
|
|
# `plugins/runtime.rs::compile` builds every plugin with
|
|
# `.with_wasi(false)` and declares no `allowed_paths`, so there are no
|
|
# preopened directories — the escape needs one to traverse out of.
|
|
# `.disallow_all_hosts()` removes outbound network on the same path.
|
|
#
|
|
# The build-level gating from the entry above still applies on top:
|
|
# `plugins` is opt-in and absent from `default`, so the CI release
|
|
# binary does not link wasmtime; runtime activation additionally needs
|
|
# OXICLOUD_ENABLE_PLUGINS=true; and plugin binaries are admin-supplied,
|
|
# not attacker input.
|
|
#
|
|
# Un-ignore trigger: EITHER extism releases a version on wasmtime
|
|
# >=46.0.3 (check with `cargo tree -i wasmtime --features plugins`),
|
|
# OR `plugins/runtime.rs` gains `allowed_paths` / `with_wasi(true)` —
|
|
# at which point this stops being unreachable and blocks release
|
|
# rather than being ignored.
|
|
"RUSTSEC-2026-0269",
|
|
|
|
# astral-tokio-tar 0.5.6 — tar extraction advisories, transitive via
|
|
# testcontainers → testcontainers-modules, a DEV-dependency used only by
|
|
# the `--cfg integration_tests` harness to spin up throwaway Postgres
|
|
# containers. Never compiled into the production binary and never fed
|
|
# attacker-controlled tar input — the only archives extracted are the
|
|
# official Postgres images the test harness pulls. testcontainers 0.25.2
|
|
# pins astral-tokio-tar to ^0.5, so the fixed 0.6.x is not reachable
|
|
# without a testcontainers major bump (tracked separately).
|
|
# RUSTSEC-2026-0145 / -0112: PAX header desynchronization
|
|
# RUSTSEC-2026-0113: unpack_in symlink-following chmod
|
|
# RUSTSEC-2026-0066: insufficient PAX extension validation (low)
|
|
"RUSTSEC-2026-0145",
|
|
"RUSTSEC-2026-0112",
|
|
"RUSTSEC-2026-0113",
|
|
"RUSTSEC-2026-0066",
|
|
]
|