Oxicloud/benches/ROUND9.md

Round 9 — decorator PUT reactivation, session/search/dedup alloc purges, PROPFIND join!, folder-level cascade

Benchmark-gated, same rule as ROUND2-8: every change ships with a BEFORE/AFTER benchmark and equivalence/safety gates; an AFTER that doesn't beat its BEFORE gets rolled back. The two decide-by-bench items this round (PROPFIND enrichment join!, folder binary-UUID) were adopted only after their gates passed; the authz change carries hard safety gates plus a new direct-grant-sibling isolation gate and was validated against the full authz-relevant unit suite.

Measured on 4 cores / 15 GiB, local PostgreSQL 16 (fsync off), release profile; frontend on Node 22 / vitest 4 (jsdom). Reproduce any row with the command in its section.

Summary

#	change	key metric	before → after
1	Blob decorators forward put_blob_from_bytes_unsynced	HEAD probes / wall, 500-chunk upload @10 ms RTT	500 → 0 probes; full stack 1571 → 812 ms (1.9x)
2	NC PROPFIND page enrichment triple → tokio::join!	p50 ms/page (500 children)	local 2.28 → 1.10 (2.07x); @5 ms RTT 22.1 → 7.7 (2.86x)
3	Search enrich consume+carry (Arc<str> result fields)	enrich_file ns/row · allocs/row	456 → 223 (2.0x) · 11.6 → 2.2; NC conversion 15.4 → 7.0 allocs/row
4	NC session end-to-end Arc (extractor/chroot/build)	allocs per authenticated NC request	extractor 8→0, chroot hit 4→0, build 11→6 (~17 fewer/req)
5	Storage micro-pack (create_new · manifest Arc · single-flight · hex)	see §5	fresh chunk writes 2.1x; 4097→0 allocs/read; herd 64→1 loads; 18→1 allocs/digest
6	OCS capabilities memoized (OnceLock<Bytes>)	50k polls wall · allocs/poll	269.6 → 1.1 ms (237x) · 102 → 0
7	Drive::is_empty COUNT(*) → EXISTS	ms/call, 100k-file drive	13.6 → 0.40 (34.4x)
8	favorites/recents row-map move (ROUND7 port)	allocs/row	12.00 → 9.25 (−2.75/row)
9	Folder rows: binary UUID decode (ROUND6 port)	500-row page mean	1.06–1.10 → 1.03–1.04 ms (1.03–1.07x, first run a wash — see §9)
10	Folder-level cascade decision (authz, ROUND8 deferred)	cold first view µs/thumb (100-photo album)	592 → 418 (1.42x); warm 1.33 µs unchanged
11	SPA: resolveLabel O(C)→O(1) index	50 frames × 30 rows @ 5k contacts	11.0 → 0.8 ms (13.9x); comparisons rows×C → C
12	SPA: selection-prune guard + matchMedia hoist	per-page Set builds / matchMedia calls	100 → 0 · P → 1

[1] Blob decorators — the trait-default fallthrough was re-adding HEAD-before-PUT

ROUND3 §8 made chunk writes skip the remote exists-probe by introducing put_blob_from_bytes_unsynced (content-addressed keys make re-PUTs overwrite-safe). But RetryBlobBackend and CachedBlobBackend never overrode it, so the trait default routed every decorated _unsynced call back through the probing put_blob_from_bytes — silently reinstating HEAD+PUT per chunk on every remote deployment with retry or cache enabled (the recommended object-store setup). EncryptedBlobBackend and MigrationBlobBackend already forwarded correctly.

Both decorators now forward put_blob_from_bytes_unsynced and sync_blobs to their inner backend (Retry wraps the former in its retry loop; the durability sweep is deliberately NOT retried — a failed fsync must surface, not be re-issued after the kernel may have dropped the dirty pages). CachedBlobBackend keeps its local write-through population on the unsynced path (shared cache_bytes_write_through helper, no eviction sweep — matching the historical write-path behavior) so post-upload readers (thumbnail/EXIF/face hooks) still hit the cache.

cargo run --release --features bench --example bench_s3_put
# 500 x 256 KiB chunk PUTs at concurrency 8, 10 ms/request stub
# [1] raw backend    BEFORE 1519 ms (500 HEADs) → AFTER 765 ms (0)   2.0x
# [3] retry(s3)      BEFORE 1524 ms (500 HEADs) → AFTER 766 ms (0)   2.0x
#     cache(s3)      BEFORE 1535 ms (500 HEADs) → AFTER 803 ms (0)   1.9x
#     cache(enc(retry(s3)))  1571 ms (500)      →       812 ms (0)   1.9x
# gates: BEFORE probes == chunks, AFTER probes == 0, cache write-through
#        populated on BOTH routes (2×chunks files present)

[2] NC PROPFIND page enrichment — 3 serial round-trips → tokio::join!

Every Depth:1 PROPFIND page enriches its ≤500 children with three INDEPENDENT batched reads (favorites = ANY, oc:fileid = ANY, dead props = ANY), previously awaited in sequence. This is the round-7 deferred "serial pairs" item, and the one pair the round-7 notes ranked worth gating (3 round-trips, per page, on the hottest sync path).

Decide-by-bench with injected per-round-trip latency (0/0.25/1/5 ms), because ROUND6 showed concurrency can LOSE on local-socket PG (the authz try_join_all rejection). It doesn't here — these are three fat batched queries whose server-side execution parallelizes across PG backends, so even the local-socket floor wins, not just the RTT overlap:

cargo run --release --features bench --example bench_nc_enrich_join
# children=500, passes=100, p50 ms/page      serial    join!   ratio
#   0 µs injected                             2.275    1.097   2.07x
#   250 µs                                    6.273    2.481   2.53x
#   1000 µs                                   9.163    3.441   2.66x
#   5000 µs                                  22.050    7.709   2.86x
# gate: identical favorite sets / id maps / dead-prop rows; adoption
#       required no local-socket regression — it's a 2x win even there

Contrast with ROUND6 §8 (rejected): that fan-out issued ~200 single-row authz checks through the engine's cache layers; this overlaps exactly 3 page-batched queries. Both files' and folders' page loops adopted it.

[3] Search enrichment — borrow+clone+reclassify → consume+carry

enrich_file took &FileDto, cloned every owned String out of it, and RE-RAN the three display classifiers whose results the DTO already carried interned (Arc<str>, computed once in FileDto::from); the recursive branch maps the ENTIRE pre-pagination match set. The NC REPORT conversion (file_dto_from_search) then re-ran all three classifiers a SECOND time per emitted row. SearchFileResultDto.{mime_type,icon_class, icon_special_class,category} are now Arc<str> (#[schema(value_type = String)] keeps the OpenAPI shape; JSON output byte-identical), both enrichers consume their DTO, the intermediate Vec<FileDto>/Vec<FolderDto> materializations are fused away, suggest reuses the interned fields, and the NC conversion carries them (refcount bumps). The search-cache byte weigher keeps counting .len() per row — now an over-count of shared bytes, i.e. the conservative direction.

cargo run --release --features bench --example bench_search_enrich
# rows=10000 passes=50 (p50 ns/row; allocs from pass 0)
# [1] enrich_file   BEFORE 455.8 ns / 11.60 allocs → AFTER 222.7 / 2.20
# [2] enrich_folder BEFORE 116.2 ns /  5.00 allocs → AFTER 127.6 / 1.00
#     (folder wall flat: the AFTER window absorbs the input drop the
#      BEFORE arm defers outside its timing; the alloc gate is the win)
# [3] NC conversion BEFORE 2.700 ms / 15.40 allocs → AFTER 1.524 / 7.00
# gates: 500 files + 500 folders field-identical; NC conversion
#        field-identical vs a fresh classifier run

[4] NC session — deep-clone per request → Arc end-to-end

Every authenticated NC request paid: the extractor's (**arc).clone() — a DEEP clone of NcSession (~8-9 String allocs) despite its doc claiming "one Arc increment"; a chroot-cache hit cloning the stored FolderDto by value (~5 allocs, moka get clones V); and a session build that cloned CurrentUser for the extension, cloned raw_username, and to_stringed the span value. Now: NC_CHROOT_CACHE stores Arc<FolderDto>, NcSession.user is the same Arc<CurrentUser> the extension holds, raw_username moves, the span renders lazily (field::display, the ROUND5 §7 pattern the NC path had missed), and handlers extract SharedNcSession — an Arc handle that derefs to NcSession, so the 64 field-access sites are untouched.

cargo run --release --features bench --example bench_nc_session
# 100k iterations                          wall ms   allocs/op
# [1] extractor  BEFORE deep clone           17.0      8.000
#                AFTER  SharedNcSession       4.2      0.000   (4.0x)
# [2] chroot hit BEFORE FolderDto value      21.3      4.000
#                AFTER  Arc<FolderDto>       11.7      0.000   (1.8x)
# [3] build      BEFORE clone×2 + span       17.6     11.000
#                AFTER  shared Arc           11.8      6.000   (1.5x)
# gate: every field handlers consume identical (incl. the URL-user check)

[5] Storage micro-pack

Four independent A/Bs in one harness (bench_storage_micro, no Postgres):

• (a) Local chunk write — try_exists (stat) + File::create → one atomic create_new open; AlreadyExists IS the idempotent skip. 20k × 4 KiB fresh writes 2707 → 1286 ms (2.1x); re-put skips 1.08x.
• (b) CDC read prep — stream_chunks took Vec<String>, forcing every read to deep-clone the cached manifest's whole hash list before the first byte; now it takes the manifest Arc and indexes. A 4096-chunk manifest × 200 reads: 819 400 → 0 allocs, 49.4 → 0.16 ms. The Range path selects by index too — a bytes=0- probe of an N-chunk video no longer clones N hashes.
• (c) Manifest miss herd — manifest_cached used get→insert; K concurrent cold readers each ran the SELECT. Now fast-get + try_get_with (sentinel miss error keeps the positive-only contract — moka never caches loader errors, so legacy blobs and DB failures stay uncached). Herd of 64: 64 → 1 loads.
• (d) Chunk Content-MD5 hex — the last format!("{b:02x}")-per-byte straggler (ROUND6 §7 shipped hex_lower); 18 → 1 allocs/digest, 10x.

cargo run --release --features bench --example bench_storage_micro

[6] OCS capabilities — rebuilt per poll → memoized bytes

/ocs/v{1,2}.php/cloud/capabilities is process-invariant (pure config), yet every poll re-built the ~40-node json! tree, re-read OXICLOUD_BASE_URL from the environment, ran three format!s and re-serialized. Both versions now serialize once into OnceLock<[Bytes; 2]>; a poll is a refcount bump. The payload builder takes its three config inputs directly (testable without AppState).

cargo run --release --features bench --example bench_capabilities_static
# 50k polls   BEFORE 269.6 ms / 102 allocs/poll → AFTER 1.1 ms / 0  (237x)
# gate: served bytes byte-identical for v1 and v2

[7] Drive::is_empty — full-drive COUNT(*) sum → EXISTS OR EXISTS

The deletion precheck only needs a boolean, but aggregated every live folder + file in the drive. EXISTS stops at the first row.

cargo run --release --features bench --example bench_drive_is_empty
# populated (100k files)  13.615 → 0.396 ms  (34.4x)
# empty                    0.219 → 0.166 ms  (1.3x)
# gate: identical booleans on both data shapes

[8] favorites/recents row-map — the ROUND7 move that never got ported

ROUND7 §3 removed the per-row name clone in /folders/{id}/resources; the same mapping in /api/favorites/resources and /api/recent/resources still cloned path + name + blob_hash per row (and folder_handler kept one blob_hash clone). All moved now — display classes computed before name moves, path/blob_hash moved instead of cloned.

cargo run --release --features bench --example bench_resource_row_map
# [2] favorites/recents shape, rows=500
# BEFORE (clone) 12.004 allocs/row → AFTER (move) 9.254  (−2.75/row)
# gate: (name, path, content_hash, icon_class, category) identical per row

[9] Folder rows — binary UUID decode (the ROUND6 §10 port)

ROUND6 adopted binary-UUID decode for file listing rows (1.17x) and queued "other repos with the same shape"; FolderDbRepository never got it. All folder-row queries (list_folders_batch — every Depth:1 PROPFIND subfolder page — get_folder, descendants, search, suggest, and the write-path RETURNINGs, which share row_to_folder) now decode id/parent_id as binary Uuid (16 B vs 36 B on the wire, no server cast) and render once app-side. Param casts ($3::text IS NULL), enum casts and the ltree path::text renders are untouched.

Honest verdict: weaker than the file side. Four interleaved runs: 1.00x (wash), 1.05x, 1.03x, and 1.07x at 1000 rows — folder rows are thinner than file rows, so the two casts are a smaller fraction of the page. Adopted on the consistent small win + growth with page size + the wire-bytes reduction; the first-run wash is inside the noise band.

cargo run --release --features bench --example bench_folder_uuid_decode
# rows/page=500 passes=400 (interleaved)   mean      p50      p95
# A ::text (before)                        1.061    1.039    1.310
# B binary (after)                         1.027    1.012    1.269   1.03x
# rows/page=1000: 1.758 → 1.639 mean                              1.07x
# gate: identical (id, name, path, parent_id) tuples

[10] Authz — folder-level cascade decision (the ROUND8 deferred item)

ROUND8 memoised the per-file cascade decision, fixing revalidation; a shared N-photo album's cold first view still ran N near-identical ltree ancestor queries. The file decision now decomposes into exactly the two branches of the historical UNION: parent point-read (new file_parent_cache, 30 s TTL — grant writes don't alter parentage; moves are the same TTL-healed indirect path as before) → the FOLDER cascade decision (one ltree query per folder, shared by every sibling via the existing cascade_grant_cache, recursing into the Folder arm) → a direct-file-grant point lookup only when the folder half denies. The old UNION query is deleted; no decision changes, including the parentless edge (folder_id IS NOT NULL guard ≡ direct-only fallback).

Safety gates (hard asserts): recipient allowed on every file, outsider denied, clear_role revoke denies IMMEDIATELY (the flush covers file and folder decisions — same cache), and NEW: a caller holding only a direct grant on one file is allowed that file and denied its siblings — proving the folder-level decomposition neither shadows direct grants nor leaks a file decision across siblings.

cargo run --release --features bench --example bench_thumbnail_cascade_cache
# thumbs=100 (folder-grant recipient, no drive membership)
# ROUND8 cold (union/file)      59.19 ms    591.91 µs/thumb
# AFTER  cold (first view)      41.77 ms    417.73 µs/thumb   (1.42x)
# AFTER  warm (revalidation)     0.13 ms      1.33 µs/thumb   (unchanged)

The first view is now bounded by the per-file parent PK reads (cheap, but still N point queries) + 1 ltree query — batching the parent resolution per page would need a wider API change; noted for a future round.

[11] SPA — resolveLabel linear directory scan → id-keyed index

resolveLabel/resolveRecipient ran contactCache.find(...) — a linear scan over the whole system address book — once per rendered grant row / lane header on /shared, re-rendering on every page and role change: O(rows × directory). Now a Map<id, Contact> built once per cache identity (exactly like the existing groupCache).

cd frontend && npx vitest run src/lib/api/endpoints/recipients.bench.test.ts --disable-console-intercept
# 50 frames × 30 rows @ C=5000: before 11.0 ms, after 0.8 ms (13.9x)
# gates: labels identical (present + absent ids); comparisons rows×C → C

[12] SPA — selection-prune guard + photos matchMedia hoist

• ResourceList's prune $effect built an O(N) id Set on every infinite-scroll page even with nothing selected; guarded with selected.size === 0 (reactive, so it re-arms when a selection appears). 100-page drain: 100 → 0 Set builds; pruned result identical when a selection exists.
• The photos timeline derive called window.matchMedia(...) per recompute (every 60-photo page); hoisted to state fed by one MediaQueryList change listener. P recomputes: P → 1 calls, identical booleans, crossings propagate.

cd frontend && npx vitest run src/lib/components/listDerives.bench.test.ts --disable-console-intercept

Deferred / flagged (not shipped this round)

• CalDAV authz-before-fetch reorder (calendar_service::get_event / list_events / by-uid fetch the calendar row before the authz check only to read .is_public; running the already-required authz first and fetching only on denial saves one SELECT per authorized private-calendar read). Behavior-preserving (the OR commutes) but it reorders an authz check relative to a data fetch — flagged for maintainer sign-off per the authz-change convention, with the bench sketch in this round's notes.
• Per-page batched parent resolution for §10 — would cut the cold first view's N parent PK reads to one = ANY per page; needs a wider engine API (batch check) — future round.
• batch_operations Arc<str> → Option<&str> widening (ROUND7 deferred) — re-audited: 1 small alloc/item vs a per-item DB roundtrip; still not worth the 2-trait/7-site churn alone. Standing verdict.
• JWT-claims Arc<str> (ROUND6 deferred) — still open; touches serde rc on TokenClaims + dozens of read sites. The 2 allocs/request remain the cheapest known win on the /api path for a future round.

Correctness-adjacent (surfaced by the round-9 hunt — not perf)

• trash_service.rs restore matches error text (format!("{}", e).contains("not found")) instead of e.kind == ErrorKind::NotFound — fragile to rewording; flagged.
• The round-7 flags remain open: fetchFolderListing seeds empty favoriteIds/sharedIds; the search page still lacks a stale-response guard.