Oxicloud/benches/ROUND9.md

# Round 9 — decorator PUT reactivation, session/search/dedup alloc purges, PROPFIND `join!`, folder-level cascade

Benchmark-gated, same rule as ROUND2-8: every change ships with a
BEFORE/AFTER benchmark and equivalence/safety gates; an AFTER that doesn't
beat its BEFORE gets rolled back. The two decide-by-bench items this round
(PROPFIND enrichment `join!`, folder binary-UUID) were adopted only after
their gates passed; the authz change carries hard safety gates plus a new
direct-grant-sibling isolation gate and was validated against the full
authz-relevant unit suite.

Measured on 4 cores / 15 GiB, local PostgreSQL 16 (fsync off), release
profile; frontend on Node 22 / vitest 4 (jsdom). Reproduce any row with the
command in its section.

## Summary

| # | change | key metric | before → after |
|--:|---|---|---|
| 1 | Blob decorators forward `put_blob_from_bytes_unsynced` | HEAD probes / wall, 500-chunk upload @10 ms RTT | 500 → 0 probes; full stack 1571 → 812 ms (**1.9x**) |
| 2 | NC PROPFIND page enrichment triple → `tokio::join!` | p50 ms/page (500 children) | local 2.28 → 1.10 (**2.07x**); @5 ms RTT 22.1 → 7.7 (**2.86x**) |
| 3 | Search enrich consume+carry (`Arc<str>` result fields) | enrich_file ns/row · allocs/row | 456 → 223 (**2.0x**) · 11.6 → 2.2; NC conversion 15.4 → 7.0 allocs/row |
| 4 | NC session end-to-end `Arc` (extractor/chroot/build) | allocs per authenticated NC request | extractor 8→0, chroot hit 4→0, build 11→6 (**~17 fewer/req**) |
| 5 | Storage micro-pack (create_new · manifest Arc · single-flight · hex) | see §5 | fresh chunk writes **2.1x**; 4097→0 allocs/read; herd 64→1 loads; 18→1 allocs/digest |
| 6 | OCS capabilities memoized (`OnceLock<Bytes>`) | 50k polls wall · allocs/poll | 269.6 → 1.1 ms (**237x**) · 102 → 0 |
| 7 | `Drive::is_empty` COUNT(*) → `EXISTS` | ms/call, 100k-file drive | 13.6 → 0.40 (**34.4x**) |
| 8 | favorites/recents row-map move (ROUND7 port) | allocs/row | 12.00 → 9.25 (**−2.75/row**) |
| 9 | Folder rows: binary UUID decode (ROUND6 port) | 500-row page mean | 1.06–1.10 → 1.03–1.04 ms (**1.03–1.07x**, first run a wash — see §9) |
| 10 | Folder-level cascade decision (authz, ROUND8 deferred) | cold first view µs/thumb (100-photo album) | 592 → 418 (**1.42x**); warm 1.33 µs unchanged |
| 11 | SPA: `resolveLabel` O(C)→O(1) index | 50 frames × 30 rows @ 5k contacts | 11.0 → 0.8 ms (**13.9x**); comparisons rows×C → C |
| 12 | SPA: selection-prune guard + `matchMedia` hoist | per-page Set builds / matchMedia calls | 100 → 0 · P → 1 |

## [1] Blob decorators — the trait-default fallthrough was re-adding HEAD-before-PUT

ROUND3 §8 made chunk writes skip the remote exists-probe by introducing
`put_blob_from_bytes_unsynced` (content-addressed keys make re-PUTs
overwrite-safe). But `RetryBlobBackend` and `CachedBlobBackend` never
overrode it, so the **trait default** routed every decorated `_unsynced`
call back through the probing `put_blob_from_bytes` — silently reinstating
HEAD+PUT per chunk on every remote deployment with retry or cache enabled
(the recommended object-store setup). `EncryptedBlobBackend` and
`MigrationBlobBackend` already forwarded correctly.

Both decorators now forward `put_blob_from_bytes_unsynced` and `sync_blobs`
to their inner backend (Retry wraps the former in its retry loop; the
durability sweep is deliberately NOT retried — a failed fsync must surface,
not be re-issued after the kernel may have dropped the dirty pages).
`CachedBlobBackend` keeps its local write-through population on the
unsynced path (shared `cache_bytes_write_through` helper, no eviction sweep
— matching the historical write-path behavior) so post-upload readers
(thumbnail/EXIF/face hooks) still hit the cache.

```
cargo run --release --features bench --example bench_s3_put
# 500 x 256 KiB chunk PUTs at concurrency 8, 10 ms/request stub
# [1] raw backend    BEFORE 1519 ms (500 HEADs) → AFTER 765 ms (0)   2.0x
# [3] retry(s3)      BEFORE 1524 ms (500 HEADs) → AFTER 766 ms (0)   2.0x
#     cache(s3)      BEFORE 1535 ms (500 HEADs) → AFTER 803 ms (0)   1.9x
#     cache(enc(retry(s3)))  1571 ms (500)      →       812 ms (0)   1.9x
# gates: BEFORE probes == chunks, AFTER probes == 0, cache write-through
#        populated on BOTH routes (2×chunks files present)
```

## [2] NC PROPFIND page enrichment — 3 serial round-trips → `tokio::join!`

Every Depth:1 PROPFIND page enriches its ≤500 children with three
INDEPENDENT batched reads (favorites `= ANY`, oc:fileid `= ANY`, dead
props `= ANY`), previously awaited in sequence. This is the round-7
deferred "serial pairs" item, and the one pair the round-7 notes ranked
worth gating (3 round-trips, per page, on the hottest sync path).

Decide-by-bench with injected per-round-trip latency (0/0.25/1/5 ms),
because ROUND6 showed concurrency can LOSE on local-socket PG (the authz
`try_join_all` rejection). It doesn't here — these are three fat batched
queries whose **server-side execution** parallelizes across PG backends,
so even the local-socket floor wins, not just the RTT overlap:

```
cargo run --release --features bench --example bench_nc_enrich_join
# children=500, passes=100, p50 ms/page      serial    join!   ratio
#   0 µs injected                             2.275    1.097   2.07x
#   250 µs                                    6.273    2.481   2.53x
#   1000 µs                                   9.163    3.441   2.66x
#   5000 µs                                  22.050    7.709   2.86x
# gate: identical favorite sets / id maps / dead-prop rows; adoption
#       required no local-socket regression — it's a 2x win even there
```

Contrast with ROUND6 §8 (rejected): that fan-out issued ~200 single-row
authz checks through the engine's cache layers; this overlaps exactly 3
page-batched queries. Both files' and folders' page loops adopted it.

## [3] Search enrichment — borrow+clone+reclassify → consume+carry

`enrich_file` took `&FileDto`, cloned every owned String out of it, and
RE-RAN the three display classifiers whose results the DTO already carried
interned (`Arc<str>`, computed once in `FileDto::from`); the recursive
branch maps the ENTIRE pre-pagination match set. The NC REPORT conversion
(`file_dto_from_search`) then re-ran all three classifiers a SECOND time
per emitted row. `SearchFileResultDto.{mime_type,icon_class,
icon_special_class,category}` are now `Arc<str>` (`#[schema(value_type =
String)]` keeps the OpenAPI shape; JSON output byte-identical), both
enrichers consume their DTO, the intermediate `Vec<FileDto>`/`Vec<FolderDto>`
materializations are fused away, suggest reuses the interned fields, and
the NC conversion carries them (refcount bumps). The search-cache byte
weigher keeps counting `.len()` per row — now an over-count of shared
bytes, i.e. the conservative direction.

```
cargo run --release --features bench --example bench_search_enrich
# rows=10000 passes=50 (p50 ns/row; allocs from pass 0)
# [1] enrich_file   BEFORE 455.8 ns / 11.60 allocs → AFTER 222.7 / 2.20
# [2] enrich_folder BEFORE 116.2 ns /  5.00 allocs → AFTER 127.6 / 1.00
#     (folder wall flat: the AFTER window absorbs the input drop the
#      BEFORE arm defers outside its timing; the alloc gate is the win)
# [3] NC conversion BEFORE 2.700 ms / 15.40 allocs → AFTER 1.524 / 7.00
# gates: 500 files + 500 folders field-identical; NC conversion
#        field-identical vs a fresh classifier run
```

## [4] NC session — deep-clone per request → `Arc` end-to-end

Every authenticated NC request paid: the extractor's `(**arc).clone()` — a
DEEP clone of `NcSession` (~8-9 String allocs) despite its doc claiming
"one Arc increment"; a chroot-cache hit cloning the stored `FolderDto` by
value (~5 allocs, moka `get` clones `V`); and a session build that cloned
`CurrentUser` for the extension, cloned `raw_username`, and `to_string`ed
the span value. Now: `NC_CHROOT_CACHE` stores `Arc<FolderDto>`,
`NcSession.user` is the same `Arc<CurrentUser>` the extension holds,
`raw_username` moves, the span renders lazily (`field::display`, the
ROUND5 §7 pattern the NC path had missed), and handlers extract
`SharedNcSession` — an `Arc` handle that derefs to `NcSession`, so the 64
field-access sites are untouched.

```
cargo run --release --features bench --example bench_nc_session
# 100k iterations                          wall ms   allocs/op
# [1] extractor  BEFORE deep clone           17.0      8.000
#                AFTER  SharedNcSession       4.2      0.000   (4.0x)
# [2] chroot hit BEFORE FolderDto value      21.3      4.000
#                AFTER  Arc<FolderDto>       11.7      0.000   (1.8x)
# [3] build      BEFORE clone×2 + span       17.6     11.000
#                AFTER  shared Arc           11.8      6.000   (1.5x)
# gate: every field handlers consume identical (incl. the URL-user check)
```

## [5] Storage micro-pack

Four independent A/Bs in one harness (`bench_storage_micro`, no Postgres):

- **(a) Local chunk write** — `try_exists` (stat) + `File::create` →
  one atomic `create_new` open; `AlreadyExists` IS the idempotent skip.
  20k × 4 KiB fresh writes 2707 → 1286 ms (**2.1x**); re-put skips 1.08x.
- **(b) CDC read prep** — `stream_chunks` took `Vec<String>`, forcing
  every read to deep-clone the cached manifest's whole hash list before
  the first byte; now it takes the manifest `Arc` and indexes. A
  4096-chunk manifest × 200 reads: 819 400 → 0 allocs, 49.4 → 0.16 ms.
  The Range path selects by index too — a `bytes=0-` probe of an N-chunk
  video no longer clones N hashes.
- **(c) Manifest miss herd** — `manifest_cached` used get→insert; K
  concurrent cold readers each ran the SELECT. Now fast-get +
  `try_get_with` (sentinel miss error keeps the positive-only contract —
  moka never caches loader errors, so legacy blobs and DB failures stay
  uncached). Herd of 64: 64 → 1 loads.
- **(d) Chunk `Content-MD5` hex** — the last `format!("{b:02x}")`-per-byte
  straggler (ROUND6 §7 shipped `hex_lower`); 18 → 1 allocs/digest, 10x.

```
cargo run --release --features bench --example bench_storage_micro
```

## [6] OCS capabilities — rebuilt per poll → memoized bytes

`/ocs/v{1,2}.php/cloud/capabilities` is process-invariant (pure config),
yet every poll re-built the ~40-node `json!` tree, re-read
`OXICLOUD_BASE_URL` from the **environment**, ran three `format!`s and
re-serialized. Both versions now serialize once into
`OnceLock<[Bytes; 2]>`; a poll is a refcount bump. The payload builder
takes its three config inputs directly (testable without `AppState`).

```
cargo run --release --features bench --example bench_capabilities_static
# 50k polls   BEFORE 269.6 ms / 102 allocs/poll → AFTER 1.1 ms / 0  (237x)
# gate: served bytes byte-identical for v1 and v2
```

## [7] `Drive::is_empty` — full-drive COUNT(*) sum → `EXISTS OR EXISTS`

The deletion precheck only needs a boolean, but aggregated every live
folder + file in the drive. `EXISTS` stops at the first row.

```
cargo run --release --features bench --example bench_drive_is_empty
# populated (100k files)  13.615 → 0.396 ms  (34.4x)
# empty                    0.219 → 0.166 ms  (1.3x)
# gate: identical booleans on both data shapes
```

## [8] favorites/recents row-map — the ROUND7 move that never got ported

ROUND7 §3 removed the per-row `name` clone in `/folders/{id}/resources`;
the same mapping in `/api/favorites/resources` and `/api/recent/resources`
still cloned `path` + `name` + `blob_hash` per row (and `folder_handler`
kept one `blob_hash` clone). All moved now — display classes computed
before `name` moves, `path`/`blob_hash` moved instead of cloned.

```
cargo run --release --features bench --example bench_resource_row_map
# [2] favorites/recents shape, rows=500
# BEFORE (clone) 12.004 allocs/row → AFTER (move) 9.254  (−2.75/row)
# gate: (name, path, content_hash, icon_class, category) identical per row
```

## [9] Folder rows — binary UUID decode (the ROUND6 §10 port)

ROUND6 adopted binary-UUID decode for file listing rows (1.17x) and queued
"other repos with the same shape"; `FolderDbRepository` never got it. All
folder-row queries (`list_folders_batch` — every Depth:1 PROPFIND subfolder
page — `get_folder`, descendants, search, suggest, and the write-path
RETURNINGs, which share `row_to_folder`) now decode `id`/`parent_id` as
binary `Uuid` (16 B vs 36 B on the wire, no server cast) and render once
app-side. Param casts (`$3::text IS NULL`), enum casts and the ltree
`path::text` renders are untouched.

**Honest verdict:** weaker than the file side. Four interleaved runs:
1.00x (wash), 1.05x, 1.03x, and 1.07x at 1000 rows — folder rows are
thinner than file rows, so the two casts are a smaller fraction of the
page. Adopted on the consistent small win + growth with page size + the
wire-bytes reduction; the first-run wash is inside the noise band.

```
cargo run --release --features bench --example bench_folder_uuid_decode
# rows/page=500 passes=400 (interleaved)   mean      p50      p95
# A ::text (before)                        1.061    1.039    1.310
# B binary (after)                         1.027    1.012    1.269   1.03x
# rows/page=1000: 1.758 → 1.639 mean                              1.07x
# gate: identical (id, name, path, parent_id) tuples
```

## [10] Authz — folder-level cascade decision (the ROUND8 deferred item)

ROUND8 memoised the per-file cascade decision, fixing revalidation; a
shared N-photo album's **cold first view** still ran N near-identical
ltree ancestor queries. The file decision now decomposes into exactly the
two branches of the historical UNION: parent point-read (new
`file_parent_cache`, 30 s TTL — grant writes don't alter parentage; moves
are the same TTL-healed indirect path as before) → the FOLDER cascade
decision (one ltree query per folder, shared by every sibling via the
existing `cascade_grant_cache`, recursing into the Folder arm) → a
direct-file-grant point lookup only when the folder half denies. The old
UNION query is deleted; no decision changes, including the parentless
edge (`folder_id IS NOT NULL` guard ≡ direct-only fallback).

Safety gates (hard asserts): recipient allowed on every file, outsider
denied, `clear_role` revoke denies IMMEDIATELY (the flush covers file and
folder decisions — same cache), and NEW: a caller holding only a direct
grant on one file is allowed that file and denied its siblings — proving
the folder-level decomposition neither shadows direct grants nor leaks a
file decision across siblings.

```
cargo run --release --features bench --example bench_thumbnail_cascade_cache
# thumbs=100 (folder-grant recipient, no drive membership)
# ROUND8 cold (union/file)      59.19 ms    591.91 µs/thumb
# AFTER  cold (first view)      41.77 ms    417.73 µs/thumb   (1.42x)
# AFTER  warm (revalidation)     0.13 ms      1.33 µs/thumb   (unchanged)
```

The first view is now bounded by the per-file parent PK reads (cheap, but
still N point queries) + 1 ltree query — batching the parent resolution
per page would need a wider API change; noted for a future round.

## [11] SPA — `resolveLabel` linear directory scan → id-keyed index

`resolveLabel`/`resolveRecipient` ran `contactCache.find(...)` — a linear
scan over the whole system address book — once per rendered grant row /
lane header on `/shared`, re-rendering on every page and role change:
O(rows × directory). Now a `Map<id, Contact>` built once per cache
identity (exactly like the existing `groupCache`).

```
cd frontend && npx vitest run src/lib/api/endpoints/recipients.bench.test.ts --disable-console-intercept
# 50 frames × 30 rows @ C=5000: before 11.0 ms, after 0.8 ms (13.9x)
# gates: labels identical (present + absent ids); comparisons rows×C → C
```

## [12] SPA — selection-prune guard + photos `matchMedia` hoist

- `ResourceList`'s prune `$effect` built an O(N) id `Set` on every
  infinite-scroll page even with nothing selected; guarded with
  `selected.size === 0` (reactive, so it re-arms when a selection
  appears). 100-page drain: 100 → 0 Set builds; pruned result identical
  when a selection exists.
- The photos timeline derive called `window.matchMedia(...)` per
  recompute (every 60-photo page); hoisted to state fed by one
  MediaQueryList `change` listener. P recomputes: P → 1 calls, identical
  booleans, crossings propagate.

```
cd frontend && npx vitest run src/lib/components/listDerives.bench.test.ts --disable-console-intercept
```

## Deferred / flagged (not shipped this round)

- **CalDAV authz-before-fetch reorder** (`calendar_service::get_event` /
  `list_events` / by-uid fetch the calendar row before the authz check
  only to read `.is_public`; running the already-required authz first and
  fetching only on denial saves one SELECT per authorized private-calendar
  read). Behavior-preserving (the OR commutes) but it reorders an authz
  check relative to a data fetch — flagged for maintainer sign-off per the
  authz-change convention, with the bench sketch in this round's notes.
- **Per-page batched parent resolution** for §10 — would cut the cold
  first view's N parent PK reads to one `= ANY` per page; needs a wider
  engine API (batch check) — future round.
- **`batch_operations` `Arc<str>` → `Option<&str>` widening** (ROUND7
  deferred) — re-audited: 1 small alloc/item vs a per-item DB roundtrip;
  still not worth the 2-trait/7-site churn alone. Standing verdict.
- **JWT-claims `Arc<str>`** (ROUND6 deferred) — still open; touches
  serde `rc` on `TokenClaims` + dozens of read sites. The 2 allocs/request
  remain the cheapest known win on the /api path for a future round.

## Correctness-adjacent (surfaced by the round-9 hunt — not perf)

- `trash_service.rs` restore matches error text
  (`format!("{}", e).contains("not found")`) instead of
  `e.kind == ErrorKind::NotFound` — fragile to rewording; flagged.
- The round-7 flags remain open: `fetchFolderListing` seeds empty
  `favoriteIds`/`sharedIds`; the search page still lacks a stale-response
  guard.