Oxicloud/benches/ROUND15.md

# Round 15 — grouped-listing O(N²) rebucket, exif/reseed allocations, tantivy zero-hit snippet skip

Benchmark-gated, same rule as ROUND2–14: every change ships with a
BEFORE/AFTER benchmark and an equivalence/safety gate; an AFTER that doesn't
beat its BEFORE is rolled back (never applied). The roll-back rule is encoded
directly into each harness as a `GATE FAIL … rollback` non-zero exit (Rust) or
a threshold `expect()` (frontend), so a regression fails CI rather than
shipping.

This round lands the ROUND14-deferred **flagship** — the grouped-listing
`sections` rebuild that was the last O(N²)-per-page accumulation left in the
SvelteKit listing surfaces — plus three backend items pulled from the same
deferred list: two allocation cuts on the photo-ingest / content-reseed worker
paths, and a wasted `SnippetGenerator` build removed from the zero-hit content
search path.

Measured on 4 cores / 15 GiB, local PostgreSQL not needed for any Round-15 arm
(all no-Postgres: release profile for the Rust examples; Node 22 / vitest for
the frontend). Reproduce any row with the command in its section.

## Summary

| # | change | key metric | before → after |
|--:|---|---|---|
| F1 | Grouped listings (trash / recent / favorites / shared-with-me) re-bucketed the WHOLE accumulated list on every infinite-scroll page; `ResourceSectionsBuilder` re-buckets only the fresh page and reuses each untouched bucket's array reference | 50×50 (2 500-item) drain, month buckets | **63 750 → 2 500 `bucketOf` calls (25.5×)** · **12.5 → 1.3 ms wall (9.9×)** · O(N²/page) → O(N) |
| B1 | exif `Make`/`Model` — `display_value().to_string().trim_matches('"').trim().to_string()` throws the display `String` away to allocate the trimmed copy; the in-place `drain`+`truncate` helper keeps one allocation | 4 sample values/op | **150.4 → 119.4 ns (1.26×)** · **8 → 4 allocs/op** (2 → 1 per field) |
| B2 | Content-index worker called `text_extractor::supports` (lowercases MIME + extension) TWICE per file per drain batch; classify once into a `Vec<bool>` and thread it through both uses | 256-file reseed batch | **34.5 → 16.7 µs (2.07×)** · **704 → 353 allocs/op** |
| B3 | Zero-hit content search still built a `SnippetGenerator` (query-compile + term weighting) though the per-hit loop was empty; return `Ok(vec![])` as soon as `top_docs.is_empty()` | no-hit query, 400-doc index | **1 575.6 → 1 237.2 ns (1.27×)** · **21 → 19 allocs/op** (widens with index size) |

## [F1] Grouped listings — incremental swimlane builder

```
cd frontend && npx vitest run src/lib/utils/resourceSections.bench.test.ts
```

Every grouped listing page (`/trash`, `/recent`, `/favorites`,
`/shared-with-me`) loads its rows via infinite scroll (`raw = [...raw,
...page]`), and `ResourceList`'s `sections` `$derived.by` re-bucketed the
**whole accumulated list** on every page: Σ ≈ O(N²/page) `bucketOf` + `ctxOf`
calls across a drain, and a brand-new rows array for *every* bucket each page
(so `VirtualList` re-diffed every swimlane every page). This was the ROUND14
"flagship follow-up" — the same O(N²)-per-page class ROUND6 fixed for the files
listing, ROUND14 §F2 for favorites' `favoriteIds`, and `PhotoTimeline` for the
photos grid.

`ResourceSectionsBuilder` (extracted to `$lib/utils/resourceSections`, off the
Svelte reactive graph so it's unit/benchmark-testable) exploits the append
invariant: a grouped listing is server-sorted by the active group's `orderBy`,
so a fresh page only ever extends existing buckets or appends new ones. It
detects the append (prefix-identity on the boundary object), re-buckets only
the fresh page, and hands back the **same array reference** for every untouched
bucket — the property `VirtualList` (which diffs its `items` prop by reference)
relies on to skip re-rendering it — while emitting a fresh array only for
buckets the page actually grew. Any non-append (group-by switch, deletion,
dotfile-filter toggle) falls back to a full rebuild, so the output is always
deep-equal to the pure `buildResourceSections` reference.

Correctness does **not** depend on bucket contiguity: the one non-monotonic
group-by in the set — trash grouped **by drive** but ordered by name, so a page
sprays items across every already-emitted drive bucket — stays byte-for-byte
equal to the full rebuild (it just refreshes more buckets per page). Header
labels are recomputed every sync (never cached): a group-by's `labelOf` can
resolve asynchronously (owner / sharer names arrive after the rows), and a
cached label would freeze the header at its fallback.

50×50 (2 500-item) month-bucketed drain: **63 750 → 2 500 `bucketOf` calls
(25.5× fewer), 12.5 → 1.3 ms wall (9.9×)**. Gates: (1) equivalence — the
incremental output is deep-equal to `buildResourceSections` at *every* page for
both a contiguous (date) and a non-contiguous (drive) group-by; (2) reference
stability — untouched buckets keep their exact array reference across an append
while a grown bucket gets a fresh one; (3) correct fallback on group-by switch,
deletion and the flat pass-through; (4) perf — `bucketOf` work is exactly O(N)
across the drain and wall drops ≥3×.

## [B1]–[B2] exif / content-reseed allocation cuts

```
cargo run --release --features bench --example bench_round15_micro
```

Counting-allocator micro-bench; each section is BEFORE (the shipped-before
shape) vs AFTER (the shipped function / shape) with a byte-identity gate.

- **[B1] exif `Make`/`Model` single-allocation trim.** `ExifService::extract`
  read the camera make + model as
  `field.display_value().to_string().trim_matches('"').trim().to_string()` —
  the first `to_string()` materializes the display value (unavoidable), then
  `.trim_matches('"').trim().to_string()` allocates a **second** `String` for
  the trimmed copy and drops the first. The new `display_value_trimmed` applies
  the same two-stage trim in place on the already-owned buffer (`drain` drops
  the stripped prefix, `truncate` the suffix — both reuse the allocation), so a
  quoted `"Canon"` costs one allocation instead of two. Per ingested photo (the
  Make + Model fields). 4 sample values/op: **8 → 4 allocs/op (2 → 1 per
  field), 1.26× wall**. Gate: byte-identical to the old chain across quoted /
  padded / clean shapes.
- **[B2] Content-index worker single `supports()` classify.** `supports`
  lowercases the MIME (and, on a generic MIME, the extension) — 1–2 allocations
  — and the drain loop called it **twice per file**: once in the
  `wanted_hashes` filter, once again in the per-file records loop. The worker
  now classifies each file once into a `Vec<bool>` and threads the flag through
  both. On a full reseed that is one redundant classify (and its allocations)
  removed for *every file in the library*. 256-file batch: **704 → 353
  allocs/op, 34.5 → 16.7 µs (2.07×)**. Gate: the `(wanted, supported)` tallies
  are identical before/after.

## [B3] Tantivy zero-hit snippet skip

```
cargo run --release --features bench --example bench_round15_tantivy
```

`TantivyContentIndex::search_blocking` built the `SnippetGenerator` from the
query right after the `TopDocs` search — but `SnippetGenerator::create`
compiles the query against the index (collects query terms, looks up each
term's document frequency, builds the weighting), and when the query matched
**no** documents that generator is never used: the per-hit loop is empty. A
content search for a term that isn't in any indexed document (a common miss)
paid that build for nothing on the request path.

The fix returns `Ok(Vec::new())` the moment `top_docs.is_empty()`, before the
create. The bench reproduces the exact skipped operation on a RAM index built
with the public tantivy API (same crate + version): BEFORE = search + create,
AFTER = search + the `is_empty()` early return; the delta is the wasted create.
No-hit query against a 400-document index: **1 575.6 → 1 237.2 ns (1.27×), 21 →
19 allocs/op** — the create adds ~338 ns + 2 allocs on top of the search on
*every* zero-hit content query, and its per-term `doc_freq` lookups grow with
the index (the RAM bench's 400 docs understate the production term dictionary).
Gates: the miss query genuinely returns zero hits, and a control arm confirms a
term that *does* hit still yields a snippet fragment (the skip only ever
triggers on a true zero-hit query).

## Not shipped — carried forward from the ROUND14 deferred list

Still queued, unchanged in scope (each wants its own decision, Postgres
fixture, or bigger refactor):

- **Query-shape (needs Postgres):** `music_storage_adapter::list_public_playlists`
  1 + N `COUNT(*)` fold (opt-in public-gallery path); contact REST listings
  (`search_contacts`, `get_contacts_by_address_book_paginated`,
  `get_contacts_in_group`) over-fetch the multi-KB `vcard` TEXT though every
  caller maps to a `ContactDto` with no `vcard` field (wants a *lite* row
  mapper, since the non-paginated sibling is shared with the CardDAV stream).
- **Frontend:** `shared/+page.svelte` rebuilds the full `lanes` tree per page
  and on every grant edit; the same incremental-builder pattern F1 uses is the
  follow-up. (F1 removed the `ResourceList.sections` half of the ROUND14
  "flagship" bullet; the `lanes` half remains.)
- **CPU/alloc (background):** REST calendar-event edit re-`format!`s the whole
  `ical_data` body once per changed property; `dedup_service` hash-`String`
  re-allocations; `exif_service` still double-allocates the GPS-ref display in
  `parse_gps_coord` (single-alloc, low-frequency — folded into B1's helper is
  possible but the ref is compared to `"S"`/`"W"` as a borrow, so it never
  needed the second alloc the Make/Model path did).
- **Storage I/O (cached-remote class):** `CachedBlobBackend` per-write
  `create_dir_all` + inline eviction `remove_file` on the reactor thread;
  `encrypted_blob_backend` 64 KiB vs 256 KiB plaintext frames.

## Environment / methodology

- `cargo run --release --features bench --example bench_round15_micro`
  — counting allocator, no Postgres (`BENCH_ITERS`, `BENCH_BATCH`).
- `cargo run --release --features bench --example bench_round15_tantivy`
  — builds a RAM tantivy index, no Postgres (`BENCH_ITERS`, `BENCH_DOCS`).
- `cd frontend && npx vitest run src/lib/utils/resourceSections.bench.test.ts`.
- Roll-back rule encoded per harness: the Rust examples `std::process::exit(1)`
  with `GATE FAIL … rollback` if an AFTER arm fails to beat its BEFORE; the
  vitest gate `expect()`s the O(N) call count and the ≥3× wall.