2026-07-18 22:02:00 +00:00
# Round 11 — StoragePath re-representation, classifier fusion, memoized static bodies, query-shape pack, SPA fine-grained stars
Benchmark-gated, same rule as ROUND2-10: every change ships with a
BEFORE/AFTER benchmark and an equivalence/safety gate; an AFTER that doesn't
2026-07-18 22:46:20 +00:00
beat its BEFORE gets rolled back or redesigned. Four candidates went through
exactly that loop this round (§Rejected below): the moka `and_upsert_with`
rate-limiter rewrite, the GET/HEAD `Last-Modified` stack-render port, the
`min(uuid)` geo-cluster cast (PostgreSQL has no such aggregate — the gate
caught it before it could ship broken), and the `tracing-appender`
non-blocking log writer (slower than sync on fast sinks, tail-loss risk on
slow ones).
2026-07-18 22:02:00 +00:00
Measured on 4 cores / 15 GiB, local PostgreSQL 16 (fsync off), release
profile; frontend on Node 22 / vitest 4 (jsdom). Reproduce any row with the
2026-07-18 22:46:20 +00:00
command in its section (`benches/ROUND11.md` §Environment).
2026-07-18 22:02:00 +00:00
## Summary
| # | change | key metric | before → after |
|--:|---|---|---|
2026-07-18 22:46:20 +00:00
| 1 | REST download: dead `FileDto` clone → capture mime/size + move | ns / allocs per download | 295.3 → 20.7 ns · 7 → 0 allocs |
| 2 | `StoragePath` → single canonical joined `String` ; `File` /`Folder` drop the duplicate `path_string` field | 500-row page (depth 4) | 121.0 → 80.3 µs (**1.51x**) · 4 000 → 1 000 allocs |
| 3 | Display classifier fusion (`classify_display` , one stack-lowered ext for the three trees) | 13-row mixed corpus | 4 390 → 2 074 ns (**2.12x**) · 21 → 0 allocs |
| 4 | `/status.php` → `OnceLock<Bytes>` | per NC client poll | 836 → 28.8 ns (**29x**) · 14 → 0 allocs |
| 5 | `/openapi.json` → `OnceLock<Bytes>` (was rebuilding the 171 KiB spec per request) | per request | 2.77 ms → 18.5 ns · 12 474 → 0 allocs |
| 6 | NC upload-session PROPFIND: `write!` + pre-sized body + stack dates | 256-chunk session | 203.7 → 75.4 µs (**2.70x**) · 2 582 → 772 allocs |
| 7 | CSRF token: borrow-only compare (+ borrowed cookie extraction) | per state-changing request | 55.2 → 2.2 ns · 1 → 0 allocs |
| 8 | Thumbnail/preview ETag via `as_str` (Debug-identical bytes — cached client ETags stay valid) | per thumbnail request | 150.5 → 87.7 ns · 3 → 2 allocs |
| 9 | Recent-handler id: stack `encode_lower` | per record/remove | 52.2 → 10.7 ns · 1 → 0 allocs |
| 10 | 4xx body: borrowed `ErrorResponse` + `ErrorKind::as_str` + `not_found` /`already_exists` clone kill | per 404 | 426 → 367 ns · 11 → 8 allocs |
| 11 | vCard emit: `write!` + borrowed address fields | per contact create/update | 804 → 386 ns (**2.08x**) · 21 → 5 allocs |
| 12 | Search page: `into_iter().skip().take()` move (both branches) | 50-item page | 108.3 → 89.4 µs · −
| 13 | Content-hit verify: parse each UUID once | 100-hit page | 7.90 → 5.20 µs (**1.52x**) |
| 14 | Group last-user check: HashSet probe | 500×
| 15 | Retry op-label: lazy closure (success path never formats) | per blob op | 71.8 → 0.7 ns · 2 → 0 allocs |
| 16 | `encrypt_bytes` : in-place detached, single buffer (write now mirrors the in-place read) | 256 KiB chunk | 231.1 → 208.1 µs (**1.11x**) · 2 → 1 allocs |
| 17 | Encrypted `collect_stream` : chunk-sized reserve | 1 MiB blob, 4 KiB frames | 60.4 → 53.2 µs · 9 → 1 allocs |
| 18 | Recluster cosine: norms precomputed once (bit-identical gate over all pairs) | 200 faces ×
| 19 | `CalendarEventDto` : `into_parts` move (the ~11 KiB `ical_data` memcpy gone) | per CalDAV event row | 497 → 283 ns (**1.76x**) · 14 → 8 allocs |
| 20 | RateLimiter: lock-free `get` (borrows the key) + `insert` | per limited request | allocs 8.0 → 6.0 · wall neutral (1 657 vs 1 695 ns, within run variance) |
| Q1 | Deferred upload registration: 3 round-trips → 1 CTE insert (the `persist_file` template) | per uploaded file (incl. cleanup DELETE) | 1.83 → 1.32 ms · gates: identical path/drive, missing-parent → not-found |
| Q2 | Calendar/AddressBook/Playlist authz `direct_grant_cache` (single-flight, invalidated on `set_role` /`clear_role` ) | per DAV check | 0.197 ms → 0.2 µs on hit (**~1000x**) · revocation-flip gate OK |
| Q3 | `expand_user` : `tokio::join!` the `is_external` read + groups CTE | per cold expansion | 0.426 → 0.204 ms (**2.1x**) |
| Q5 | Recluster persistence: per-face UPDATE loop → one UNNEST batch | 200-face apply (incl. reset) | 80.0 → 8.7 ms (**9.2x**) · final column state identical |
| S1 | SPA `ResourceList.selectedEntries` : O(N)×
| S2 | SPA Recent: star reads the new `favoriteIds` prop — mapper no longer depends on the set | rows re-mapped per star click (N=400) | 400 → 0 · identical star states gated |
| S3 | SPA admin `timeAgo` >30d: cached `Intl.DateTimeFormat` | constructions per 1 000 formats | ≤1 · output equals `toLocaleDateString()` |
Also shipped without a dedicated row: CardDAV `getlastmodified` per-contact
stack render (ROUND10-§13 helper + chrono fallback), NC capabilities poll
logs demoted to `debug` (each poll forced a formatted line + locked stdout
write), trash `to_dto` `into_parts` move + fused/interned display fields
(trash listing and path-resolver rows now share the ROUND9 interning),
`TrashedItemDto` name/path moves.
Cross-round regression guards re-run after the StoragePath / classifier
rework — both gates PASS byte-identical, and the shipped code now beats the
numbers those rounds recorded:
- `bench_row_path` (round 4): file row 705 → 280 ns/row (2.52x), allocs
15.75 → 6.00; folder row 684 → 365 ns (1.87x), 14.08 → 5.24.
- `bench_dto_map` (round 3): File→FileDto 839 → 606 ns/row, 10.09 → 3.08
allocs; Folder→FolderDto 314 → 161 ns, 11.80 → 1.00 allocs.
## [1] REST download dead clone → move
```
cargo run --release --features bench --example bench_round11_micro # §1
```
`download_file_impl` cloned the whole `FileDto` (7 owned Strings) into
`get_file_optimized_preloaded` on every authenticated download, purely to
read `mime_type` /`size` afterwards — the share path already captured+moved
(ROUND10 §3 fixed its double-*fetch*, not this clone). Now: one `Arc<str>`
bump + a `u64` copy, then move. 295.3 → 20.7 ns, 7 → 0 allocs per download.
## [2] StoragePath joined-only representation
```
cargo run --release --features bench --example bench_round11_micro # §20
cargo run --release --features bench --example bench_row_path # cross-round gate
```
`StoragePath` stored `segments: Vec<String>` — one heap String per path
component built on EVERY hydrated row — while the DTO path only ever
consumed the joined form, and the entities carried a second `path_string`
duplicate. The value object now stores the canonical joined `String` alone
(`"/"` or `/seg(/seg)*` ); `file_name` /`parent` /`segments()` /`Display`
derive on demand; `File` /`Folder` lost the duplicate field (`path_string()`
borrows). `.segments()` had zero external callers — verified before the
rework. Equivalence gates: identical `path_string` , `file_name` , `parent` ,
`Display` across the corpus, plus the round-4 harness's byte-identical
gate. 500-row page: 121.0 → 80.3 µs, 4 000 → 1 000 allocs; per-row RAM
drops by the Vec + per-segment String headers + the duplicate path.
## [3] Display classifier fusion
```
cargo run --release --features bench --example bench_round11_micro # §21
```
Every listed file ran the full MIME classification three times
(`icon_class_for` , `icon_special_class_for` , `category_for` ), each
heap-allocating its own `to_ascii_lowercase()` on the extension-fallback
path. The three decision trees are byte-for-byte preserved (they diverge
deliberately, so no merged tree); `classify_display` lowers the extension
once into a 16-byte stack buffer shared by all three. Extensions longer
than any table entry short-circuit to the same `_` -arm defaults (gated).
Call sites: `FileDto::from` , folder/favorites/recent handlers, trash
listing (×
interning they had missed (`Arc::from` per row → refcount bump).
13-row corpus: 4 390 → 2 074 ns, 21 → 0 allocs.
## [4][5] Memoized process-invariant bodies
```
cargo run --release --features bench --example bench_round11_micro # §3, §18
```
`/status.php` rebuilt its `json!` tree per NC client poll (836 ns / 14
allocs → 28.8 ns / 0). `/openapi.json` was the extreme case: utoipa
reconstructed and re-serialized the whole 171 KiB spec on every request —
2.77 ms / 12 474 allocs → 18.5 ns / 0 via the same `OnceLock<Bytes>`
pattern as ROUND9's capabilities memoization. Byte-identical gates on both.
## [6] NC upload-session PROPFIND emit
The last hand-built XML handler: `String::new()` + `push_str(&format!(…))`
per element per chunk + chrono `to_rfc2822()` per chunk. Now pre-sized +
`write!` + `common::fmt::rfc2822_utc` (chrono fallback out-of-range).
Byte-identical gates at 16 and 256 chunks (escape distributes over
concatenation; RFC 2822 output has no XML-special chars). 16 chunks:
13.2 → 5.2 µs; 256 chunks: 203.7 → 75.4 µs, 2 582 → 772 allocs.
## [Q1] Deferred upload registration 3 → 1
```
cargo run --release --features bench --example bench_round11_queries # §1
```
The write-behind REST upload path ran parent-drive SELECT → INSERT →
parent-path SELECT, the first and third re-reading the identical
`storage.folders` row. Ported to the `WITH parent AS (…) INSERT … SELECT
… RETURNING (SELECT path FROM parent)` template `persist_file` has used
since ROUND2 — 0 rows ⇒ the same `not_found("Folder")` the old first query
produced (gated, anti-enum shape preserved). Root uploads (no parent)
keep their previous two-step shape.
## [Q2] direct_grant_cache
Calendar/AddressBook/Playlist were the only `check()` arms with no result
cache — every CalDAV/CardDAV/music request re-ran the `role_grants` point
query, and DAV clients poll continuously. Added
`direct_grant_cache: Cache<(Subject, Resource, Permission), bool>`
(30 s TTL / 100k, `try_get_with` single-flight — the ROUND8/10 pattern),
flushed on `set_role` /`clear_role` for those resource types; group/expiry
churn self-heals within the TTL exactly like `cascade_grant_cache` .
Gates: identical verdict; a revocation + flush flips the next check.
0.197 ms → 0.2 µs on hit.
## [Q3] expand_user join!
The `is_external` point read and the recursive groups CTE are independent;
serial await paid two round-trips end-to-end on every cold expansion
(per user per 30 s TTL window). `tokio::join!` : 0.426 → 0.204 ms.
## [Q5] Recluster UNNEST batch
`POST /api/people/recluster` issued one `UPDATE faces.faces SET person_id`
per face sequentially (both the unassign-small-clusters and assign loops).
Assignments now accumulate and apply as a single
`UPDATE … FROM unnest($1::uuid[], $2::uuid[])` (the ROUND10 `save_faces`
pattern) — 200-face library: 80.0 → 8.7 ms, final column state identical.
Pairs with §18's norm precomputation on the CPU side (7.70 → 7.17 ms for
the O(N²) pass, bit-identical similarity gated over every pair).
## [S1][S2][S3] SPA pack (vitest gates)
```
cd frontend && npx vitest run src/lib/components/round11.bench.test.ts
```
- **`ResourceList.selectedEntries` ** (the ROUND10 flagged item): the
component re-filtered the ENTIRE items array per selection change, and
favorites/recent ignored the snippet param and recomputed their own
`entries.filter(…)` shadow — two full O(N) scans per toggle, O(N²)-ish
across a shift-range. Now an id→index Map (rebuilt only when `items`
changes) projects the selection in O(k·log k) preserving item order;
hosts consume the snippet param and their dead `selectedIds` mirror is
gone (the component's prune effect already self-heals on reload).
51-toggle gesture on 2 000 items: 204 000 → <2 602 comparisons.
- **Recent favorite star**: the entry mapper read `favoriteIds.has(id)` ,
subscribing the whole O(N) map to the SvelteSet — one star click rebuilt
all N entries and re-rendered every visible row. `ResourceList` gained a
`favoriteIds` prop read directly by the star widget; the mapper is
set-independent. Star click: N → 0 rows re-mapped, star states gated
identical.
- **admin `timeAgo` **: the >30-day fallback called `toLocaleDateString()`
(a fresh `Intl.DateTimeFormat` per call); now the app-wide cached
formatter — output equality gated, ≤1 construction per 1 000 formats.
2026-07-18 22:02:00 +00:00
## Rejected / reworked this round (the discipline working)
2026-07-18 22:46:20 +00:00
- **`min(fm.file_id)::text` geo-cluster cast (Q4)**: PostgreSQL has **no
`min(uuid)` aggregate** — the "cast once per cluster" rewrite fails to
parse (`42883` ). The gate caught it before it could ship broken; the
per-row-cast original stays (22.6 ms per 5k-row viewport, admin-shaped
traffic), and a custom `CREATE AGGREGATE` was judged schema surface this
query doesn't justify. The bench section now reproduces the rejection.
- **RateLimiter `entry().and_upsert_with` **: 1 657 → 2 365 ns and
8.0 → 9.1 allocs — moka's compute-entry machinery costs more than the
two ops it replaced. Redesigned as lock-free `get` (borrows the key, no
alloc) + `insert` : 6.0 allocs, wall within variance; identical counter
sequence gated. Adopted in that form.
2026-07-18 22:02:00 +00:00
- **GET/HEAD `Last-Modified` stack-render port**: chrono's `to_rfc2822()`
2026-07-18 22:46:20 +00:00
String IS the terminal allocation the header needs (38.7 ns incl. alloc
vs 47.0 ns stack render + the same alloc). Only body-emit sites (where
`write!` lands in an existing buffer, 31.8 ns / 0 allocs) benefit —
those were ported (§6 + CardDAV); header sites stay on chrono.
- **`tracing-appender` non-blocking log writer (L1)**: on a fast sink
(stdout→/dev/null — the containerized default) sync sustains 1.41M ev/s
vs 0.99M non-blocking, with a better tail (p999 76 vs 151 µs, max 0.8
vs 8.3 ms — the channel hop costs more than the write). Non-blocking
only wins on a slow sink (20 µs/line: 6.4x wall, p50 22 → 1.9 µs), but
there the drain gate FAILED — buffered tail lines can be lost at
shutdown, unacceptable for the audit channel. Not adopted;
`tracing-appender` stays as a dev-dependency for the reproducible
harness (`bench_log_writer` , 4 arms via `BENCH_LOG_ARM` /`_WRITER` ).
- **First search-page model (`drain(range)` )**: −
wall (tail memmove). Reshaped as `into_iter().skip().take().collect()`
— moves the page, drops the rest, wins both axes (§12 row in Summary).
2026-07-18 22:02:00 +00:00
## Deferred / flagged (not shipped this round)
2026-07-18 22:46:20 +00:00
- **NC preview 304 still runs `get_file` **: dropping the fetch on the
revalidation path changes existence semantics (deleted file → 304
instead of 404) — needs maintainer sign-off, same class as the standing
CalDAV authz-before-fetch reorder (ROUND9/10 flag).
2026-07-18 22:02:00 +00:00
- **`CachedBlobBackend` 's `Mutex<LruCache>` index** serializes every
2026-07-18 22:46:20 +00:00
cached read on remote+cache deployments; the moka byte-weigher
migration (file-content-cache pattern) deserves its own round with a
concurrency bench and eviction-unlink care.
2026-07-18 22:02:00 +00:00
- **Capture-metadata extraction reads each media file 2-3×
2026-07-18 22:46:20 +00:00
(`media_metadata_service` : kamadak full read + nom-exif path re-read +
track fallback). Feeding nom-exif from the in-memory buffer needs its
`MediaSource` API verified on the pinned version.
2026-07-18 22:02:00 +00:00
- **`CachedBlobBackend::local_blob_path` sync `stat` ** (ROUND10 flag
2026-07-18 22:46:20 +00:00
stands — needs an async port variant).
2026-07-18 22:02:00 +00:00
- **Azure SDK 0.21 stack** drags duplicate dependency trees (h2 0.3+0.4,
2026-07-18 22:46:20 +00:00
three hashbrown generations, base64 0.13, getrandom 0.1) into every
build — an SDK bump is a dedicated migration.
2026-07-18 22:02:00 +00:00
- **`AudioMetadataRepository::list_by_{artist,album,genre}` ** are dead
2026-07-18 22:46:20 +00:00
code with seq-scan `ILIKE` shapes — flag for deletion, not indexing.
2026-07-18 22:02:00 +00:00
- **`CachedBlobBackend::put_blob` cache population** silently fails for
2026-07-18 22:46:20 +00:00
S3/Azure whole-file puts (inner backend deletes the source before the
cache copy) — correctness note for maintainers.
- **Grouped file/grid views unvirtualized** (ROUND10 flag stands).
2026-07-18 22:02:00 +00:00
## Environment / methodology
- `cargo run --release --features bench --example bench_round11_micro`
2026-07-18 22:46:20 +00:00
— 21 sections, counting allocator, BEFORE replicas vs shipped shapes,
equivalence gates inline (`BENCH_ITERS` , default 100k).
2026-07-18 22:02:00 +00:00
- `cargo run --release --features bench --example bench_round11_queries`
2026-07-18 22:46:20 +00:00
— needs Postgres; seeds and sweeps its own fixtures (`BENCH_PASSES` ).
2026-07-18 22:02:00 +00:00
- `BENCH_LOG_ARM=sync|nonblocking [BENCH_LOG_WRITER=slow] cargo run
--release --features bench --example bench_log_writer >/dev/null`.
- ` cd frontend && npx vitest run src/lib/components/round11.bench.test.ts`.
2026-07-18 22:46:20 +00:00
- Cross-round guards: ` bench_row_path`, ` bench_dto_map`.
