perf: round 11 — StoragePath joined-only, classifier fusion, memoized bodies, query-shape pack, SPA fine-grained stars

Backend (each change benchmark-gated with BEFORE replicas + equivalence
gates; see examples/bench_round11_micro.rs, bench_round11_queries.rs,
bench_log_writer.rs and benches/ROUND11.md — final numbers land in the
follow-up doc commit):

- StoragePath re-representation: single canonical joined String, segments
  derived on demand; File/Folder drop the duplicated path_string field
  (4000→1000 allocs per 500-row listing page)
- Display classifier fusion: classify_display shares one stack-lowered
  extension across the three decision trees; call sites in FileDto,
  folder/favorites/recent handlers, trash, path-resolver (+ interning
  where Arc::from was still used)
- /status.php and /openapi.json memoized into OnceLock<Bytes> (openapi
  rebuilt a 171 KiB spec per request: 2.8 ms → 18 ns)
- NC upload-session PROPFIND: write! + pre-sized body + stack RFC2822
  dates (2.3-2.6x, 2582→772 allocs at 256 chunks)
- REST download: dead FileDto clone removed (capture mime/size + move)
- CalendarEventDto/TrashedItem into_parts moves (11 KiB ical_data memcpy
  gone per CalDAV row); CardDAV getlastmodified stack render
- 4xx path: borrowed ErrorResponse serialize, ErrorKind::as_str,
  not_found/already_exists clone kill
- vCard emit via write!; search page moved out with into_iter skip/take;
  content-hit UUIDs parsed once; group last-user check via HashSet
- RateLimiter: lock-free get + insert (and_upsert_with variant REJECTED
  by benchmark); CSRF token borrow-compare + borrowed cookie extraction
- Thumbnail/preview ETags built from as_str (Debug-identical bytes)
- Encrypted backend: encrypt_in_place_detached single-buffer write path,
  chunk-sized reserve in collect_stream; retry labels made lazy
- PG: deferred upload registration 3→1 round-trips (persist_file CTE
  template); direct_grant_cache for Calendar/AddressBook/Playlist authz
  (single-flight + set_role/clear_role invalidation); expand_user
  tokio::join!; geo clusters min(uuid)::text; recluster face assignment
  batched into one UNNEST update
- People recluster cosine: norms precomputed once (bit-identical gate)
- NC capabilities poll logs demoted to debug; tracing-appender dep added
  for the log-writer benchmark

Frontend:
- ResourceList.selectedEntries O(N)-per-toggle → id-index projection
  O(k log k); favorites/recent consume the batchToolbar snippet param and
  drop their duplicate filter + dead selectedIds mirror
- Recent: star state via new favoriteIds prop — a star click no longer
  rebuilds all N entries
- admin timeAgo >30d fallback uses the cached Intl.DateTimeFormat
- vitest gates in src/lib/components/round11.bench.test.ts

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
This commit is contained in:
Claude
2026-07-18 22:02:00 +00:00
parent 637478e7bd
commit 221c1f31b0
54 changed files with 4060 additions and 630 deletions
+116
View File
@@ -0,0 +1,116 @@
# Round 11 — StoragePath re-representation, classifier fusion, memoized static bodies, query-shape pack, SPA fine-grained stars
Benchmark-gated, same rule as ROUND2-10: every change ships with a
BEFORE/AFTER benchmark and an equivalence/safety gate; an AFTER that doesn't
beat its BEFORE gets rolled back or redesigned. Three candidates went
through exactly that loop this round (§Rejected): the moka `and_upsert_with`
rate-limiter rewrite measured SLOWER than the two-op shape it was meant to
replace and was redesigned as a lock-free `get`+`insert`; the GET/HEAD
`Last-Modified` stack-render port measured neutral-to-worse (the chrono
String is already the terminal allocation) and was dropped; the first
search-page `drain(range)` model lost to `to_vec` on wall and was reshaped
as `into_iter().skip().take()`.
Measured on 4 cores / 15 GiB, local PostgreSQL 16 (fsync off), release
profile; frontend on Node 22 / vitest 4 (jsdom). Reproduce any row with the
command in its section.
## Summary
(numbers filled from the final runs below)
| # | change | key metric | before → after |
|--:|---|---|---|
| 1 | REST download: dead `FileDto` clone → capture mime/size + move | allocs per download | 7 → 0 (dead clone removed) |
| 2 | `StoragePath` → single canonical joined `String` (segments derived lazily; entity `path_string` duplicate field removed) | allocs / wall per 500-row page | TBD |
| 3 | Display classifier fusion (`classify_display`, stack-lowered ext shared by the three trees) | ns / allocs per listing row | TBD |
| 4 | `/status.php` → `OnceLock<Bytes>` | ns / allocs per poll | TBD |
| 5 | `/openapi.json` → `OnceLock<Bytes>` (was: rebuild 171 KiB spec per request) | ns per request | TBD |
| 6 | NC upload-session PROPFIND: `write!` + capacity + stack dates | ns / allocs per 256-chunk PROPFIND | TBD |
| 7 | CSRF header token borrow compare | ns / allocs per state-changing request | TBD |
| 8 | Thumbnail/preview ETag: `as_str` push (Debug-identical bytes) | ns per thumbnail request | TBD |
| 9 | Recent-handler id: stack `encode_lower` | ns / allocs per call | TBD |
| 10 | 4xx body: borrowed serialize + `ErrorKind::as_str` + `not_found` clone kill | ns / allocs per 404 | TBD |
| 11 | vCard emit: `write!` (+ borrowed address fields) | ns / allocs per vCard | TBD |
| 12 | Search page: `into_iter().skip().take()` move | allocs per 50-item page | TBD |
| 13 | Content-hit verify: parse-once pairs | ns per 100-hit page | TBD |
| 14 | Group last-user check: HashSet probe | ns per 500×500 check | TBD |
| 15 | Retry op-label: lazy closure | ns / allocs per blob op | TBD |
| 16 | `encrypt_bytes`: in-place detached (write side now mirrors the in-place read) | ns / allocs per 256 KiB chunk | TBD |
| 17 | Encrypted `collect_stream`: chunk-sized reserve | allocs per 1 MiB read | TBD |
| 18 | Recluster cosine: precomputed norms (bit-identical) | ns per 200-face pass | TBD |
| 19 | `CalendarEventDto`: `into_parts` move (11 KiB `ical_data` copy gone) | ns / allocs per event | TBD |
| 20 | RateLimiter: lock-free `get` + `insert` | ns / allocs per limited request | TBD |
| Q1 | Deferred upload registration: 3 round-trips → 1 CTE insert | ms per uploaded file | TBD |
| Q2 | Calendar/AddressBook/Playlist authz `direct_grant_cache` | ms per DAV check | TBD |
| Q3 | `expand_user`: `tokio::join!` the 2 independent queries | ms per cold expansion | TBD |
| Q4 | Geo clusters: `min(file_id)::text` (cast per cluster, not per row) | ms per viewport | TBD |
| Q5 | Recluster persistence: per-face UPDATEs → one UNNEST batch | ms per 200-face apply | TBD |
| L1 | Log writer: `tracing-appender` non_blocking (lossy=false) | p99 emit µs under contention | TBD |
| S1 | SPA `ResourceList.selectedEntries`: O(N)×2 per toggle → O(k·log k), hosts consume the snippet param | comparisons per toggle | 2N → k |
| S2 | SPA Recent: star reads `favoriteIds` prop, mapper no longer set-dependent | rows re-mapped per star click | N → 0 |
| S3 | SPA admin `timeAgo`: cached `Intl.DateTimeFormat` | constructions per 1000 formats | ≤1 (was 1000) |
Also shipped without a dedicated row: `already_exists` clone kill (same
shape as `not_found`), CardDAV `getlastmodified` stack render (per-contact
REPORT path, ROUND10-§13 helper + fallback), NC capabilities poll logs
demoted to `debug` (INFO forced a locked-stdout write per client poll),
trash `to_dto` `into_parts` move + interned display fields (trash listing
+ path-resolver rows now share the ROUND9 interning).
## Rejected / reworked this round (the discipline working)
- **RateLimiter `entry().and_upsert_with`**: 1 846.5 → 1 862.1 ns and
8.0 → 9.1 allocs/op — moka's compute-entry machinery costs more than the
two-op shape it replaced. Redesigned as lock-free `get` (borrows the
key, no alloc) + `insert`; identical counter sequence gated.
- **GET/HEAD `Last-Modified` stack-render port**: chrono's `to_rfc2822()`
String IS the terminal allocation the header needs (44.3 ns incl. the
alloc vs 46.5 ns for stack render + the same alloc). Only body-emit
sites (where `write!` lands in an existing buffer) benefit — those were
ported (§6, CardDAV); the header sites were left on chrono.
- **Search page `drain(range).collect()`**: −300 allocs but slower on
wall than `to_vec` in the first model (tail memmove). Reshaped as
`into_iter().skip().take().collect()` — moves the page, drops the rest,
no tail shift.
## Deferred / flagged (not shipped this round)
- **NC preview 304 still runs `get_file`** (`preview_handler.rs`): the
object-id → file fetch on the revalidation path is only needed for
existence semantics (a deleted file must 404, not 304). Dropping it is
a behavior decision — same class as the standing CalDAV
authz-before-fetch reorder — flagged for maintainer sign-off.
- **`CachedBlobBackend`'s `Mutex<LruCache>` index** serializes every
cached read; a moka byte-weigher migration (the file-content-cache
pattern) is the natural fix but touches eviction-unlink semantics —
deserves its own round with a concurrency bench.
- **Capture-metadata extraction reads each media file 2-3×**
(`media_metadata_service.rs`: kamadak full read + nom-exif path re-read
+ track fallback re-read). Feeding nom-exif from the in-memory buffer
needs its `MediaSource` API verified on the pinned version.
- **`CachedBlobBackend::local_blob_path` sync `stat`** (ROUND10 flag
stands): needs an async port variant.
- **Azure SDK 0.21 stack** drags duplicate dependency trees (h2 0.3+0.4,
two hashbrown generations, base64 0.13) into the binary; an SDK bump is
a dedicated migration, not a perf tweak.
- **`AudioMetadataRepository::list_by_{artist,album,genre}`** are dead
code (never called) with seq-scan `ILIKE` shapes — flag for deletion
rather than indexing.
- **`CachedBlobBackend::put_blob` cache population** silently fails for
S3/Azure whole-file puts (the inner backend deletes the source before
the cache copy runs) — correctness note for maintainers, not perf.
- **CalDAV authz-before-fetch reorder** — ROUND9/10 flag stands.
## Environment / methodology
- `cargo run --release --features bench --example bench_round11_micro`
(pure CPU, counting allocator, BEFORE replicas vs shipped code).
- `cargo run --release --features bench --example bench_round11_queries`
(Postgres; seeds + sweeps its own fixtures).
- `BENCH_LOG_ARM=sync|nonblocking [BENCH_LOG_WRITER=slow] cargo run
--release --features bench --example bench_log_writer >/dev/null`.
- `cd frontend && npx vitest run src/lib/components/round11.bench.test.ts`.
- Regression guards from earlier rounds re-run after the StoragePath /
classifier changes: `bench_row_path` (round-4 gates) and `bench_dto_map`
(round-3 gates).