Claude
aba89c4f5d
Every change is benchmark-verified (harness + before/after numbers in benches/, measured on this branch; reproduction commands in each doc): DAV / sync-client hot paths - PROPFIND dead-properties: one = ANY($1) query per 500-child page instead of one sequential query per child, and indexable `=` predicates instead of IS NOT DISTINCT FROM (seq scans). 2,000-child folder: 1.07-4.54 s of DB chatter -> 4-6 ms (258-773x). Applied to native + NC PROPFIND and both NC REPORT handlers. [benches/DEAD-PROPS.md] - Folder paging: keyset cursor (name > $last) + new partial index (folder_id, name) replaces LIMIT/OFFSET full-folder rescan per page. Full 20k-file walk: 1266 ms -> 77 ms (16.5x). New migration 20260917000000. [benches/PROPFIND-PAGING.md] - NC chroot / default-drive resolution: moka caches (30 s TTL, explicit invalidation on drive mutations) for find_default_for_user and the markerless chroot FolderDto. 2 uncached queries + 2 pool checkouts per NC/WebDAV/WOPI request -> sub-us moka hit (p50 0.7-3.6 ms -> ~1 us). [benches/CHROOT-CACHE.md] - Quota: PROPFINDs whose prop list never names a quota prop skip the 2-query resolution entirely (wants_quota()); the remaining lookups read 2 columns instead of the full auth.users row with its <=512 KiB avatar (11-16x, p50 3.4 ms -> 0.29 ms). Same narrow read now gates every upload quota check. [benches/QUOTA-PATH.md] CPU on the request path - ZIP exports (folder download, share ZIP, batch download): entries whose MIME says already-compressed (JPEG/MP4/zip/pdf/...) are Stored instead of Deflate - deflate ran inline on the tokio writer task at ~41 MB/s for ~0% size gain. Mixed media corpus: 4.31x wall and CPU, archive size unchanged. Shared predicate in common::mime_detect. [benches/ZIP-MEDIA.md] - Compression layers: tower-http's default maps to Brotli QUALITY 11 (verified in brotli-8.0.2 source and empirically: 90 ms per 64 KiB JSON response, 1.3 s per 700 KiB bundle). Both layers pinned to Precise(4): 99x less CPU for ~15% more bytes. SPA assets are now precompressed at build time (scripts/precompress.mjs, 77% smaller) and served via ServeDir::precompressed_br/gzip: 2016x less per-request work, and clients get the better q11 bytes. [benches/STATIC-PRECOMPRESSED.md] Batched / cached backend paths [benches/NPLUS1-AND-CACHES.md] - Content-search ReBAC re-verification: new AuthorizationEngine::check_files_read_batch (default = old loop; PgAclEngine override batches drive resolution + reuses role cache). 200 sequential point SELECTs per search -> 1-2 queries. - Batch-ZIP subtree downloads: drop per-file re-authz + per-file Recent recording (2 writes/file) for subtree entries already authorized at the root - mirrors the native folder-download path. ~6,000 statements removed from a 2,000-file archive. - CDC chunk manifests: immutable by content address, now moka-cached (weight-bounded 32 MiB, 60 s TTL, positive-only, invalidated on delete) - removes one manifest query (p50 0.44-4.4 ms) from every stream, range and full blob read. - People tab: grouped COUNT + batched cover lookup instead of dragging every face row with its 2 KiB embedding (10k faces: 30.4 ms & 21 MB -> 3.8 ms & 1.3 KB, 8.1x); merge() is one set-based UPDATE. [benches/PEOPLE-LIST.md] - Photos timeline cursor: raw timestamptz comparison instead of EXTRACT(EPOCH ...) wrapper + IS NULL OR disjunction - cursor is an index boundary again, deep scroll stops re-scanning skipped rows. - Public share landing: one atomic UPDATE ... access_count + 1 (was SELECT + full-row write-back: racy, lost updates, clobbered concurrent owner edits) - 3 round-trips -> 2 per visit. - move_to_trash: dead full-entity SELECT feeding a documented no-op removed from both branches; dead fields dropped from TrashService. - NFC normalization: is_nfc_quick fast path skips the decompose/recompose state machine for the ~100% already-NFC case (every row loaded from PG). Frontend - Large folders paint after page one (~200 items) via fetchFolderListing's new onPage hook instead of waiting for every sequential page. - Tested-and-reverted (kept for the record): cached Intl.Collator for name sorts - vitest showed it 2x SLOWER than V8's argument-less localeCompare fast path (5.6 ms vs 12.1 ms / 5k names). Sort order untouched. New bench harnesses under examples/ (bench feature): zip_media, dead_props, chroot_cache, quota_path, people_list, propfind_paging, static_precompress. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w
91 lines
5.0 KiB
Markdown
91 lines
5.0 KiB
Markdown
# Companion fixes — same measured unit economics, no dedicated harness
|
||
|
||
These changes share their cost model with benches that already exist, so
|
||
instead of near-duplicate harnesses each entry cites the bench that measured
|
||
its unit price. (The per-query unit prices below: sequential indexed point
|
||
SELECT ≈ 0.25–0.55 ms and `= ANY($1)` batch ≈ 1–1.5 ms/500 ids from
|
||
benches/DEAD-PROPS.md; manifest-row fetch p50 0.44–4.4 ms from
|
||
benches/BLOB-MANIFEST.md; moka hit ≈ 1 µs from benches/CHROOT-CACHE.md.)
|
||
|
||
## 1. Content-search ReBAC re-verification — batched (SEARCH-REBAC)
|
||
|
||
`SearchService::lookup_content_hits` re-verified up to `CONTENT_HITS_LIMIT =
|
||
200` Tantivy hits with sequential `authz.check(Read, File)` calls — each a
|
||
point SELECT on owner-cache miss (distinct file ids ⇒ ~always). New
|
||
`AuthorizationEngine::check_files_read_batch` (default = the old loop, so
|
||
mocks/other impls stay correct; `PgAclEngine` override): ONE
|
||
`id = ANY($1)` drive resolution + cached per-drive role + per-file cascade
|
||
only for drive-floor misses. Decision-equivalent; per 200-hit search:
|
||
**~200 sequential round-trips (≈ 50–110 ms of DB chatter) → 1–2 queries
|
||
(≈ 1–3 ms)**. Also primes the owner cache for the hits' follow-up requests.
|
||
|
||
## 2. Batch-ZIP downloads — no per-file authz/Recent (ZIP-BATCH-AUTHZ)
|
||
|
||
`BatchOperations::add_folder_subtree_to_zip` had already authorized the
|
||
subtree ROOT (`get_folder_with_perms`), yet every enumerated file still paid
|
||
`get_file_stream_with_perms` = 1 authz point SELECT + a Recent-hook spawn
|
||
issuing 2 writes (INSERT … ON CONFLICT + prune DELETE). A 2,000-file folder
|
||
ZIP ⇒ ~6,000 extra statements. Subtree entries now use the plain
|
||
`get_file_stream` — exactly what `ZipService::create_folder_zip` (the native
|
||
folder-download path) has always done. Explicitly-selected top-level files
|
||
keep per-file authz + Recent. Unit price: DEAD-PROPS.md sequential rows —
|
||
**~1.5–4.5 s of DB chatter removed** from a 2,000-file archive, plus the ZIP
|
||
no longer floods Recents with every archived file.
|
||
|
||
## 3. CDC manifest RAM cache (MANIFEST-CACHE)
|
||
|
||
Every stream / range / full read of a CDC blob paid one
|
||
`chunk_manifests` row fetch first — p50 0.44 ms (4.4 ms under pool pressure,
|
||
benches/BLOB-MANIFEST.md), on the hottest read paths there are (media
|
||
serving, thumbnails, range seeks). Manifests are immutable by content
|
||
address, so `DedupService` now memoises them (moka, weight-bounded 32 MiB,
|
||
60 s TTL, positive-only so background rechunking is honoured immediately;
|
||
invalidated post-commit on the two delete paths). Warm read: **0.44–4.4 ms →
|
||
~1 µs** (CHROOT-CACHE.md's moka row) and one fewer pool checkout per read —
|
||
range-seek storms (video scrubbing) hit this every request.
|
||
|
||
## 4. Public share landing — 3 round-trips → 1 atomic UPDATE (SHARE-ACCESS)
|
||
|
||
`GET /api/s/{token}` ran find_share_by_token (with a correlated
|
||
`MIN(expires_at)` subquery), a full-row UPDATE writing back a Rust-side
|
||
increment (racy: lost updates between concurrent visitors, and it rewrote
|
||
`item_name`/`password_hash` wholesale — clobbering concurrent owner edits),
|
||
then the handler's follow-up fetched the share a third time.
|
||
`ShareStoragePort::increment_access_count` is now one
|
||
`UPDATE … SET access_count = access_count + 1 WHERE token = $1 AND <expiry>`:
|
||
**3 subquery round-trips → 2** for the landing (register + fetch), no
|
||
read-modify-write race, no collateral column rewrites.
|
||
|
||
## 5. Trash — dead SELECT removed
|
||
|
||
`TrashService::move_to_trash` fetched the full file/folder entity to build a
|
||
`TrashedItem` consumed only by `TrashRepository::add_to_trash` — a documented
|
||
no-op in the soft-delete model. Both branches now go straight to the
|
||
`move_to_trash` UPDATE: **one uncached SELECT + entity hydration removed per
|
||
trash operation** (file and folder).
|
||
|
||
## 6. NFC normalization fast path
|
||
|
||
`normalize_storage_name` ran unicode-normalization's full
|
||
decompose/recompose state machine on every name of every row loaded from PG
|
||
(listings, PROPFIND, photos — 27 constructor call sites), even though the DB
|
||
invariant guarantees stored names are already NFC. `is_nfc_quick` (a
|
||
per-char table lookup) now short-circuits the ~100 % case to a plain copy;
|
||
`Maybe`/`No` still run the full pipeline, so semantics are unchanged.
|
||
|
||
## 7. Frontend — first-page render for large folders
|
||
|
||
`fetchFolderListing` paged the ENTIRE folder (sequential 200-item requests)
|
||
before returning anything — a 2,000-item folder waited ~10 round-trips
|
||
before first paint. The files route now paints page one immediately via the
|
||
new `onPage` hook and fills in as later pages land (skipped when a cached
|
||
listing is already on screen, so views never shrink). First-paint latency
|
||
for an N-item folder drops from ⌈N/200⌉ sequential RTTs to 1.
|
||
|
||
## Refuted by benchmark (reverted, kept for the record)
|
||
|
||
- **Cached `Intl.Collator` for name sorts (frontend):** sorting 5,000 names —
|
||
argument-less `localeCompare` 5.6 ms vs cached collator **12.1 ms (2×
|
||
slower)**. V8 fast-paths argument-less `localeCompare`; the "cache the
|
||
collator" folklore does not apply. Reverted, ordering untouched.
|