perf: round 9 — decorator PUT reactivation, session/search/dedup alloc purges, PROPFIND join!, folder-level cascade
Benchmark-gated round (benches/ROUND9.md): every change carries a BEFORE/AFTER bench with equivalence/safety gates; verdicts below are from the committed harnesses on 4 cores / local PG 16. Backend: - Blob decorators (Retry/Cached) now forward put_blob_from_bytes_unsynced + sync_blobs — the trait default had silently reinstated HEAD-before-PUT per chunk on decorated remote stacks, undoing ROUND3 §8. Full production stack: 500 probes -> 0, 1.9x wall at 10 ms RTT (bench_s3_put §3). - NC PROPFIND per-page enrichment triple (favorites / oc:fileid / dead props) overlapped with tokio::join!: 2.07x local, 2.86x at 5 ms RTT (bench_nc_enrich_join, injected-latency decide-by-bench). - Search enrichment consumes its DTOs and carries the interned Arc<str> display fields end-to-end (SearchFileResultDto type change, OpenAPI shape preserved): enrich_file 2.0x, 11.6 -> 2.2 allocs/row; the NC REPORT conversion stops re-running all three classifiers per row (bench_search_enrich). - NC session Arc end-to-end: SharedNcSession extractor (8 -> 0 allocs), Arc<FolderDto> chroot cache (4 -> 0/hit), single shared Arc<CurrentUser> + lazy span render (11 -> 6/build) (bench_nc_session). - Storage micro-pack: atomic create_new chunk writes (2.1x fresh), stream_chunks over the manifest Arc (4097 -> 0 allocs/read incl. the Range path), manifest single-flight (herd 64 -> 1 loads), hex_lower for chunk Content-MD5 (18 -> 1 allocs) (bench_storage_micro). - OCS capabilities memoized into OnceLock<[Bytes;2]>: 237x, 102 -> 0 allocs/poll, byte-identical (bench_capabilities_static). - Drive::is_empty COUNT(*) sum -> EXISTS: 34.4x on a 100k-file drive (bench_drive_is_empty). - favorites/recents row-map ROUND7 port: path/name/blob_hash moved, -2.75 allocs/row (bench_resource_row_map §2). - Folder rows decode binary UUIDs (ROUND6 §10 port): 1.03-1.07x page fetch, honest verdict incl. one noise-band wash documented (bench_folder_uuid_decode). - Authz: file cascade decision decomposed into memoized folder-level decision + direct-grant lookup (ROUND8 deferred item). Cold shared-album first view 592 -> 418 µs/thumb; warm path unchanged; safety gates incl. new direct-grant sibling isolation, revoke-flush re-verified, full integration authz suite green (bench_thumbnail_cascade_cache). Frontend (vitest gates committed beside the code): - resolveLabel/resolveRecipient O(directory) scan -> id-keyed Map: 13.9x (recipients.bench.test.ts). - ResourceList selection-prune effect skips when nothing is selected (100 -> 0 Set builds per drain) and the photos timeline reads a listener-fed mobile flag instead of matchMedia per recompute (listDerives.bench.test.ts). Verification: cargo fmt + clippy --all-features --all-targets -D warnings clean; 524 unit + 554 integration (--cfg integration_tests) tests pass; frontend npm run check clean with 293 vitest tests green. Deferred with rationale in ROUND9.md: CalDAV authz-before-fetch reorder (maintainer sign-off), per-page batched parent resolution, JWT-claims Arc<str>, batch_operations signature widening. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDc9VtXvskJ6dnMRraSndn
This commit is contained in:
Claude
+51
@@ -350,6 +350,57 @@ name = "bench_micro_allocs"
|
||||
path = "examples/bench_micro_allocs.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# Round-9 battery ─────────────────────────────────────────────────────────────
|
||||
|
||||
# Search enrichment — borrow+clone+reclassify vs consume+carry (file/folder
|
||||
# enrich + the NC REPORT search→FileDto conversion). No Postgres.
|
||||
[[example]]
|
||||
name = "bench_search_enrich"
|
||||
path = "examples/bench_search_enrich.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# Storage micro-pack — local chunk write create_new, manifest Vec-clone vs
|
||||
# Arc-index, manifest miss single-flight, Content-MD5 hex. No Postgres.
|
||||
[[example]]
|
||||
name = "bench_storage_micro"
|
||||
path = "examples/bench_storage_micro.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# NC per-request session — extractor deep-clone vs Arc handle, chroot-cache
|
||||
# value vs Arc, session build double-clone vs shared Arc. No Postgres.
|
||||
[[example]]
|
||||
name = "bench_nc_session"
|
||||
path = "examples/bench_nc_session.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# OCS capabilities poll — rebuild+serialize per request vs OnceLock<Bytes>
|
||||
# memoization. No Postgres.
|
||||
[[example]]
|
||||
name = "bench_capabilities_static"
|
||||
path = "examples/bench_capabilities_static.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# Drive::is_empty — full-drive COUNT(*) sum vs short-circuit EXISTS
|
||||
# (needs the dev Postgres up).
|
||||
[[example]]
|
||||
name = "bench_drive_is_empty"
|
||||
path = "examples/bench_drive_is_empty.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# Folder-listing rows — `id::text`/`parent_id::text` casts vs binary UUID
|
||||
# decode + app-side render, the round-6 file-side port (needs Postgres).
|
||||
[[example]]
|
||||
name = "bench_folder_uuid_decode"
|
||||
path = "examples/bench_folder_uuid_decode.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# NC PROPFIND per-page enrichment triple — serial 3×RTT vs tokio::join!,
|
||||
# with injected-latency arms at 0/0.25/1/5 ms (needs Postgres).
|
||||
[[example]]
|
||||
name = "bench_nc_enrich_join"
|
||||
path = "examples/bench_nc_enrich_join.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# Round-8 battery ─────────────────────────────────────────────────────────────
|
||||
|
||||
# Shared-album thumbnail authz — folder-grant cascade query per thumbnail vs
|
||||
|
||||
@@ -0,0 +1,328 @@
|
||||
# Round 9 — decorator PUT reactivation, session/search/dedup alloc purges, PROPFIND `join!`, folder-level cascade
|
||||
|
||||
Benchmark-gated, same rule as ROUND2-8: every change ships with a
|
||||
BEFORE/AFTER benchmark and equivalence/safety gates; an AFTER that doesn't
|
||||
beat its BEFORE gets rolled back. The two decide-by-bench items this round
|
||||
(PROPFIND enrichment `join!`, folder binary-UUID) were adopted only after
|
||||
their gates passed; the authz change carries hard safety gates plus a new
|
||||
direct-grant-sibling isolation gate and was validated against the full
|
||||
authz-relevant unit suite.
|
||||
|
||||
Measured on 4 cores / 15 GiB, local PostgreSQL 16 (fsync off), release
|
||||
profile; frontend on Node 22 / vitest 4 (jsdom). Reproduce any row with the
|
||||
command in its section.
|
||||
|
||||
## Summary
|
||||
|
||||
| # | change | key metric | before → after |
|
||||
|--:|---|---|---|
|
||||
| 1 | Blob decorators forward `put_blob_from_bytes_unsynced` | HEAD probes / wall, 500-chunk upload @10 ms RTT | 500 → 0 probes; full stack 1571 → 812 ms (**1.9x**) |
|
||||
| 2 | NC PROPFIND page enrichment triple → `tokio::join!` | p50 ms/page (500 children) | local 2.28 → 1.10 (**2.07x**); @5 ms RTT 22.1 → 7.7 (**2.86x**) |
|
||||
| 3 | Search enrich consume+carry (`Arc<str>` result fields) | enrich_file ns/row · allocs/row | 456 → 223 (**2.0x**) · 11.6 → 2.2; NC conversion 15.4 → 7.0 allocs/row |
|
||||
| 4 | NC session end-to-end `Arc` (extractor/chroot/build) | allocs per authenticated NC request | extractor 8→0, chroot hit 4→0, build 11→6 (**~17 fewer/req**) |
|
||||
| 5 | Storage micro-pack (create_new · manifest Arc · single-flight · hex) | see §5 | fresh chunk writes **2.1x**; 4097→0 allocs/read; herd 64→1 loads; 18→1 allocs/digest |
|
||||
| 6 | OCS capabilities memoized (`OnceLock<Bytes>`) | 50k polls wall · allocs/poll | 269.6 → 1.1 ms (**237x**) · 102 → 0 |
|
||||
| 7 | `Drive::is_empty` COUNT(*) → `EXISTS` | ms/call, 100k-file drive | 13.6 → 0.40 (**34.4x**) |
|
||||
| 8 | favorites/recents row-map move (ROUND7 port) | allocs/row | 12.00 → 9.25 (**−2.75/row**) |
|
||||
| 9 | Folder rows: binary UUID decode (ROUND6 port) | 500-row page mean | 1.06–1.10 → 1.03–1.04 ms (**1.03–1.07x**, first run a wash — see §9) |
|
||||
| 10 | Folder-level cascade decision (authz, ROUND8 deferred) | cold first view µs/thumb (100-photo album) | 592 → 418 (**1.42x**); warm 1.33 µs unchanged |
|
||||
| 11 | SPA: `resolveLabel` O(C)→O(1) index | 50 frames × 30 rows @ 5k contacts | 11.0 → 0.8 ms (**13.9x**); comparisons rows×C → C |
|
||||
| 12 | SPA: selection-prune guard + `matchMedia` hoist | per-page Set builds / matchMedia calls | 100 → 0 · P → 1 |
|
||||
|
||||
## [1] Blob decorators — the trait-default fallthrough was re-adding HEAD-before-PUT
|
||||
|
||||
ROUND3 §8 made chunk writes skip the remote exists-probe by introducing
|
||||
`put_blob_from_bytes_unsynced` (content-addressed keys make re-PUTs
|
||||
overwrite-safe). But `RetryBlobBackend` and `CachedBlobBackend` never
|
||||
overrode it, so the **trait default** routed every decorated `_unsynced`
|
||||
call back through the probing `put_blob_from_bytes` — silently reinstating
|
||||
HEAD+PUT per chunk on every remote deployment with retry or cache enabled
|
||||
(the recommended object-store setup). `EncryptedBlobBackend` and
|
||||
`MigrationBlobBackend` already forwarded correctly.
|
||||
|
||||
Both decorators now forward `put_blob_from_bytes_unsynced` and `sync_blobs`
|
||||
to their inner backend (Retry wraps the former in its retry loop; the
|
||||
durability sweep is deliberately NOT retried — a failed fsync must surface,
|
||||
not be re-issued after the kernel may have dropped the dirty pages).
|
||||
`CachedBlobBackend` keeps its local write-through population on the
|
||||
unsynced path (shared `cache_bytes_write_through` helper, no eviction sweep
|
||||
— matching the historical write-path behavior) so post-upload readers
|
||||
(thumbnail/EXIF/face hooks) still hit the cache.
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_s3_put
|
||||
# 500 x 256 KiB chunk PUTs at concurrency 8, 10 ms/request stub
|
||||
# [1] raw backend BEFORE 1519 ms (500 HEADs) → AFTER 765 ms (0) 2.0x
|
||||
# [3] retry(s3) BEFORE 1524 ms (500 HEADs) → AFTER 766 ms (0) 2.0x
|
||||
# cache(s3) BEFORE 1535 ms (500 HEADs) → AFTER 803 ms (0) 1.9x
|
||||
# cache(enc(retry(s3))) 1571 ms (500) → 812 ms (0) 1.9x
|
||||
# gates: BEFORE probes == chunks, AFTER probes == 0, cache write-through
|
||||
# populated on BOTH routes (2×chunks files present)
|
||||
```
|
||||
|
||||
## [2] NC PROPFIND page enrichment — 3 serial round-trips → `tokio::join!`
|
||||
|
||||
Every Depth:1 PROPFIND page enriches its ≤500 children with three
|
||||
INDEPENDENT batched reads (favorites `= ANY`, oc:fileid `= ANY`, dead
|
||||
props `= ANY`), previously awaited in sequence. This is the round-7
|
||||
deferred "serial pairs" item, and the one pair the round-7 notes ranked
|
||||
worth gating (3 round-trips, per page, on the hottest sync path).
|
||||
|
||||
Decide-by-bench with injected per-round-trip latency (0/0.25/1/5 ms),
|
||||
because ROUND6 showed concurrency can LOSE on local-socket PG (the authz
|
||||
`try_join_all` rejection). It doesn't here — these are three fat batched
|
||||
queries whose **server-side execution** parallelizes across PG backends,
|
||||
so even the local-socket floor wins, not just the RTT overlap:
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_nc_enrich_join
|
||||
# children=500, passes=100, p50 ms/page serial join! ratio
|
||||
# 0 µs injected 2.275 1.097 2.07x
|
||||
# 250 µs 6.273 2.481 2.53x
|
||||
# 1000 µs 9.163 3.441 2.66x
|
||||
# 5000 µs 22.050 7.709 2.86x
|
||||
# gate: identical favorite sets / id maps / dead-prop rows; adoption
|
||||
# required no local-socket regression — it's a 2x win even there
|
||||
```
|
||||
|
||||
Contrast with ROUND6 §8 (rejected): that fan-out issued ~200 single-row
|
||||
authz checks through the engine's cache layers; this overlaps exactly 3
|
||||
page-batched queries. Both files' and folders' page loops adopted it.
|
||||
|
||||
## [3] Search enrichment — borrow+clone+reclassify → consume+carry
|
||||
|
||||
`enrich_file` took `&FileDto`, cloned every owned String out of it, and
|
||||
RE-RAN the three display classifiers whose results the DTO already carried
|
||||
interned (`Arc<str>`, computed once in `FileDto::from`); the recursive
|
||||
branch maps the ENTIRE pre-pagination match set. The NC REPORT conversion
|
||||
(`file_dto_from_search`) then re-ran all three classifiers a SECOND time
|
||||
per emitted row. `SearchFileResultDto.{mime_type,icon_class,
|
||||
icon_special_class,category}` are now `Arc<str>` (`#[schema(value_type =
|
||||
String)]` keeps the OpenAPI shape; JSON output byte-identical), both
|
||||
enrichers consume their DTO, the intermediate `Vec<FileDto>`/`Vec<FolderDto>`
|
||||
materializations are fused away, suggest reuses the interned fields, and
|
||||
the NC conversion carries them (refcount bumps). The search-cache byte
|
||||
weigher keeps counting `.len()` per row — now an over-count of shared
|
||||
bytes, i.e. the conservative direction.
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_search_enrich
|
||||
# rows=10000 passes=50 (p50 ns/row; allocs from pass 0)
|
||||
# [1] enrich_file BEFORE 455.8 ns / 11.60 allocs → AFTER 222.7 / 2.20
|
||||
# [2] enrich_folder BEFORE 116.2 ns / 5.00 allocs → AFTER 127.6 / 1.00
|
||||
# (folder wall flat: the AFTER window absorbs the input drop the
|
||||
# BEFORE arm defers outside its timing; the alloc gate is the win)
|
||||
# [3] NC conversion BEFORE 2.700 ms / 15.40 allocs → AFTER 1.524 / 7.00
|
||||
# gates: 500 files + 500 folders field-identical; NC conversion
|
||||
# field-identical vs a fresh classifier run
|
||||
```
|
||||
|
||||
## [4] NC session — deep-clone per request → `Arc` end-to-end
|
||||
|
||||
Every authenticated NC request paid: the extractor's `(**arc).clone()` — a
|
||||
DEEP clone of `NcSession` (~8-9 String allocs) despite its doc claiming
|
||||
"one Arc increment"; a chroot-cache hit cloning the stored `FolderDto` by
|
||||
value (~5 allocs, moka `get` clones `V`); and a session build that cloned
|
||||
`CurrentUser` for the extension, cloned `raw_username`, and `to_string`ed
|
||||
the span value. Now: `NC_CHROOT_CACHE` stores `Arc<FolderDto>`,
|
||||
`NcSession.user` is the same `Arc<CurrentUser>` the extension holds,
|
||||
`raw_username` moves, the span renders lazily (`field::display`, the
|
||||
ROUND5 §7 pattern the NC path had missed), and handlers extract
|
||||
`SharedNcSession` — an `Arc` handle that derefs to `NcSession`, so the 64
|
||||
field-access sites are untouched.
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_nc_session
|
||||
# 100k iterations wall ms allocs/op
|
||||
# [1] extractor BEFORE deep clone 17.0 8.000
|
||||
# AFTER SharedNcSession 4.2 0.000 (4.0x)
|
||||
# [2] chroot hit BEFORE FolderDto value 21.3 4.000
|
||||
# AFTER Arc<FolderDto> 11.7 0.000 (1.8x)
|
||||
# [3] build BEFORE clone×2 + span 17.6 11.000
|
||||
# AFTER shared Arc 11.8 6.000 (1.5x)
|
||||
# gate: every field handlers consume identical (incl. the URL-user check)
|
||||
```
|
||||
|
||||
## [5] Storage micro-pack
|
||||
|
||||
Four independent A/Bs in one harness (`bench_storage_micro`, no Postgres):
|
||||
|
||||
- **(a) Local chunk write** — `try_exists` (stat) + `File::create` →
|
||||
one atomic `create_new` open; `AlreadyExists` IS the idempotent skip.
|
||||
20k × 4 KiB fresh writes 2707 → 1286 ms (**2.1x**); re-put skips 1.08x.
|
||||
- **(b) CDC read prep** — `stream_chunks` took `Vec<String>`, forcing
|
||||
every read to deep-clone the cached manifest's whole hash list before
|
||||
the first byte; now it takes the manifest `Arc` and indexes. A
|
||||
4096-chunk manifest × 200 reads: 819 400 → 0 allocs, 49.4 → 0.16 ms.
|
||||
The Range path selects by index too — a `bytes=0-` probe of an N-chunk
|
||||
video no longer clones N hashes.
|
||||
- **(c) Manifest miss herd** — `manifest_cached` used get→insert; K
|
||||
concurrent cold readers each ran the SELECT. Now fast-get +
|
||||
`try_get_with` (sentinel miss error keeps the positive-only contract —
|
||||
moka never caches loader errors, so legacy blobs and DB failures stay
|
||||
uncached). Herd of 64: 64 → 1 loads.
|
||||
- **(d) Chunk `Content-MD5` hex** — the last `format!("{b:02x}")`-per-byte
|
||||
straggler (ROUND6 §7 shipped `hex_lower`); 18 → 1 allocs/digest, 10x.
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_storage_micro
|
||||
```
|
||||
|
||||
## [6] OCS capabilities — rebuilt per poll → memoized bytes
|
||||
|
||||
`/ocs/v{1,2}.php/cloud/capabilities` is process-invariant (pure config),
|
||||
yet every poll re-built the ~40-node `json!` tree, re-read
|
||||
`OXICLOUD_BASE_URL` from the **environment**, ran three `format!`s and
|
||||
re-serialized. Both versions now serialize once into
|
||||
`OnceLock<[Bytes; 2]>`; a poll is a refcount bump. The payload builder
|
||||
takes its three config inputs directly (testable without `AppState`).
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_capabilities_static
|
||||
# 50k polls BEFORE 269.6 ms / 102 allocs/poll → AFTER 1.1 ms / 0 (237x)
|
||||
# gate: served bytes byte-identical for v1 and v2
|
||||
```
|
||||
|
||||
## [7] `Drive::is_empty` — full-drive COUNT(*) sum → `EXISTS OR EXISTS`
|
||||
|
||||
The deletion precheck only needs a boolean, but aggregated every live
|
||||
folder + file in the drive. `EXISTS` stops at the first row.
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_drive_is_empty
|
||||
# populated (100k files) 13.615 → 0.396 ms (34.4x)
|
||||
# empty 0.219 → 0.166 ms (1.3x)
|
||||
# gate: identical booleans on both data shapes
|
||||
```
|
||||
|
||||
## [8] favorites/recents row-map — the ROUND7 move that never got ported
|
||||
|
||||
ROUND7 §3 removed the per-row `name` clone in `/folders/{id}/resources`;
|
||||
the same mapping in `/api/favorites/resources` and `/api/recent/resources`
|
||||
still cloned `path` + `name` + `blob_hash` per row (and `folder_handler`
|
||||
kept one `blob_hash` clone). All moved now — display classes computed
|
||||
before `name` moves, `path`/`blob_hash` moved instead of cloned.
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_resource_row_map
|
||||
# [2] favorites/recents shape, rows=500
|
||||
# BEFORE (clone) 12.004 allocs/row → AFTER (move) 9.254 (−2.75/row)
|
||||
# gate: (name, path, content_hash, icon_class, category) identical per row
|
||||
```
|
||||
|
||||
## [9] Folder rows — binary UUID decode (the ROUND6 §10 port)
|
||||
|
||||
ROUND6 adopted binary-UUID decode for file listing rows (1.17x) and queued
|
||||
"other repos with the same shape"; `FolderDbRepository` never got it. All
|
||||
folder-row queries (`list_folders_batch` — every Depth:1 PROPFIND subfolder
|
||||
page — `get_folder`, descendants, search, suggest, and the write-path
|
||||
RETURNINGs, which share `row_to_folder`) now decode `id`/`parent_id` as
|
||||
binary `Uuid` (16 B vs 36 B on the wire, no server cast) and render once
|
||||
app-side. Param casts (`$3::text IS NULL`), enum casts and the ltree
|
||||
`path::text` renders are untouched.
|
||||
|
||||
**Honest verdict:** weaker than the file side. Four interleaved runs:
|
||||
1.00x (wash), 1.05x, 1.03x, and 1.07x at 1000 rows — folder rows are
|
||||
thinner than file rows, so the two casts are a smaller fraction of the
|
||||
page. Adopted on the consistent small win + growth with page size + the
|
||||
wire-bytes reduction; the first-run wash is inside the noise band.
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_folder_uuid_decode
|
||||
# rows/page=500 passes=400 (interleaved) mean p50 p95
|
||||
# A ::text (before) 1.061 1.039 1.310
|
||||
# B binary (after) 1.027 1.012 1.269 1.03x
|
||||
# rows/page=1000: 1.758 → 1.639 mean 1.07x
|
||||
# gate: identical (id, name, path, parent_id) tuples
|
||||
```
|
||||
|
||||
## [10] Authz — folder-level cascade decision (the ROUND8 deferred item)
|
||||
|
||||
ROUND8 memoised the per-file cascade decision, fixing revalidation; a
|
||||
shared N-photo album's **cold first view** still ran N near-identical
|
||||
ltree ancestor queries. The file decision now decomposes into exactly the
|
||||
two branches of the historical UNION: parent point-read (new
|
||||
`file_parent_cache`, 30 s TTL — grant writes don't alter parentage; moves
|
||||
are the same TTL-healed indirect path as before) → the FOLDER cascade
|
||||
decision (one ltree query per folder, shared by every sibling via the
|
||||
existing `cascade_grant_cache`, recursing into the Folder arm) → a
|
||||
direct-file-grant point lookup only when the folder half denies. The old
|
||||
UNION query is deleted; no decision changes, including the parentless
|
||||
edge (`folder_id IS NOT NULL` guard ≡ direct-only fallback).
|
||||
|
||||
Safety gates (hard asserts): recipient allowed on every file, outsider
|
||||
denied, `clear_role` revoke denies IMMEDIATELY (the flush covers file and
|
||||
folder decisions — same cache), and NEW: a caller holding only a direct
|
||||
grant on one file is allowed that file and denied its siblings — proving
|
||||
the folder-level decomposition neither shadows direct grants nor leaks a
|
||||
file decision across siblings.
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_thumbnail_cascade_cache
|
||||
# thumbs=100 (folder-grant recipient, no drive membership)
|
||||
# ROUND8 cold (union/file) 59.19 ms 591.91 µs/thumb
|
||||
# AFTER cold (first view) 41.77 ms 417.73 µs/thumb (1.42x)
|
||||
# AFTER warm (revalidation) 0.13 ms 1.33 µs/thumb (unchanged)
|
||||
```
|
||||
|
||||
The first view is now bounded by the per-file parent PK reads (cheap, but
|
||||
still N point queries) + 1 ltree query — batching the parent resolution
|
||||
per page would need a wider API change; noted for a future round.
|
||||
|
||||
## [11] SPA — `resolveLabel` linear directory scan → id-keyed index
|
||||
|
||||
`resolveLabel`/`resolveRecipient` ran `contactCache.find(...)` — a linear
|
||||
scan over the whole system address book — once per rendered grant row /
|
||||
lane header on `/shared`, re-rendering on every page and role change:
|
||||
O(rows × directory). Now a `Map<id, Contact>` built once per cache
|
||||
identity (exactly like the existing `groupCache`).
|
||||
|
||||
```
|
||||
cd frontend && npx vitest run src/lib/api/endpoints/recipients.bench.test.ts --disable-console-intercept
|
||||
# 50 frames × 30 rows @ C=5000: before 11.0 ms, after 0.8 ms (13.9x)
|
||||
# gates: labels identical (present + absent ids); comparisons rows×C → C
|
||||
```
|
||||
|
||||
## [12] SPA — selection-prune guard + photos `matchMedia` hoist
|
||||
|
||||
- `ResourceList`'s prune `$effect` built an O(N) id `Set` on every
|
||||
infinite-scroll page even with nothing selected; guarded with
|
||||
`selected.size === 0` (reactive, so it re-arms when a selection
|
||||
appears). 100-page drain: 100 → 0 Set builds; pruned result identical
|
||||
when a selection exists.
|
||||
- The photos timeline derive called `window.matchMedia(...)` per
|
||||
recompute (every 60-photo page); hoisted to state fed by one
|
||||
MediaQueryList `change` listener. P recomputes: P → 1 calls, identical
|
||||
booleans, crossings propagate.
|
||||
|
||||
```
|
||||
cd frontend && npx vitest run src/lib/components/listDerives.bench.test.ts --disable-console-intercept
|
||||
```
|
||||
|
||||
## Deferred / flagged (not shipped this round)
|
||||
|
||||
- **CalDAV authz-before-fetch reorder** (`calendar_service::get_event` /
|
||||
`list_events` / by-uid fetch the calendar row before the authz check
|
||||
only to read `.is_public`; running the already-required authz first and
|
||||
fetching only on denial saves one SELECT per authorized private-calendar
|
||||
read). Behavior-preserving (the OR commutes) but it reorders an authz
|
||||
check relative to a data fetch — flagged for maintainer sign-off per the
|
||||
authz-change convention, with the bench sketch in this round's notes.
|
||||
- **Per-page batched parent resolution** for §10 — would cut the cold
|
||||
first view's N parent PK reads to one `= ANY` per page; needs a wider
|
||||
engine API (batch check) — future round.
|
||||
- **`batch_operations` `Arc<str>` → `Option<&str>` widening** (ROUND7
|
||||
deferred) — re-audited: 1 small alloc/item vs a per-item DB roundtrip;
|
||||
still not worth the 2-trait/7-site churn alone. Standing verdict.
|
||||
- **JWT-claims `Arc<str>`** (ROUND6 deferred) — still open; touches
|
||||
serde `rc` on `TokenClaims` + dozens of read sites. The 2 allocs/request
|
||||
remain the cheapest known win on the /api path for a future round.
|
||||
|
||||
## Correctness-adjacent (surfaced by the round-9 hunt — not perf)
|
||||
|
||||
- `trash_service.rs` restore matches error text
|
||||
(`format!("{}", e).contains("not found")`) instead of
|
||||
`e.kind == ErrorKind::NotFound` — fragile to rewording; flagged.
|
||||
- The round-7 flags remain open: `fetchFolderListing` seeds empty
|
||||
`favoriteIds`/`sharedIds`; the search page still lacks a stale-response
|
||||
guard.
|
||||
@@ -0,0 +1,157 @@
|
||||
//! OCS capabilities poll benchmark — rebuild-per-request vs memoized bytes.
|
||||
//!
|
||||
//! `/ocs/v{1,2}.php/cloud/capabilities` returns a payload that is
|
||||
//! process-invariant (pure config: base URL + emulated NC version), yet
|
||||
//! every NC desktop/mobile client polls it on connect and periodically.
|
||||
//! The old handler re-built the ~40-node `json!` tree — including a
|
||||
//! `std::env::var("OXICLOUD_BASE_URL")` lookup and three `format!`s —
|
||||
//! and re-serialized it on EVERY poll. Round 9 serializes both versions
|
||||
//! once into a `OnceLock<[Bytes; 2]>`; a poll is a `Bytes` refcount bump.
|
||||
//!
|
||||
//! The BEFORE arm is the production payload builder invoked per request
|
||||
//! (via the bench wrapper) + `serde_json::to_vec`, exactly the old
|
||||
//! handler flow (`Json(payload)` serializes with `to_vec`). The AFTER
|
||||
//! arm is the memoized-bytes flow. The equivalence gate asserts the
|
||||
//! served bytes are identical.
|
||||
//!
|
||||
//! Run (no Postgres needed):
|
||||
//! cargo run --release --features bench --example bench_capabilities_static
|
||||
//! Tunables (env): BENCH_POLLS (50000)
|
||||
|
||||
use std::alloc::{GlobalAlloc, Layout, System};
|
||||
use std::env;
|
||||
use std::hint::black_box;
|
||||
use std::sync::OnceLock;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
use std::time::Instant;
|
||||
|
||||
use bytes::Bytes;
|
||||
use oxicloud::interfaces::nextcloud::ocs_handler::capabilities_payload_for_bench;
|
||||
|
||||
static ALLOC_CALLS: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
struct CountingAlloc;
|
||||
|
||||
unsafe impl GlobalAlloc for CountingAlloc {
|
||||
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.alloc(layout) }
|
||||
}
|
||||
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
|
||||
unsafe { System.dealloc(ptr, layout) }
|
||||
}
|
||||
unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.realloc(ptr, layout, new_size) }
|
||||
}
|
||||
unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.alloc_zeroed(layout) }
|
||||
}
|
||||
}
|
||||
|
||||
#[global_allocator]
|
||||
static GLOBAL: CountingAlloc = CountingAlloc;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
const EMULATED: (u32, u32, u32) = (28, 0, 4);
|
||||
const VERSION_STRING: &str = "28.0.4";
|
||||
|
||||
/// BEFORE flow, verbatim shape: env lookup + tree build + serialize per poll.
|
||||
fn before_poll(ocs_version: u8) -> Vec<u8> {
|
||||
let base_url =
|
||||
env::var("OXICLOUD_BASE_URL").unwrap_or_else(|_| "http://localhost:8086".to_string());
|
||||
let payload = capabilities_payload_for_bench(&base_url, EMULATED, VERSION_STRING, ocs_version);
|
||||
serde_json::to_vec(&payload).expect("serialize")
|
||||
}
|
||||
|
||||
/// AFTER flow: the production memoization shape (OnceLock + Bytes clone).
|
||||
fn after_poll(cache: &OnceLock<[Bytes; 2]>, ocs_version: u8) -> Bytes {
|
||||
let bodies = cache.get_or_init(|| {
|
||||
let base_url =
|
||||
env::var("OXICLOUD_BASE_URL").unwrap_or_else(|_| "http://localhost:8086".to_string());
|
||||
[1u8, 2u8].map(|v| {
|
||||
Bytes::from(
|
||||
serde_json::to_vec(&capabilities_payload_for_bench(
|
||||
&base_url,
|
||||
EMULATED,
|
||||
VERSION_STRING,
|
||||
v,
|
||||
))
|
||||
.expect("serialize"),
|
||||
)
|
||||
})
|
||||
});
|
||||
bodies[usize::from(ocs_version != 1)].clone()
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let polls: usize = env_or("BENCH_POLLS", 50_000);
|
||||
let cache: OnceLock<[Bytes; 2]> = OnceLock::new();
|
||||
|
||||
// Equivalence gate: identical served bytes for both OCS versions.
|
||||
for v in [1u8, 2u8] {
|
||||
assert_eq!(
|
||||
before_poll(v),
|
||||
after_poll(&cache, v).as_ref(),
|
||||
"capabilities v{v} bytes differ"
|
||||
);
|
||||
}
|
||||
println!("# equivalence gate: v1 + v2 served bytes identical — OK");
|
||||
|
||||
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
for i in 0..polls {
|
||||
black_box(before_poll(if i % 2 == 0 { 1 } else { 2 }));
|
||||
}
|
||||
let before_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let before_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
|
||||
|
||||
let a1 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
for i in 0..polls {
|
||||
black_box(after_poll(&cache, if i % 2 == 0 { 1 } else { 2 }));
|
||||
}
|
||||
let after_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let after_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1;
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# OCS capabilities poll — rebuild+serialize vs memoized Bytes");
|
||||
println!("# polls={polls}");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<26} | {:>10} | {:>12} | {:>12} |",
|
||||
"arm", "wall ms", "allocs", "allocs/poll"
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.1} | {:>12} | {:>12.2} |",
|
||||
"BEFORE (rebuild)",
|
||||
before_ms,
|
||||
before_allocs,
|
||||
before_allocs as f64 / polls as f64
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.1} | {:>12} | {:>12.2} |",
|
||||
"AFTER (memoized)",
|
||||
after_ms,
|
||||
after_allocs,
|
||||
after_allocs as f64 / polls as f64
|
||||
);
|
||||
println!(
|
||||
"\n{:.1}x faster, {:.0}x fewer allocs",
|
||||
before_ms / after_ms,
|
||||
before_allocs as f64 / after_allocs.max(1) as f64
|
||||
);
|
||||
|
||||
if after_ms >= before_ms || after_allocs >= before_allocs {
|
||||
eprintln!("GATE FAIL: memoized arm not strictly better — rollback");
|
||||
std::process::exit(1);
|
||||
}
|
||||
println!("GATE PASS");
|
||||
}
|
||||
@@ -0,0 +1,218 @@
|
||||
//! `Drive::is_empty` benchmark — full-drive `COUNT(*)` sum vs short-circuit
|
||||
//! `EXISTS OR EXISTS`.
|
||||
//!
|
||||
//! The drive-deletion precheck only needs a boolean, but the old query
|
||||
//! aggregated every live folder AND file in the drive (two full index/heap
|
||||
//! scans) to compare the sum with 0. `EXISTS` stops at the first matching
|
||||
//! row, so a populated drive answers from one probe.
|
||||
//!
|
||||
//! Both query shapes run against the same seeded data; the equivalence
|
||||
//! gate asserts identical booleans for a populated and an empty drive.
|
||||
//!
|
||||
//! Run (needs Postgres up; reads DATABASE_URL from .env):
|
||||
//! cargo run --release --features bench --example bench_drive_is_empty
|
||||
//! Tunables (env): BENCH_FILES (100000), BENCH_REPS (25)
|
||||
|
||||
use std::env;
|
||||
use std::time::Instant;
|
||||
|
||||
use sqlx::PgPool;
|
||||
use sqlx::postgres::PgPoolOptions;
|
||||
use uuid::Uuid;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
async fn seed_drive(pool: &PgPool, files: usize) -> Uuid {
|
||||
// Drive + root folder must commit together (deferred root-folder trigger).
|
||||
let mut tx = pool.begin().await.expect("begin");
|
||||
let drive_id: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO storage.drives (kind, quota_bytes) VALUES ('shared', NULL) RETURNING id",
|
||||
)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("drive");
|
||||
let root: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO storage.folders (name, path, lpath, drive_id)
|
||||
VALUES ('bench_is_empty', '/bench_is_empty', 'bench_is_empty', $1)
|
||||
RETURNING id",
|
||||
)
|
||||
.bind(drive_id)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("root");
|
||||
sqlx::query("UPDATE storage.drives SET root_folder_id = $1 WHERE id = $2")
|
||||
.bind(root)
|
||||
.bind(drive_id)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.expect("stamp root");
|
||||
tx.commit().await.expect("commit");
|
||||
|
||||
if files > 0 {
|
||||
sqlx::query(
|
||||
"INSERT INTO storage.files (name, folder_id, blob_hash, size, mime_type, drive_id)
|
||||
SELECT 'f' || i, $1,
|
||||
'benchempty00000000000000000000000000000000000000000000000000000',
|
||||
1024, 'image/jpeg', $2
|
||||
FROM generate_series(1, $3) AS i",
|
||||
)
|
||||
.bind(root)
|
||||
.bind(drive_id)
|
||||
.bind(files as i32)
|
||||
.execute(pool)
|
||||
.await
|
||||
.expect("seed files");
|
||||
}
|
||||
drive_id
|
||||
}
|
||||
|
||||
async fn cleanup(pool: &PgPool, drive_id: Uuid) {
|
||||
sqlx::query("DELETE FROM storage.files WHERE drive_id = $1")
|
||||
.bind(drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("UPDATE storage.drives SET root_folder_id = NULL WHERE id = $1")
|
||||
.bind(drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM storage.folders WHERE drive_id = $1")
|
||||
.bind(drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM storage.drives WHERE id = $1")
|
||||
.bind(drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
}
|
||||
|
||||
/// BEFORE — verbatim old query shape.
|
||||
async fn is_empty_count(pool: &PgPool, drive_id: Uuid) -> bool {
|
||||
let count: (i64,) = sqlx::query_as(
|
||||
r#"
|
||||
SELECT (
|
||||
(SELECT COUNT(*) FROM storage.folders
|
||||
WHERE drive_id = $1 AND parent_id IS NOT NULL AND NOT is_trashed)
|
||||
+ (SELECT COUNT(*) FROM storage.files
|
||||
WHERE drive_id = $1 AND NOT is_trashed)
|
||||
)
|
||||
"#,
|
||||
)
|
||||
.bind(drive_id)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.expect("count query");
|
||||
count.0 == 0
|
||||
}
|
||||
|
||||
/// AFTER — the production EXISTS shape.
|
||||
async fn is_empty_exists(pool: &PgPool, drive_id: Uuid) -> bool {
|
||||
let occupied: (bool,) = sqlx::query_as(
|
||||
r#"
|
||||
SELECT EXISTS(
|
||||
SELECT 1 FROM storage.folders
|
||||
WHERE drive_id = $1 AND parent_id IS NOT NULL AND NOT is_trashed)
|
||||
OR EXISTS(
|
||||
SELECT 1 FROM storage.files
|
||||
WHERE drive_id = $1 AND NOT is_trashed)
|
||||
"#,
|
||||
)
|
||||
.bind(drive_id)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.expect("exists query");
|
||||
!occupied.0
|
||||
}
|
||||
|
||||
#[tokio::main(flavor = "current_thread")]
|
||||
async fn main() {
|
||||
dotenvy::dotenv().ok();
|
||||
let url = env::var("DATABASE_URL").expect("set DATABASE_URL — the dev Postgres URL");
|
||||
let pool = PgPoolOptions::new()
|
||||
.max_connections(4)
|
||||
.connect(&url)
|
||||
.await
|
||||
.expect("connect");
|
||||
|
||||
let files: usize = env_or("BENCH_FILES", 100_000);
|
||||
let reps: usize = env_or("BENCH_REPS", 25);
|
||||
|
||||
let populated = seed_drive(&pool, files).await;
|
||||
let empty = seed_drive(&pool, 0).await;
|
||||
|
||||
// Equivalence gate on both data shapes.
|
||||
assert_eq!(
|
||||
is_empty_count(&pool, populated).await,
|
||||
is_empty_exists(&pool, populated).await,
|
||||
"populated drive verdict differs"
|
||||
);
|
||||
assert_eq!(
|
||||
is_empty_count(&pool, empty).await,
|
||||
is_empty_exists(&pool, empty).await,
|
||||
"empty drive verdict differs"
|
||||
);
|
||||
assert!(!is_empty_exists(&pool, populated).await);
|
||||
assert!(is_empty_exists(&pool, empty).await);
|
||||
println!("# equivalence gate: identical booleans on populated + empty drives — OK");
|
||||
|
||||
// Warm both shapes.
|
||||
for _ in 0..3 {
|
||||
is_empty_count(&pool, populated).await;
|
||||
is_empty_exists(&pool, populated).await;
|
||||
}
|
||||
|
||||
let mut rows = Vec::new();
|
||||
for (label, drive) in [("populated (100k files)", populated), ("empty", empty)] {
|
||||
let t = Instant::now();
|
||||
for _ in 0..reps {
|
||||
std::hint::black_box(is_empty_count(&pool, drive).await);
|
||||
}
|
||||
let before_ms = t.elapsed().as_secs_f64() * 1e3 / reps as f64;
|
||||
|
||||
let t = Instant::now();
|
||||
for _ in 0..reps {
|
||||
std::hint::black_box(is_empty_exists(&pool, drive).await);
|
||||
}
|
||||
let after_ms = t.elapsed().as_secs_f64() * 1e3 / reps as f64;
|
||||
rows.push((label, before_ms, after_ms));
|
||||
}
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# Drive::is_empty — COUNT(*) sum vs EXISTS OR EXISTS");
|
||||
println!("# files={files} reps={reps} (ms per call)");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<24} | {:>14} | {:>14} | {:>8} |",
|
||||
"drive", "BEFORE ms", "AFTER ms", "speedup"
|
||||
);
|
||||
let mut populated_gain = 0.0;
|
||||
for (label, before_ms, after_ms) in &rows {
|
||||
println!(
|
||||
"| {:<24} | {:>14.3} | {:>14.3} | {:>7.1}x |",
|
||||
label,
|
||||
before_ms,
|
||||
after_ms,
|
||||
before_ms / after_ms
|
||||
);
|
||||
if label.starts_with("populated") {
|
||||
populated_gain = before_ms / after_ms;
|
||||
}
|
||||
}
|
||||
|
||||
cleanup(&pool, populated).await;
|
||||
cleanup(&pool, empty).await;
|
||||
|
||||
if populated_gain <= 1.0 {
|
||||
eprintln!("\nGATE FAIL: EXISTS not faster on the populated drive — rollback");
|
||||
std::process::exit(1);
|
||||
}
|
||||
println!("\nGATE PASS: identical verdicts, populated drive {populated_gain:.1}x faster.");
|
||||
}
|
||||
@@ -0,0 +1,247 @@
|
||||
//! Folder-listing UUID decode benchmark — `id::text`/`parent_id::text`
|
||||
//! server casts vs binary `Uuid` decode + one app-side render.
|
||||
//!
|
||||
//! Round 6 adopted binary decode for the FILE listing rows
|
||||
//! (`row_to_file`, benches/ROUND6.md §10: 1.17x on 500-row pages) and
|
||||
//! queued "other repos with the same shape" — `FolderDbRepository` never
|
||||
//! got the port. Its rows (`list_folders`, `list_folders_batch` — every
|
||||
//! Depth:1 PROPFIND subfolder page — descendants, suggest) still shipped
|
||||
//! two `::text` casts per row: 36+36 B on the wire instead of 16+16 and
|
||||
//! a server-side cast per column.
|
||||
//!
|
||||
//! Same methodology as `bench_uuid_text_cast` (the round-6 A/B this
|
||||
//! ports): seeded page, equivalence gate on identical `(id, parent_id,
|
||||
//! name, path)` string tuples, warm-up, interleaved passes.
|
||||
//!
|
||||
//! Run (needs Postgres up; reads DATABASE_URL from .env):
|
||||
//! cargo run --release --features bench --example bench_folder_uuid_decode
|
||||
//! Tunables (env): BENCH_ROWS (500), BENCH_PASSES (200)
|
||||
|
||||
use std::env;
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use sqlx::PgPool;
|
||||
use sqlx::postgres::PgPoolOptions;
|
||||
use uuid::Uuid;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
struct Seeded {
|
||||
drive_id: Uuid,
|
||||
parent_id: Uuid,
|
||||
}
|
||||
|
||||
async fn seed(pool: &PgPool, rows: usize) -> Seeded {
|
||||
let mut tx = pool.begin().await.expect("begin");
|
||||
let drive_id: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO storage.drives (kind, quota_bytes) VALUES ('shared', NULL) RETURNING id",
|
||||
)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("drive");
|
||||
let root: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO storage.folders (name, path, lpath, drive_id)
|
||||
VALUES ('bench_uuid_folders', '/bench_uuid_folders', 'bench_uuid_folders', $1)
|
||||
RETURNING id",
|
||||
)
|
||||
.bind(drive_id)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("root");
|
||||
sqlx::query("UPDATE storage.drives SET root_folder_id = $1 WHERE id = $2")
|
||||
.bind(root)
|
||||
.bind(drive_id)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.expect("stamp root");
|
||||
tx.commit().await.expect("commit");
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO storage.folders (name, parent_id, path, lpath, drive_id)
|
||||
SELECT 'sub' || i, $1, '/bench_uuid_folders/sub' || i,
|
||||
('bench_uuid_folders.sub' || i)::ltree, $2
|
||||
FROM generate_series(1, $3) AS i",
|
||||
)
|
||||
.bind(root)
|
||||
.bind(drive_id)
|
||||
.bind(rows as i32)
|
||||
.execute(pool)
|
||||
.await
|
||||
.expect("seed subfolders");
|
||||
|
||||
Seeded {
|
||||
drive_id,
|
||||
parent_id: root,
|
||||
}
|
||||
}
|
||||
|
||||
async fn cleanup(pool: &PgPool, s: &Seeded) {
|
||||
sqlx::query("DELETE FROM storage.folders WHERE drive_id = $1 AND parent_id IS NOT NULL")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("UPDATE storage.drives SET root_folder_id = NULL WHERE id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM storage.folders WHERE drive_id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM storage.drives WHERE id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
}
|
||||
|
||||
/// Materialized tuple both arms must produce identically.
|
||||
type FolderTuple = (String, String, String, Option<String>);
|
||||
|
||||
/// BEFORE — verbatim old query shape: two server-side `::text` casts,
|
||||
/// decode as String.
|
||||
async fn fetch_text_cast(pool: &PgPool, parent_id: Uuid) -> Vec<FolderTuple> {
|
||||
sqlx::query_as::<_, (String, String, String, Option<String>)>(
|
||||
r#"
|
||||
SELECT id::text, name, path, parent_id::text
|
||||
FROM storage.folders
|
||||
WHERE parent_id = $1 AND NOT is_trashed
|
||||
ORDER BY name
|
||||
"#,
|
||||
)
|
||||
.bind(parent_id)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.expect("text-cast fetch")
|
||||
}
|
||||
|
||||
/// AFTER — the production shape: binary decode, one `to_string` app-side
|
||||
/// (exactly what `row_to_folder` does now).
|
||||
async fn fetch_binary_uuid(pool: &PgPool, parent_id: Uuid) -> Vec<FolderTuple> {
|
||||
let rows = sqlx::query_as::<_, (Uuid, String, String, Option<Uuid>)>(
|
||||
r#"
|
||||
SELECT id, name, path, parent_id
|
||||
FROM storage.folders
|
||||
WHERE parent_id = $1 AND NOT is_trashed
|
||||
ORDER BY name
|
||||
"#,
|
||||
)
|
||||
.bind(parent_id)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.expect("binary fetch");
|
||||
rows.into_iter()
|
||||
.map(|(id, name, path, pid)| (id.to_string(), name, path, pid.map(|u| u.to_string())))
|
||||
.collect()
|
||||
}
|
||||
|
||||
struct Stats {
|
||||
mean_ms: f64,
|
||||
p50_ms: f64,
|
||||
p95_ms: f64,
|
||||
}
|
||||
|
||||
fn summarize(mut xs: Vec<f64>) -> Stats {
|
||||
xs.sort_by(|a, b| a.partial_cmp(b).unwrap());
|
||||
let n = xs.len();
|
||||
Stats {
|
||||
mean_ms: xs.iter().sum::<f64>() / n as f64,
|
||||
p50_ms: xs[n / 2],
|
||||
p95_ms: xs[((n as f64 * 0.95) as usize).min(n - 1)],
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::main(flavor = "multi_thread")]
|
||||
async fn main() {
|
||||
dotenvy::dotenv().ok();
|
||||
let url = env::var("DATABASE_URL")
|
||||
.or_else(|_| env::var("OXICLOUD_DB_CONNECTION_STRING"))
|
||||
.expect("set DATABASE_URL — the dev Postgres URL");
|
||||
let rows: usize = env_or("BENCH_ROWS", 500);
|
||||
let passes: usize = env_or("BENCH_PASSES", 200);
|
||||
|
||||
let pool = Arc::new(
|
||||
PgPoolOptions::new()
|
||||
.max_connections(4)
|
||||
.min_connections(4)
|
||||
.acquire_timeout(Duration::from_secs(10))
|
||||
.connect(&url)
|
||||
.await
|
||||
.expect("connect Postgres"),
|
||||
);
|
||||
|
||||
let seeded = seed(&pool, rows).await;
|
||||
|
||||
// Equivalence gate: identical string tuples in identical order.
|
||||
let a = fetch_text_cast(&pool, seeded.parent_id).await;
|
||||
let b = fetch_binary_uuid(&pool, seeded.parent_id).await;
|
||||
if a != b || a.len() != rows {
|
||||
eprintln!(
|
||||
"EQUIVALENCE GATE FAILED: rows differ (a={}, b={})",
|
||||
a.len(),
|
||||
b.len()
|
||||
);
|
||||
cleanup(&pool, &seeded).await;
|
||||
std::process::exit(1);
|
||||
}
|
||||
println!("# equivalence gate: {rows} identical (id, name, path, parent_id) tuples — OK");
|
||||
|
||||
for _ in 0..10 {
|
||||
std::hint::black_box(fetch_text_cast(&pool, seeded.parent_id).await);
|
||||
std::hint::black_box(fetch_binary_uuid(&pool, seeded.parent_id).await);
|
||||
}
|
||||
|
||||
// Interleaved A/B passes so drift (autovacuum, CPU governor) hits both.
|
||||
let mut lat_a = Vec::with_capacity(passes);
|
||||
let mut lat_b = Vec::with_capacity(passes);
|
||||
for _ in 0..passes {
|
||||
let t = Instant::now();
|
||||
std::hint::black_box(fetch_text_cast(&pool, seeded.parent_id).await);
|
||||
lat_a.push(t.elapsed().as_secs_f64() * 1e3);
|
||||
let t = Instant::now();
|
||||
std::hint::black_box(fetch_binary_uuid(&pool, seeded.parent_id).await);
|
||||
lat_b.push(t.elapsed().as_secs_f64() * 1e3);
|
||||
}
|
||||
|
||||
let sa = summarize(lat_a);
|
||||
let sb = summarize(lat_b);
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# folder page: `::text` casts vs binary UUID decode + app fmt");
|
||||
println!("# rows/page={rows} passes={passes} (interleaved)");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<22} | {:>9} | {:>9} | {:>9} |",
|
||||
"arm", "mean ms", "p50 ms", "p95 ms"
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>9.3} | {:>9.3} | {:>9.3} |",
|
||||
"A ::text (before)", sa.mean_ms, sa.p50_ms, sa.p95_ms
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>9.3} | {:>9.3} | {:>9.3} |",
|
||||
"B binary (after)", sb.mean_ms, sb.p50_ms, sb.p95_ms
|
||||
);
|
||||
println!(
|
||||
"\nB/A mean ratio: {:.3} ({:.2}x)",
|
||||
sb.mean_ms / sa.mean_ms,
|
||||
sa.mean_ms / sb.mean_ms
|
||||
);
|
||||
|
||||
cleanup(&pool, &seeded).await;
|
||||
|
||||
if sb.mean_ms >= sa.mean_ms {
|
||||
eprintln!("GATE FAIL: binary decode not faster than ::text — rollback");
|
||||
std::process::exit(1);
|
||||
}
|
||||
println!("GATE PASS");
|
||||
}
|
||||
@@ -135,9 +135,15 @@ fn suggest_before(files: &[File], q: &str) -> Vec<SearchSuggestionItem> {
|
||||
item_type: "file".to_string(),
|
||||
id: file_dto.id.clone(),
|
||||
path: file_dto.path.clone(),
|
||||
icon_class: icon_class_for(&file_dto.name, &file_dto.mime_type).to_string(),
|
||||
// `.into()` bridges the round-9 `Arc<str>` field type; the
|
||||
// conversion is identical on both arms so the round-5 delta
|
||||
// this bench gates (clone vs move) is unaffected.
|
||||
icon_class: icon_class_for(&file_dto.name, &file_dto.mime_type)
|
||||
.to_string()
|
||||
.into(),
|
||||
icon_special_class: icon_special_class_for(&file_dto.name, &file_dto.mime_type)
|
||||
.to_string(),
|
||||
.to_string()
|
||||
.into(),
|
||||
relevance_score: score,
|
||||
});
|
||||
}
|
||||
@@ -159,8 +165,9 @@ fn suggest_after(files: Vec<File>, q: &str) -> Vec<SearchSuggestionItem> {
|
||||
item_type: "file".to_string(),
|
||||
id: file_dto.id,
|
||||
path: file_dto.path,
|
||||
icon_class,
|
||||
icon_special_class,
|
||||
// Same `.into()` bridge as the BEFORE arm — see note there.
|
||||
icon_class: icon_class.into(),
|
||||
icon_special_class: icon_special_class.into(),
|
||||
relevance_score: score,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -0,0 +1,369 @@
|
||||
//! NC PROPFIND per-page enrichment — 3 serial round-trips vs `tokio::join!`.
|
||||
//!
|
||||
//! Every Depth:1 PROPFIND page on the NextCloud surface enriches its ≤500
|
||||
//! children with three INDEPENDENT batched reads: favorites
|
||||
//! (`user_favorites … = ANY`), oc:fileid resolution
|
||||
//! (`nextcloud_object_ids … = ANY`) and WebDAV dead properties
|
||||
//! (`webdav_dead_properties … = ANY`). The old code awaited them in
|
||||
//! sequence — 3×RTT per page; overlapping them costs ~max(RTT).
|
||||
//!
|
||||
//! Decide-by-bench (the round-7 deferred "serial pairs" item): round 6
|
||||
//! showed concurrency can LOSE on local-socket PG (authz `try_join_all`
|
||||
//! regressed), so this A/B carries an **injected-latency arm** — each
|
||||
//! round-trip is prefixed with `tokio::time::sleep(L)` to model network
|
||||
//! RTT at L = 0 / 0.25 / 1 / 5 ms. Adoption rule: `join!` must not
|
||||
//! regress at L=0 (the local-socket floor) and must win under injected
|
||||
//! RTT; the L=0 row is the rollback gate.
|
||||
//!
|
||||
//! The three queries are the production shapes bound over the same seeded
|
||||
//! 500-child page; the equivalence gate asserts both arms return
|
||||
//! identical favorite sets / id maps / dead-prop rows.
|
||||
//!
|
||||
//! Run (needs Postgres up; reads DATABASE_URL from .env):
|
||||
//! cargo run --release --features bench --example bench_nc_enrich_join
|
||||
//! Tunables (env): BENCH_CHILDREN (500), BENCH_PASSES (100)
|
||||
|
||||
use std::collections::HashSet;
|
||||
use std::env;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use sqlx::{PgPool, Row, postgres::PgPoolOptions};
|
||||
use uuid::Uuid;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
struct Seeded {
|
||||
drive_id: Uuid,
|
||||
user_id: Uuid,
|
||||
file_ids: Vec<Uuid>,
|
||||
}
|
||||
|
||||
async fn seed(pool: &PgPool, children: usize) -> Seeded {
|
||||
let user_id: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO auth.users (username, email, role)
|
||||
VALUES ('bench_enrich', 'bench_enrich@example.com', 'user') RETURNING id",
|
||||
)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.expect("seed user");
|
||||
|
||||
let mut tx = pool.begin().await.expect("begin");
|
||||
let drive_id: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO storage.drives (kind, quota_bytes) VALUES ('shared', NULL) RETURNING id",
|
||||
)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("drive");
|
||||
let root: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO storage.folders (name, path, lpath, drive_id)
|
||||
VALUES ('bench_enrich', '/bench_enrich', 'bench_enrich', $1) RETURNING id",
|
||||
)
|
||||
.bind(drive_id)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("root");
|
||||
sqlx::query("UPDATE storage.drives SET root_folder_id = $1 WHERE id = $2")
|
||||
.bind(root)
|
||||
.bind(drive_id)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.expect("stamp root");
|
||||
tx.commit().await.expect("commit");
|
||||
|
||||
let file_ids: Vec<Uuid> = sqlx::query_scalar(
|
||||
"INSERT INTO storage.files (name, folder_id, blob_hash, size, mime_type, drive_id)
|
||||
SELECT 'f' || i, $1,
|
||||
'benchenrich0000000000000000000000000000000000000000000000000000',
|
||||
1024, 'image/jpeg', $2
|
||||
FROM generate_series(1, $3) AS i
|
||||
RETURNING id",
|
||||
)
|
||||
.bind(root)
|
||||
.bind(drive_id)
|
||||
.bind(children as i32)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.expect("seed files");
|
||||
|
||||
// Every 5th file favorited, all files carry an oc:fileid mapping,
|
||||
// every 10th file has a dead property — a realistic mixed page.
|
||||
sqlx::query(
|
||||
"INSERT INTO auth.user_favorites (user_id, item_id, item_type)
|
||||
SELECT $1, id::text, 'file' FROM storage.files
|
||||
WHERE folder_id = $2 AND (('x' || substr(md5(id::text), 1, 4))::bit(16)::int % 5) = 0",
|
||||
)
|
||||
.bind(user_id)
|
||||
.bind(root)
|
||||
.execute(pool)
|
||||
.await
|
||||
.expect("seed favorites");
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO storage.nextcloud_object_ids (object_type, object_id)
|
||||
SELECT 'file', id FROM storage.files WHERE folder_id = $1
|
||||
ON CONFLICT DO NOTHING",
|
||||
)
|
||||
.bind(root)
|
||||
.execute(pool)
|
||||
.await
|
||||
.expect("seed object ids");
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO storage.webdav_dead_properties (file_id, namespace, local_name, value)
|
||||
SELECT id, 'urn:bench', 'displayname', 'v'
|
||||
FROM storage.files
|
||||
WHERE folder_id = $1 AND (('x' || substr(md5(id::text), 1, 4))::bit(16)::int % 10) = 0",
|
||||
)
|
||||
.bind(root)
|
||||
.execute(pool)
|
||||
.await
|
||||
.expect("seed dead props");
|
||||
|
||||
Seeded {
|
||||
drive_id,
|
||||
user_id,
|
||||
file_ids,
|
||||
}
|
||||
}
|
||||
|
||||
async fn cleanup(pool: &PgPool, s: &Seeded) {
|
||||
sqlx::query("DELETE FROM storage.webdav_dead_properties WHERE file_id = ANY($1)")
|
||||
.bind(&s.file_ids)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM storage.nextcloud_object_ids WHERE object_id = ANY($1)")
|
||||
.bind(&s.file_ids)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM auth.user_favorites WHERE user_id = $1")
|
||||
.bind(s.user_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM storage.files WHERE drive_id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("UPDATE storage.drives SET root_folder_id = NULL WHERE id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM storage.folders WHERE drive_id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM storage.drives WHERE id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM auth.users WHERE id = $1")
|
||||
.bind(s.user_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
}
|
||||
|
||||
// ── The three production-shaped round-trips ─────────────────────────────────
|
||||
|
||||
async fn q_favorites(
|
||||
pool: &PgPool,
|
||||
user_id: Uuid,
|
||||
ids: &[String],
|
||||
lat: Duration,
|
||||
) -> HashSet<String> {
|
||||
if !lat.is_zero() {
|
||||
tokio::time::sleep(lat).await;
|
||||
}
|
||||
let id_refs: Vec<&str> = ids.iter().map(String::as_str).collect();
|
||||
sqlx::query("SELECT item_id FROM auth.user_favorites WHERE user_id = $1 AND item_id = ANY($2)")
|
||||
.bind(user_id)
|
||||
.bind(&id_refs)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.expect("favorites")
|
||||
.into_iter()
|
||||
.map(|r| r.get::<String, _>(0))
|
||||
.collect()
|
||||
}
|
||||
|
||||
async fn q_object_ids(pool: &PgPool, uuids: &[Uuid], lat: Duration) -> Vec<(i64, Uuid)> {
|
||||
if !lat.is_zero() {
|
||||
tokio::time::sleep(lat).await;
|
||||
}
|
||||
let mut rows: Vec<(i64, Uuid)> = sqlx::query(
|
||||
"SELECT id, object_id FROM storage.nextcloud_object_ids
|
||||
WHERE object_type = 'file' AND object_id = ANY($1::uuid[])",
|
||||
)
|
||||
.bind(uuids)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.expect("object ids")
|
||||
.into_iter()
|
||||
.map(|r| (r.get::<i64, _>(0), r.get::<Uuid, _>(1)))
|
||||
.collect();
|
||||
rows.sort_unstable();
|
||||
rows
|
||||
}
|
||||
|
||||
async fn q_dead_props(pool: &PgPool, uuids: &[Uuid], lat: Duration) -> Vec<(Uuid, String)> {
|
||||
if !lat.is_zero() {
|
||||
tokio::time::sleep(lat).await;
|
||||
}
|
||||
let mut rows: Vec<(Uuid, String)> = sqlx::query(
|
||||
"SELECT file_id, local_name FROM storage.webdav_dead_properties
|
||||
WHERE file_id = ANY($1)",
|
||||
)
|
||||
.bind(uuids)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.expect("dead props")
|
||||
.into_iter()
|
||||
.map(|r| (r.get::<Uuid, _>(0), r.get::<String, _>(1)))
|
||||
.collect();
|
||||
rows.sort_unstable();
|
||||
rows
|
||||
}
|
||||
|
||||
type PageResult = (HashSet<String>, Vec<(i64, Uuid)>, Vec<(Uuid, String)>);
|
||||
|
||||
/// BEFORE — the old serial shape.
|
||||
async fn page_serial(
|
||||
pool: &PgPool,
|
||||
user_id: Uuid,
|
||||
ids: &[String],
|
||||
uuids: &[Uuid],
|
||||
lat: Duration,
|
||||
) -> PageResult {
|
||||
let favs = q_favorites(pool, user_id, ids, lat).await;
|
||||
let oc = q_object_ids(pool, uuids, lat).await;
|
||||
let dead = q_dead_props(pool, uuids, lat).await;
|
||||
(favs, oc, dead)
|
||||
}
|
||||
|
||||
/// AFTER — the production `join!` shape.
|
||||
async fn page_joined(
|
||||
pool: &PgPool,
|
||||
user_id: Uuid,
|
||||
ids: &[String],
|
||||
uuids: &[Uuid],
|
||||
lat: Duration,
|
||||
) -> PageResult {
|
||||
let (favs, oc, dead) = tokio::join!(
|
||||
q_favorites(pool, user_id, ids, lat),
|
||||
q_object_ids(pool, uuids, lat),
|
||||
q_dead_props(pool, uuids, lat),
|
||||
);
|
||||
(favs, oc, dead)
|
||||
}
|
||||
|
||||
fn p50(mut xs: Vec<f64>) -> f64 {
|
||||
xs.sort_by(|a, b| a.partial_cmp(b).unwrap());
|
||||
xs[xs.len() / 2]
|
||||
}
|
||||
|
||||
#[tokio::main(flavor = "multi_thread")]
|
||||
async fn main() {
|
||||
dotenvy::dotenv().ok();
|
||||
let url = env::var("DATABASE_URL").expect("set DATABASE_URL — the dev Postgres URL");
|
||||
let children: usize = env_or("BENCH_CHILDREN", 500);
|
||||
let passes: usize = env_or("BENCH_PASSES", 100);
|
||||
|
||||
// 4 connections: the production pool always has slack beyond 3.
|
||||
let pool = PgPoolOptions::new()
|
||||
.max_connections(4)
|
||||
.min_connections(4)
|
||||
.connect(&url)
|
||||
.await
|
||||
.expect("connect");
|
||||
|
||||
let seeded = seed(&pool, children).await;
|
||||
let ids: Vec<String> = seeded.file_ids.iter().map(|u| u.to_string()).collect();
|
||||
let uuids = seeded.file_ids.clone();
|
||||
|
||||
// Equivalence gate.
|
||||
let a = page_serial(&pool, seeded.user_id, &ids, &uuids, Duration::ZERO).await;
|
||||
let b = page_joined(&pool, seeded.user_id, &ids, &uuids, Duration::ZERO).await;
|
||||
if a != b {
|
||||
eprintln!("EQUIVALENCE GATE FAILED: serial and joined results differ");
|
||||
cleanup(&pool, &seeded).await;
|
||||
std::process::exit(1);
|
||||
}
|
||||
assert!(
|
||||
!a.0.is_empty() && !a.1.is_empty() && !a.2.is_empty(),
|
||||
"seed produced empty enrichment"
|
||||
);
|
||||
println!(
|
||||
"# equivalence gate: identical results (favs={}, oc_ids={}, dead={}) — OK",
|
||||
a.0.len(),
|
||||
a.1.len(),
|
||||
a.2.len()
|
||||
);
|
||||
|
||||
for _ in 0..10 {
|
||||
std::hint::black_box(
|
||||
page_serial(&pool, seeded.user_id, &ids, &uuids, Duration::ZERO).await,
|
||||
);
|
||||
std::hint::black_box(
|
||||
page_joined(&pool, seeded.user_id, &ids, &uuids, Duration::ZERO).await,
|
||||
);
|
||||
}
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# NC PROPFIND page enrichment — serial 3×RTT vs tokio::join!");
|
||||
println!("# children={children} passes={passes} (interleaved, p50 ms/page)");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<14} | {:>12} | {:>12} | {:>8} |",
|
||||
"injected RTT", "serial ms", "join! ms", "ratio"
|
||||
);
|
||||
|
||||
let mut zero_lat_ratio = 0.0;
|
||||
for lat_us in [0u64, 250, 1_000, 5_000] {
|
||||
let lat = Duration::from_micros(lat_us);
|
||||
let mut serial = Vec::with_capacity(passes);
|
||||
let mut joined = Vec::with_capacity(passes);
|
||||
for _ in 0..passes {
|
||||
let t = Instant::now();
|
||||
std::hint::black_box(page_serial(&pool, seeded.user_id, &ids, &uuids, lat).await);
|
||||
serial.push(t.elapsed().as_secs_f64() * 1e3);
|
||||
let t = Instant::now();
|
||||
std::hint::black_box(page_joined(&pool, seeded.user_id, &ids, &uuids, lat).await);
|
||||
joined.push(t.elapsed().as_secs_f64() * 1e3);
|
||||
}
|
||||
let (s, j) = (p50(serial), p50(joined));
|
||||
if lat_us == 0 {
|
||||
zero_lat_ratio = j / s;
|
||||
}
|
||||
println!(
|
||||
"| {:>11} µs | {:>12.3} | {:>12.3} | {:>7.2}x |",
|
||||
lat_us,
|
||||
s,
|
||||
j,
|
||||
s / j
|
||||
);
|
||||
}
|
||||
|
||||
cleanup(&pool, &seeded).await;
|
||||
|
||||
// Adoption gate: join! must not regress the local-socket floor by >5%
|
||||
// (measurement noise band); the injected-RTT rows document the win.
|
||||
if zero_lat_ratio > 1.05 {
|
||||
eprintln!(
|
||||
"\nGATE FAIL: join! is {:.1}% slower at 0 RTT — rollback the overlap",
|
||||
(zero_lat_ratio - 1.0) * 100.0
|
||||
);
|
||||
std::process::exit(1);
|
||||
}
|
||||
println!("\nGATE PASS: no local-socket regression; overlap wins under injected RTT.");
|
||||
}
|
||||
@@ -0,0 +1,334 @@
|
||||
//! NextCloud per-request session benchmark — deep-clone vs `Arc` end-to-end.
|
||||
//!
|
||||
//! Every authenticated NC request (all six DAV dispatchers + OCS) extracts
|
||||
//! the session. The old pipeline paid, per request:
|
||||
//!
|
||||
//! • extractor: `(**arc).clone()` — a DEEP clone of `NcSession`
|
||||
//! (`CurrentUser` 3 Strings + `raw_username` + chroot `FolderDto`
|
||||
//! ~5 Strings ≈ 8-9 heap allocs) despite the doc claiming "one Arc
|
||||
//! increment";
|
||||
//! • chroot cache hit: moka `get` clones the stored `FolderDto` by value
|
||||
//! (~5 more allocs) on the markerless (default-drive) branch;
|
||||
//! • session build: `CurrentUser` built then cloned for the extension,
|
||||
//! `raw_username` cloned, `user_id.to_string()` for the span.
|
||||
//!
|
||||
//! Round 9 stores `Arc<FolderDto>` in the cache, shares one
|
||||
//! `Arc<CurrentUser>` between the extension and the session, and extracts
|
||||
//! `SharedNcSession` (an `Arc` handle that derefs to `NcSession`).
|
||||
//!
|
||||
//! `mod before` replicates the old struct shapes + clone flows verbatim;
|
||||
//! equivalence gates assert every field consumed by handlers is identical.
|
||||
//!
|
||||
//! Sections:
|
||||
//! 1. Extractor — allocs/extract + ns/extract (BEFORE deep clone vs
|
||||
//! AFTER production `SharedNcSession::from_request_parts`)
|
||||
//! 2. Chroot-cache hit — allocs/hit (FolderDto-by-value vs Arc)
|
||||
//! 3. Session build — allocs/build (double CurrentUser + clones vs
|
||||
//! single shared Arc + moves)
|
||||
//!
|
||||
//! Run (no Postgres needed):
|
||||
//! cargo run --release --features bench --example bench_nc_session
|
||||
//! Tunables (env): BENCH_REQS (100000)
|
||||
|
||||
use std::alloc::{GlobalAlloc, Layout, System};
|
||||
use std::env;
|
||||
use std::hint::black_box;
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
use std::time::Instant;
|
||||
|
||||
use axum::extract::FromRequestParts;
|
||||
use oxicloud::application::dtos::folder_dto::FolderDto;
|
||||
use oxicloud::interfaces::middleware::auth::CurrentUser;
|
||||
use oxicloud::interfaces::nextcloud::session::{NcSession, SharedNcSession};
|
||||
|
||||
// ─── Counting allocator ─────────────────────────────────────────────────────
|
||||
|
||||
static ALLOC_CALLS: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
struct CountingAlloc;
|
||||
|
||||
unsafe impl GlobalAlloc for CountingAlloc {
|
||||
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.alloc(layout) }
|
||||
}
|
||||
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
|
||||
unsafe { System.dealloc(ptr, layout) }
|
||||
}
|
||||
unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.realloc(ptr, layout, new_size) }
|
||||
}
|
||||
unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.alloc_zeroed(layout) }
|
||||
}
|
||||
}
|
||||
|
||||
#[global_allocator]
|
||||
static GLOBAL: CountingAlloc = CountingAlloc;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
// ─── BEFORE replicas (verbatim old shapes) ──────────────────────────────────
|
||||
|
||||
mod before {
|
||||
use super::*;
|
||||
|
||||
/// Old `NcSession` shape: owned `CurrentUser`, chroot by value.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct OldNcSession {
|
||||
pub user: CurrentUser,
|
||||
pub raw_username: String,
|
||||
pub chroot: Option<FolderDto>,
|
||||
}
|
||||
|
||||
/// Old extractor body: deep clone out of the shared Arc.
|
||||
pub fn extract(arc: &Arc<OldNcSession>) -> OldNcSession {
|
||||
(**arc).clone()
|
||||
}
|
||||
}
|
||||
|
||||
fn fixture_folder() -> FolderDto {
|
||||
FolderDto {
|
||||
id: uuid::Uuid::new_v4().to_string(),
|
||||
name: "Personal".to_string(),
|
||||
path: "Personal".to_string(),
|
||||
parent_id: None,
|
||||
drive_id: uuid::Uuid::new_v4(),
|
||||
created_at: 1_700_000_000,
|
||||
modified_at: 1_700_000_100,
|
||||
is_root: true,
|
||||
etag: "8f2e5a1c9b3d4e6f".to_string(),
|
||||
icon_class: Arc::from("fas fa-folder"),
|
||||
icon_special_class: Arc::from("folder-icon"),
|
||||
category: Arc::from("Folder"),
|
||||
created_by: None,
|
||||
updated_by: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn fixture_user(id: uuid::Uuid) -> CurrentUser {
|
||||
CurrentUser {
|
||||
id,
|
||||
username: "alice.longname".to_string(),
|
||||
email: "alice.longname@example.com".to_string(),
|
||||
role: "user".to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::main(flavor = "current_thread")]
|
||||
async fn main() {
|
||||
let reqs: usize = env_or("BENCH_REQS", 100_000);
|
||||
let user_id = uuid::Uuid::new_v4();
|
||||
|
||||
// ── Section 1: extractor ────────────────────────────────────────────────
|
||||
let old_session = Arc::new(before::OldNcSession {
|
||||
user: fixture_user(user_id),
|
||||
raw_username: "alice.longname".to_string(),
|
||||
chroot: Some(fixture_folder()),
|
||||
});
|
||||
let new_session = Arc::new(NcSession {
|
||||
user: Arc::new(fixture_user(user_id)),
|
||||
raw_username: "alice.longname".to_string(),
|
||||
chroot: Some(Arc::new(fixture_folder())),
|
||||
});
|
||||
|
||||
// Equivalence gate: every field handlers consume is identical.
|
||||
{
|
||||
let old = before::extract(&old_session);
|
||||
let (mut parts, _) = axum::http::Request::builder()
|
||||
.uri("/ocs/v2.php/cloud/user")
|
||||
.extension(Arc::clone(&new_session))
|
||||
.body(())
|
||||
.expect("request")
|
||||
.into_parts();
|
||||
let new = SharedNcSession::from_request_parts(&mut parts, &())
|
||||
.await
|
||||
.expect("extract");
|
||||
assert_eq!(old.user.id, new.user.id);
|
||||
assert_eq!(old.user.username, new.user.username);
|
||||
assert_eq!(old.user.email, new.user.email);
|
||||
assert_eq!(old.user.role, new.user.role);
|
||||
assert_eq!(old.raw_username, new.raw_username);
|
||||
let (oc, nc) = (old.chroot.as_ref().unwrap(), new.require_chroot().unwrap());
|
||||
assert_eq!(oc.name, nc.name);
|
||||
assert_eq!(oc.path, nc.path);
|
||||
assert_eq!(oc.etag, nc.etag);
|
||||
println!("# equivalence gate: extracted session fields identical — OK");
|
||||
}
|
||||
|
||||
// The URL cross-check runs in both arms' request flow; the BEFORE arm
|
||||
// replicates only the clone (its cross-check was identical string
|
||||
// compare — unchanged by round 9), so both arms time the same work
|
||||
// minus the measured clone-vs-bump difference.
|
||||
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
for _ in 0..reqs {
|
||||
black_box(before::extract(black_box(&old_session)));
|
||||
}
|
||||
let before_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let before_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
|
||||
|
||||
let (mut parts, _) = axum::http::Request::builder()
|
||||
.uri("/ocs/v2.php/cloud/user")
|
||||
.extension(Arc::clone(&new_session))
|
||||
.body(())
|
||||
.expect("request")
|
||||
.into_parts();
|
||||
let a1 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
for _ in 0..reqs {
|
||||
let s = SharedNcSession::from_request_parts(black_box(&mut parts), &())
|
||||
.await
|
||||
.expect("extract");
|
||||
black_box(&s);
|
||||
}
|
||||
let after_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let after_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1;
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# [1] NC session extractor — deep clone vs Arc handle");
|
||||
println!("# extracts={reqs}");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<26} | {:>10} | {:>12} | {:>14} |",
|
||||
"arm", "wall ms", "allocs", "allocs/extract"
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.1} | {:>12} | {:>14.3} |",
|
||||
"BEFORE (deep clone)",
|
||||
before_ms,
|
||||
before_allocs,
|
||||
before_allocs as f64 / reqs as f64
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.1} | {:>12} | {:>14.3} |",
|
||||
"AFTER (SharedNcSession)",
|
||||
after_ms,
|
||||
after_allocs,
|
||||
after_allocs as f64 / reqs as f64
|
||||
);
|
||||
let s1_ok = after_allocs < before_allocs && after_ms < before_ms;
|
||||
|
||||
// ── Section 2: chroot-cache hit ─────────────────────────────────────────
|
||||
let by_value: moka::sync::Cache<uuid::Uuid, FolderDto> = moka::sync::Cache::new(100);
|
||||
let by_arc: moka::sync::Cache<uuid::Uuid, Arc<FolderDto>> = moka::sync::Cache::new(100);
|
||||
let root_id = uuid::Uuid::new_v4();
|
||||
by_value.insert(root_id, fixture_folder());
|
||||
by_arc.insert(root_id, Arc::new(fixture_folder()));
|
||||
|
||||
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
for _ in 0..reqs {
|
||||
black_box(by_value.get(black_box(&root_id)));
|
||||
}
|
||||
let bv_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let bv_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
|
||||
|
||||
let a1 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
for _ in 0..reqs {
|
||||
black_box(by_arc.get(black_box(&root_id)));
|
||||
}
|
||||
let ba_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let ba_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1;
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# [2] chroot-cache hit — FolderDto by value vs Arc<FolderDto>");
|
||||
println!("# hits={reqs}");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<26} | {:>10} | {:>12} | {:>12} |",
|
||||
"arm", "wall ms", "allocs", "allocs/hit"
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.1} | {:>12} | {:>12.3} |",
|
||||
"BEFORE (by value)",
|
||||
bv_ms,
|
||||
bv_allocs,
|
||||
bv_allocs as f64 / reqs as f64
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.1} | {:>12} | {:>12.3} |",
|
||||
"AFTER (Arc)",
|
||||
ba_ms,
|
||||
ba_allocs,
|
||||
ba_allocs as f64 / reqs as f64
|
||||
);
|
||||
let s2_ok = ba_allocs < bv_allocs;
|
||||
|
||||
// ── Section 3: session build ────────────────────────────────────────────
|
||||
// BEFORE: build CurrentUser, clone it for the extension Arc, clone
|
||||
// raw_username, `to_string` the span value. AFTER: one Arc shared by
|
||||
// extension + session, raw_username moved, span rendered lazily (the
|
||||
// lazy render costs nothing here; the removed `to_string` did).
|
||||
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
for _ in 0..reqs {
|
||||
let raw_username = String::from("alice.longname");
|
||||
let span_value = user_id.to_string();
|
||||
let current_user = fixture_user(user_id);
|
||||
let ext = Arc::new(current_user.clone());
|
||||
let session = Arc::new(before::OldNcSession {
|
||||
user: current_user,
|
||||
raw_username: raw_username.clone(),
|
||||
chroot: None,
|
||||
});
|
||||
black_box((&span_value, &ext, &session));
|
||||
}
|
||||
let sb_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let sb_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
|
||||
|
||||
let a1 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
for _ in 0..reqs {
|
||||
let raw_username = String::from("alice.longname");
|
||||
let current_user = Arc::new(fixture_user(user_id));
|
||||
let ext = Arc::clone(¤t_user);
|
||||
let session = Arc::new(NcSession {
|
||||
user: current_user,
|
||||
raw_username,
|
||||
chroot: None,
|
||||
});
|
||||
black_box((&ext, &session));
|
||||
}
|
||||
let sa_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let sa_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1;
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# [3] session build — double CurrentUser + clones vs shared Arc");
|
||||
println!("# builds={reqs}");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<26} | {:>10} | {:>12} | {:>12} |",
|
||||
"arm", "wall ms", "allocs", "allocs/build"
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.1} | {:>12} | {:>12.3} |",
|
||||
"BEFORE (clone x2 + span)",
|
||||
sb_ms,
|
||||
sb_allocs,
|
||||
sb_allocs as f64 / reqs as f64
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.1} | {:>12} | {:>12.3} |",
|
||||
"AFTER (shared Arc)",
|
||||
sa_ms,
|
||||
sa_allocs,
|
||||
sa_allocs as f64 / reqs as f64
|
||||
);
|
||||
let s3_ok = sa_allocs < sb_allocs;
|
||||
|
||||
if !(s1_ok && s2_ok && s3_ok) {
|
||||
eprintln!("\nGATE FAIL: (extractor={s1_ok} cache={s2_ok} build={s3_ok}) — rollback");
|
||||
std::process::exit(1);
|
||||
}
|
||||
println!("\nGATE PASS: all three session stages allocate less with identical fields.");
|
||||
}
|
||||
@@ -7,6 +7,14 @@
|
||||
//! category classes first (they borrow `&row.name`), then MOVES `row.name`
|
||||
//! into the DTO — the same output, one fewer alloc per row.
|
||||
//!
|
||||
//! Section 2 (round 9): the SAME clone-vs-move port applied to the
|
||||
//! favorites/recents listings (`/api/favorites/resources`,
|
||||
//! `/api/recent/resources`), which the round-7 rewrite never reached. Their
|
||||
//! per-row mapping additionally cloned `row.path` (owner rows) and
|
||||
//! `row.blob_hash` (file rows), so the saving is up to 3 allocs per file row.
|
||||
//! The two handlers share one mapping shape (only the `favorited_at` /
|
||||
//! `accessed_at` passthrough differs), so the favorites row stands for both.
|
||||
//!
|
||||
//! Run:
|
||||
//! cargo run --release --features bench --example bench_resource_row_map
|
||||
//! Tunables (env): BENCH_ROWS (500).
|
||||
@@ -21,6 +29,7 @@ use oxicloud::application::dtos::display_helpers::{
|
||||
category_for, format_file_size, icon_class_for, icon_special_class_for, intern_display,
|
||||
intern_mime,
|
||||
};
|
||||
use oxicloud::application::dtos::favorites_dto::FavoriteResourceRow;
|
||||
use oxicloud::application::dtos::file_dto::FileDto;
|
||||
use oxicloud::application::dtos::folder_dto::{FolderDto, FolderResourceRow};
|
||||
use oxicloud::domain::entities::file::File;
|
||||
@@ -223,6 +232,218 @@ fn map_after(rows: Vec<FolderResourceRow>) -> Vec<Probe> {
|
||||
.collect()
|
||||
}
|
||||
|
||||
// ── Section 2: favorites/recents row→DTO mapping (round 9 port) ─────────────
|
||||
|
||||
fn fav_rows(n: usize) -> Vec<FavoriteResourceRow> {
|
||||
let ts: DateTime<Utc> = Utc.timestamp_opt(1_700_000_000, 0).unwrap();
|
||||
(0..n)
|
||||
.map(|i| {
|
||||
let is_folder = i % 4 == 0;
|
||||
FavoriteResourceRow {
|
||||
resource_type: if is_folder { "folder" } else { "file" }.to_string(),
|
||||
resource_id: Uuid::new_v4(),
|
||||
name: if is_folder {
|
||||
format!("Folder {i:05}")
|
||||
} else {
|
||||
format!("document-{i:05}.pdf")
|
||||
},
|
||||
parent_id: Some(Uuid::new_v4()),
|
||||
mime_type: if is_folder {
|
||||
None
|
||||
} else {
|
||||
Some("application/pdf".to_string())
|
||||
},
|
||||
size: if is_folder { -1 } else { 4096 },
|
||||
resource_created_at: ts,
|
||||
modified_at: ts,
|
||||
drive_id: Uuid::new_v4(),
|
||||
blob_hash: if is_folder {
|
||||
None
|
||||
} else {
|
||||
Some("a".repeat(64))
|
||||
},
|
||||
is_owner: true,
|
||||
favorited_at: ts,
|
||||
path: Some(format!("Documents/Work/item-{i:05}")),
|
||||
sort_str: Some(format!("row {i}")),
|
||||
sort_int: None,
|
||||
sort_ts: None,
|
||||
}
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// (name, path, content_hash, icon_class, category) — every field the
|
||||
/// clone→move rewrite touches on the favorites/recents mapping.
|
||||
type FavProbe = (
|
||||
String,
|
||||
String,
|
||||
String,
|
||||
std::sync::Arc<str>,
|
||||
std::sync::Arc<str>,
|
||||
);
|
||||
|
||||
/// BEFORE — verbatim favorites/recents mapping: `row.path.clone()`,
|
||||
/// `row.name.clone()` (both branches) and `row.blob_hash.clone()`.
|
||||
fn fav_map_before(rows: Vec<FavoriteResourceRow>) -> Vec<FavProbe> {
|
||||
rows.into_iter()
|
||||
.map(|row| {
|
||||
let path = if row.is_owner {
|
||||
row.path.clone().unwrap_or_default()
|
||||
} else {
|
||||
String::new()
|
||||
};
|
||||
if row.resource_type == "folder" {
|
||||
let resource_id = row.resource_id.to_string();
|
||||
let dto = FolderDto {
|
||||
etag: resource_id.clone(),
|
||||
id: resource_id,
|
||||
name: row.name.clone(),
|
||||
path,
|
||||
parent_id: row.parent_id.map(|u| u.to_string()),
|
||||
drive_id: row.drive_id,
|
||||
created_at: row.resource_created_at.timestamp() as u64,
|
||||
modified_at: row.modified_at.timestamp() as u64,
|
||||
is_root: false,
|
||||
icon_class: intern_display("fas fa-folder"),
|
||||
icon_special_class: intern_display("folder-icon"),
|
||||
category: intern_display("Folder"),
|
||||
created_by: None,
|
||||
updated_by: None,
|
||||
};
|
||||
(
|
||||
dto.name,
|
||||
dto.path,
|
||||
String::new(),
|
||||
dto.icon_class,
|
||||
dto.category,
|
||||
)
|
||||
} else {
|
||||
let mime = row
|
||||
.mime_type
|
||||
.as_deref()
|
||||
.unwrap_or("application/octet-stream");
|
||||
let size_bytes = row.size.max(0) as u64;
|
||||
let modified_at_u = row.modified_at.timestamp() as u64;
|
||||
let content_hash = row.blob_hash.clone().unwrap_or_default();
|
||||
let etag = if content_hash.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
File::compute_etag(&content_hash, modified_at_u)
|
||||
};
|
||||
let dto = FileDto {
|
||||
id: row.resource_id.to_string(),
|
||||
name: row.name.clone(),
|
||||
path,
|
||||
size: size_bytes,
|
||||
mime_type: intern_mime(mime),
|
||||
folder_id: row.parent_id.map(|u| u.to_string()),
|
||||
created_at: row.resource_created_at.timestamp() as u64,
|
||||
modified_at: modified_at_u,
|
||||
icon_class: intern_display(icon_class_for(&row.name, mime)),
|
||||
icon_special_class: intern_display(icon_special_class_for(&row.name, mime)),
|
||||
category: intern_display(category_for(&row.name, mime)),
|
||||
size_formatted: format_file_size(size_bytes),
|
||||
sort_date: None,
|
||||
content_hash,
|
||||
etag,
|
||||
created_by: None,
|
||||
updated_by: None,
|
||||
};
|
||||
(
|
||||
dto.name,
|
||||
dto.path,
|
||||
dto.content_hash,
|
||||
dto.icon_class,
|
||||
dto.category,
|
||||
)
|
||||
}
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// AFTER — the round-9 handler code: `path`/`blob_hash` moved, classes
|
||||
/// computed before `row.name` moves.
|
||||
fn fav_map_after(rows: Vec<FavoriteResourceRow>) -> Vec<FavProbe> {
|
||||
rows.into_iter()
|
||||
.map(|row| {
|
||||
let path = if row.is_owner {
|
||||
row.path.unwrap_or_default()
|
||||
} else {
|
||||
String::new()
|
||||
};
|
||||
if row.resource_type == "folder" {
|
||||
let resource_id = row.resource_id.to_string();
|
||||
let dto = FolderDto {
|
||||
etag: resource_id.clone(),
|
||||
id: resource_id,
|
||||
name: row.name,
|
||||
path,
|
||||
parent_id: row.parent_id.map(|u| u.to_string()),
|
||||
drive_id: row.drive_id,
|
||||
created_at: row.resource_created_at.timestamp() as u64,
|
||||
modified_at: row.modified_at.timestamp() as u64,
|
||||
is_root: false,
|
||||
icon_class: intern_display("fas fa-folder"),
|
||||
icon_special_class: intern_display("folder-icon"),
|
||||
category: intern_display("Folder"),
|
||||
created_by: None,
|
||||
updated_by: None,
|
||||
};
|
||||
(
|
||||
dto.name,
|
||||
dto.path,
|
||||
String::new(),
|
||||
dto.icon_class,
|
||||
dto.category,
|
||||
)
|
||||
} else {
|
||||
let mime = row
|
||||
.mime_type
|
||||
.as_deref()
|
||||
.unwrap_or("application/octet-stream");
|
||||
let size_bytes = row.size.max(0) as u64;
|
||||
let modified_at_u = row.modified_at.timestamp() as u64;
|
||||
let content_hash = row.blob_hash.unwrap_or_default();
|
||||
let etag = if content_hash.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
File::compute_etag(&content_hash, modified_at_u)
|
||||
};
|
||||
let icon_class = intern_display(icon_class_for(&row.name, mime));
|
||||
let icon_special_class = intern_display(icon_special_class_for(&row.name, mime));
|
||||
let category = intern_display(category_for(&row.name, mime));
|
||||
let dto = FileDto {
|
||||
id: row.resource_id.to_string(),
|
||||
name: row.name,
|
||||
path,
|
||||
size: size_bytes,
|
||||
mime_type: intern_mime(mime),
|
||||
folder_id: row.parent_id.map(|u| u.to_string()),
|
||||
created_at: row.resource_created_at.timestamp() as u64,
|
||||
modified_at: modified_at_u,
|
||||
icon_class,
|
||||
icon_special_class,
|
||||
category,
|
||||
size_formatted: format_file_size(size_bytes),
|
||||
sort_date: None,
|
||||
content_hash,
|
||||
etag,
|
||||
created_by: None,
|
||||
updated_by: None,
|
||||
};
|
||||
(
|
||||
dto.name,
|
||||
dto.path,
|
||||
dto.content_hash,
|
||||
dto.icon_class,
|
||||
dto.category,
|
||||
)
|
||||
}
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let n: usize = env_or("BENCH_ROWS", 500);
|
||||
|
||||
@@ -279,4 +500,56 @@ fn main() {
|
||||
before_allocs.saturating_sub(after_allocs),
|
||||
(before_allocs.saturating_sub(after_allocs)) as f64 / n as f64
|
||||
);
|
||||
|
||||
// ── Section 2: favorites/recents mapping (round-9 port) ────────────────
|
||||
if fav_map_before(fav_rows(n)) != fav_map_after(fav_rows(n)) {
|
||||
eprintln!("EQUIVALENCE GATE FAILED: favorites mapping output differs");
|
||||
std::process::exit(1);
|
||||
}
|
||||
std::hint::black_box(fav_map_before(fav_rows(n)));
|
||||
std::hint::black_box(fav_map_after(fav_rows(n)));
|
||||
|
||||
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
std::hint::black_box(fav_map_before(fav_rows(n)));
|
||||
let fb_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let fb_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
|
||||
|
||||
let a1 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
std::hint::black_box(fav_map_after(fav_rows(n)));
|
||||
let fa_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let fa_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1;
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# [2] favorites/recents row→DTO mapping: clone path+name+hash vs move");
|
||||
println!("# rows={n} (same mapping shape in both handlers)");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<20} | {:>12} | {:>10} | {:>14} |",
|
||||
"arm", "allocs", "wall ms", "allocs/row"
|
||||
);
|
||||
println!(
|
||||
"| {:<20} | {:>12} | {:>10.3} | {:>14.3} |",
|
||||
"BEFORE (clone)",
|
||||
fb_allocs,
|
||||
fb_ms,
|
||||
fb_allocs as f64 / n as f64
|
||||
);
|
||||
println!(
|
||||
"| {:<20} | {:>12} | {:>10.3} | {:>14.3} |",
|
||||
"AFTER (move)",
|
||||
fa_allocs,
|
||||
fa_ms,
|
||||
fa_allocs as f64 / n as f64
|
||||
);
|
||||
println!(
|
||||
"\nSaved {} allocs ({:.2}/row) — path + name + blob_hash clones removed.",
|
||||
fb_allocs.saturating_sub(fa_allocs),
|
||||
(fb_allocs.saturating_sub(fa_allocs)) as f64 / n as f64
|
||||
);
|
||||
if fa_allocs >= fb_allocs {
|
||||
eprintln!("GATE FAIL: AFTER allocs not below BEFORE — rollback");
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
+218
-6
@@ -13,7 +13,20 @@
|
||||
//!
|
||||
//! Section 2 measures the removed Azure `data.to_vec()` copy in isolation.
|
||||
//!
|
||||
//! Gates: AFTER request count == chunks (vs 2x), AFTER wall < BEFORE wall.
|
||||
//! Section 3 (round 9) drives the same A/B **through the decorator stacks**
|
||||
//! (`RetryBlobBackend`, `CachedBlobBackend`, and the full production
|
||||
//! Cache(Encrypted(Retry(S3))) composition). Until round 9 neither Retry nor
|
||||
//! Cached overrode `put_blob_from_bytes_unsynced`/`sync_blobs`, so the trait
|
||||
//! default silently re-routed every decorated chunk write back through the
|
||||
//! probing synced path — undoing this bench's own Section-1 win on every
|
||||
//! remote deployment with retry or cache enabled. The BEFORE arm is the
|
||||
//! still-present synced route (`put_blob_from_bytes`, byte-identical requests
|
||||
//! to what the fallthrough produced); the AFTER arm is the now-forwarded
|
||||
//! unsynced route. A write-through equivalence gate asserts the Cached stack
|
||||
//! still populates its local cache identically on both routes.
|
||||
//!
|
||||
//! Gates: AFTER request count == chunks (vs 2x), AFTER wall < BEFORE wall,
|
||||
//! per-stack AFTER HEADs == 0, cache population identical on both routes.
|
||||
//!
|
||||
//! No Postgres. Run:
|
||||
//! cargo run --release --features bench --example bench_s3_put
|
||||
@@ -28,6 +41,9 @@ use std::time::{Duration, Instant};
|
||||
use bytes::Bytes;
|
||||
use oxicloud::application::ports::blob_storage_ports::BlobStorageBackend;
|
||||
use oxicloud::common::config::S3StorageConfig;
|
||||
use oxicloud::infrastructure::services::cached_blob_backend::{BlobCacheConfig, CachedBlobBackend};
|
||||
use oxicloud::infrastructure::services::encrypted_blob_backend::EncryptedBlobBackend;
|
||||
use oxicloud::infrastructure::services::retry_blob_backend::{RetryBlobBackend, RetryPolicy};
|
||||
use oxicloud::infrastructure::services::s3_blob_backend::S3BlobBackend;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
@@ -37,6 +53,23 @@ fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
/// Recursively count regular files under `dir` (the blob cache shards blobs
|
||||
/// into 2-hex-char prefix subdirectories).
|
||||
fn count_files(dir: &std::path::Path) -> usize {
|
||||
let mut n = 0;
|
||||
if let Ok(entries) = std::fs::read_dir(dir) {
|
||||
for entry in entries.flatten() {
|
||||
let path = entry.path();
|
||||
if path.is_dir() {
|
||||
n += count_files(&path);
|
||||
} else {
|
||||
n += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
n
|
||||
}
|
||||
|
||||
#[derive(Clone, Default)]
|
||||
struct Counters {
|
||||
heads: Arc<AtomicU64>,
|
||||
@@ -75,11 +108,12 @@ async fn stub_s3(latency: Duration, counters: Counters) -> String {
|
||||
}
|
||||
|
||||
async fn drive(
|
||||
backend: Arc<S3BlobBackend>,
|
||||
backend: Arc<dyn BlobStorageBackend>,
|
||||
chunks: usize,
|
||||
chunk_kb: usize,
|
||||
concurrency: usize,
|
||||
unsynced: bool,
|
||||
hash_prefix: &str,
|
||||
) -> f64 {
|
||||
let payload = Bytes::from(vec![0x5au8; chunk_kb * 1024]);
|
||||
let sem = Arc::new(tokio::sync::Semaphore::new(concurrency));
|
||||
@@ -89,15 +123,17 @@ async fn drive(
|
||||
let b = backend.clone();
|
||||
let p = payload.clone();
|
||||
let sem = sem.clone();
|
||||
let hash = format!("{hash_prefix}{i:060x}");
|
||||
set.spawn(async move {
|
||||
let _permit = sem.acquire().await.expect("sem");
|
||||
let hash = format!("{i:064x}");
|
||||
let n = if unsynced {
|
||||
b.put_blob_from_bytes_unsynced(&hash, p).await.expect("put")
|
||||
} else {
|
||||
b.put_blob_from_bytes(&hash, p).await.expect("put")
|
||||
};
|
||||
assert_eq!(n as usize, chunk_kb * 1024);
|
||||
// Encrypted arms return the ciphertext size (plaintext + AEAD
|
||||
// framing), so gate on >= rather than == for stack generality.
|
||||
assert!(n as usize >= chunk_kb * 1024);
|
||||
});
|
||||
}
|
||||
while let Some(r) = set.join_next().await {
|
||||
@@ -106,6 +142,81 @@ async fn drive(
|
||||
t.elapsed().as_secs_f64() * 1000.0
|
||||
}
|
||||
|
||||
/// Run BEFORE (synced route == the pre-round-9 unsynced fallthrough) and
|
||||
/// AFTER (forwarded unsynced route) through one backend stack, printing the
|
||||
/// two rows and gating AFTER on zero probe requests. `prefixes` carries the
|
||||
/// (BEFORE, AFTER) hash namespaces keeping the arms' key spaces disjoint.
|
||||
async fn stack_ab(
|
||||
label: &str,
|
||||
backend: Arc<dyn BlobStorageBackend>,
|
||||
counters: &Counters,
|
||||
chunks: usize,
|
||||
chunk_kb: usize,
|
||||
concurrency: usize,
|
||||
prefixes: (&str, &str),
|
||||
) -> (f64, f64) {
|
||||
let (prefix_before, prefix_after) = prefixes;
|
||||
let before = drive(
|
||||
backend.clone(),
|
||||
chunks,
|
||||
chunk_kb,
|
||||
concurrency,
|
||||
false,
|
||||
prefix_before,
|
||||
)
|
||||
.await;
|
||||
let before_heads = counters.heads.swap(0, Ordering::Relaxed);
|
||||
let before_puts = counters.puts.swap(0, Ordering::Relaxed);
|
||||
println!(
|
||||
"{:<34} {:>10.0} {:>8} {:>8} {:>8}",
|
||||
format!("{label} BEFORE (synced route)"),
|
||||
before,
|
||||
before_heads,
|
||||
before_puts,
|
||||
"1.0x"
|
||||
);
|
||||
|
||||
let after = drive(
|
||||
backend.clone(),
|
||||
chunks,
|
||||
chunk_kb,
|
||||
concurrency,
|
||||
true,
|
||||
prefix_after,
|
||||
)
|
||||
.await;
|
||||
let after_heads = counters.heads.swap(0, Ordering::Relaxed);
|
||||
let after_puts = counters.puts.swap(0, Ordering::Relaxed);
|
||||
println!(
|
||||
"{:<34} {:>10.0} {:>8} {:>8} {:>8}",
|
||||
format!("{label} AFTER (unsynced)"),
|
||||
after,
|
||||
after_heads,
|
||||
after_puts,
|
||||
format!("{:.1}x", before / after)
|
||||
);
|
||||
|
||||
if before_heads != chunks as u64 {
|
||||
eprintln!(
|
||||
"GATE FAIL [{label}]: BEFORE issued {before_heads} HEADs (expected {chunks} — the probing route must still probe)"
|
||||
);
|
||||
std::process::exit(1);
|
||||
}
|
||||
if after_heads != 0 || after_puts != chunks as u64 {
|
||||
eprintln!(
|
||||
"GATE FAIL [{label}]: AFTER issued {after_heads} HEADs / {after_puts} PUTs (expected 0 / {chunks})"
|
||||
);
|
||||
std::process::exit(1);
|
||||
}
|
||||
if after >= before {
|
||||
eprintln!(
|
||||
"GATE FAIL [{label}]: AFTER ({after:.0} ms) not faster than BEFORE ({before:.0} ms) — rollback"
|
||||
);
|
||||
std::process::exit(1);
|
||||
}
|
||||
(before, after)
|
||||
}
|
||||
|
||||
#[tokio::main(flavor = "multi_thread")]
|
||||
async fn main() {
|
||||
let chunks: usize = env_or("BENCH_CHUNKS", 500);
|
||||
@@ -133,7 +244,15 @@ async fn main() {
|
||||
);
|
||||
|
||||
// BEFORE: the trait-default route (put_blob_from_bytes = HEAD + PUT).
|
||||
let before = drive(backend.clone(), chunks, chunk_kb, concurrency, false).await;
|
||||
let before = drive(
|
||||
backend.clone() as Arc<dyn BlobStorageBackend>,
|
||||
chunks,
|
||||
chunk_kb,
|
||||
concurrency,
|
||||
false,
|
||||
"a0a0",
|
||||
)
|
||||
.await;
|
||||
let before_heads = counters.heads.swap(0, Ordering::Relaxed);
|
||||
let before_puts = counters.puts.swap(0, Ordering::Relaxed);
|
||||
println!(
|
||||
@@ -142,7 +261,15 @@ async fn main() {
|
||||
);
|
||||
|
||||
// AFTER: the unsynced override (PUT only).
|
||||
let after = drive(backend.clone(), chunks, chunk_kb, concurrency, true).await;
|
||||
let after = drive(
|
||||
backend.clone() as Arc<dyn BlobStorageBackend>,
|
||||
chunks,
|
||||
chunk_kb,
|
||||
concurrency,
|
||||
true,
|
||||
"a0a1",
|
||||
)
|
||||
.await;
|
||||
let after_heads = counters.heads.swap(0, Ordering::Relaxed);
|
||||
let after_puts = counters.puts.swap(0, Ordering::Relaxed);
|
||||
println!(
|
||||
@@ -168,6 +295,91 @@ async fn main() {
|
||||
"\n# [2] removed Azure per-chunk copy: to_vec() of {mb} MiB = {copy_ms:.2} ms + {mb} MiB transient alloc per chunk"
|
||||
);
|
||||
|
||||
// ── Section 3: the same A/B through the decorator stacks ────────────
|
||||
println!(
|
||||
"\n# [3] decorated stacks — pre-round-9 the unsynced call fell through to the synced (probing) route"
|
||||
);
|
||||
println!(
|
||||
"{:<34} {:>10} {:>8} {:>8} {:>8}",
|
||||
"variant", "wall ms", "HEADs", "PUTs", "vs OLD"
|
||||
);
|
||||
|
||||
// Retry(S3)
|
||||
let retry_stack: Arc<dyn BlobStorageBackend> = Arc::new(RetryBlobBackend::new(
|
||||
backend.clone() as Arc<dyn BlobStorageBackend>,
|
||||
RetryPolicy::default(),
|
||||
));
|
||||
stack_ab(
|
||||
"retry(s3)",
|
||||
retry_stack,
|
||||
&counters,
|
||||
chunks,
|
||||
chunk_kb,
|
||||
concurrency,
|
||||
("b0b0", "b0b1"),
|
||||
)
|
||||
.await;
|
||||
|
||||
// Cache(S3) — count cache write-through population on both routes.
|
||||
let cache_dir_a = tempfile::tempdir().expect("tempdir");
|
||||
let cached_stack: Arc<dyn BlobStorageBackend> = Arc::new(CachedBlobBackend::new(
|
||||
backend.clone() as Arc<dyn BlobStorageBackend>,
|
||||
&BlobCacheConfig {
|
||||
cache_dir: cache_dir_a.path().to_path_buf(),
|
||||
max_cache_bytes: u64::MAX,
|
||||
},
|
||||
));
|
||||
stack_ab(
|
||||
"cache(s3)",
|
||||
cached_stack,
|
||||
&counters,
|
||||
chunks,
|
||||
chunk_kb,
|
||||
concurrency,
|
||||
("c0c0", "c0c1"),
|
||||
)
|
||||
.await;
|
||||
// Write-through equivalence gate: BOTH routes populated the local cache
|
||||
// (the round-9 override keeps post-upload read locality intact).
|
||||
let cached_files = count_files(cache_dir_a.path());
|
||||
if cached_files != 2 * chunks {
|
||||
eprintln!(
|
||||
"GATE FAIL [cache(s3)]: cache holds {cached_files} blobs (expected {} — write-through must populate on BOTH routes)",
|
||||
2 * chunks
|
||||
);
|
||||
std::process::exit(1);
|
||||
}
|
||||
|
||||
// Full production composition: Cache(Encrypted(Retry(S3))).
|
||||
let cache_dir_b = tempfile::tempdir().expect("tempdir");
|
||||
let full_stack: Arc<dyn BlobStorageBackend> = Arc::new(CachedBlobBackend::new(
|
||||
Arc::new(EncryptedBlobBackend::new(
|
||||
Arc::new(RetryBlobBackend::new(
|
||||
backend.clone() as Arc<dyn BlobStorageBackend>,
|
||||
RetryPolicy::default(),
|
||||
)),
|
||||
&[0x42u8; 32],
|
||||
)),
|
||||
&BlobCacheConfig {
|
||||
cache_dir: cache_dir_b.path().to_path_buf(),
|
||||
max_cache_bytes: u64::MAX,
|
||||
},
|
||||
));
|
||||
let (full_before, full_after) = stack_ab(
|
||||
"cache(enc(retry(s3)))",
|
||||
full_stack,
|
||||
&counters,
|
||||
chunks,
|
||||
chunk_kb,
|
||||
concurrency,
|
||||
("d0d0", "d0d1"),
|
||||
)
|
||||
.await;
|
||||
println!(
|
||||
"# full stack: a {chunks}-chunk upload sheds {} probe round-trips ({:.0} -> {:.0} ms at {rtt_ms} ms RTT)",
|
||||
chunks, full_before, full_after
|
||||
);
|
||||
|
||||
// ── Gates ───────────────────────────────────────────────────────────
|
||||
if after_heads != 0 || after_puts != chunks as u64 {
|
||||
eprintln!(
|
||||
|
||||
@@ -133,15 +133,15 @@ fn synth_entry(idx: u64) -> Arc<SearchResultsDto> {
|
||||
name,
|
||||
path,
|
||||
size: 831_942,
|
||||
mime_type: MIMES[row % MIMES.len()].to_string(),
|
||||
mime_type: MIMES[row % MIMES.len()].into(),
|
||||
folder_id: Some(pseudo_uuid(&mut rng)),
|
||||
created_at: 1_752_700_000,
|
||||
modified_at: 1_752_800_000,
|
||||
relevance_score: 50,
|
||||
size_formatted: "812.4 KB".to_string(),
|
||||
icon_class: "fas fa-file-pdf".to_string(),
|
||||
icon_special_class: "pdf-icon".to_string(),
|
||||
category: "document".to_string(),
|
||||
icon_class: "fas fa-file-pdf".into(),
|
||||
icon_special_class: "pdf-icon".into(),
|
||||
category: "document".into(),
|
||||
blob_hash: pseudo_hex(&mut rng, 64),
|
||||
snippet: content_hit.then(|| SNIPPET.to_string()),
|
||||
match_source: Some(match_source.to_string()),
|
||||
|
||||
@@ -0,0 +1,566 @@
|
||||
//! Search-result enrichment benchmark — borrow+clone+reclassify vs consume.
|
||||
//!
|
||||
//! `SearchService::enrich_file` took `&FileDto`, cloned every owned `String`
|
||||
//! out of it (id/name/path/folder_id/content_hash), allocated fresh `String`s
|
||||
//! for `mime_type` + the three display fields, and RE-RAN the three display
|
||||
//! classifiers (`icon_class_for` / `icon_special_class_for` / `category_for`)
|
||||
//! whose results the `FileDto` already carried interned (`Arc<str>`, computed
|
||||
//! once in `FileDto::from`). The recursive search branch runs this map over
|
||||
//! the ENTIRE pre-pagination match set, so a subtree query matching thousands
|
||||
//! of files paid ~11 allocs + 3 classifier passes per row. `enrich_folder`
|
||||
//! cloned its 4 strings the same way, and the NC REPORT conversion
|
||||
//! (`file_dto_from_search`) re-ran all three classifiers a SECOND time per
|
||||
//! emitted row.
|
||||
//!
|
||||
//! Round 9 changes `SearchFileResultDto.{mime_type,icon_class,
|
||||
//! icon_special_class,category}` to `Arc<str>`, makes both enrichers consume
|
||||
//! their DTO (strings move, interned fields transfer as refcount bumps), and
|
||||
//! has the NC conversion reuse the carried values.
|
||||
//!
|
||||
//! `mod before` holds the pre-round-9 logic verbatim (old struct shape
|
||||
//! included); the equivalence gate asserts field-by-field identical output
|
||||
//! for every row, and the NC-conversion gate asserts the reused display
|
||||
//! fields byte-equal a fresh classifier run.
|
||||
//!
|
||||
//! Sections:
|
||||
//! 1. enrich_file — ns/row + allocs/row, BEFORE vs AFTER
|
||||
//! 2. enrich_folder — ns/row + allocs/row, BEFORE vs AFTER
|
||||
//! 3. NC REPORT search→FileDto conversion — allocs/row, BEFORE vs AFTER
|
||||
//!
|
||||
//! Run (no Postgres needed):
|
||||
//! cargo run --release --features bench --example bench_search_enrich
|
||||
//! Tunables (env): BENCH_ROWS (10000), BENCH_PASSES (50)
|
||||
|
||||
use std::alloc::{GlobalAlloc, Layout, System};
|
||||
use std::env;
|
||||
use std::hint::black_box;
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
use std::time::Instant;
|
||||
|
||||
use oxicloud::application::dtos::file_dto::FileDto;
|
||||
use oxicloud::application::dtos::folder_dto::FolderDto;
|
||||
use oxicloud::application::services::search_service::SearchService;
|
||||
|
||||
// ─── Counting allocator ─────────────────────────────────────────────────────
|
||||
|
||||
static ALLOC_CALLS: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
struct CountingAlloc;
|
||||
|
||||
unsafe impl GlobalAlloc for CountingAlloc {
|
||||
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.alloc(layout) }
|
||||
}
|
||||
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
|
||||
unsafe { System.dealloc(ptr, layout) }
|
||||
}
|
||||
unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.realloc(ptr, layout, new_size) }
|
||||
}
|
||||
unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.alloc_zeroed(layout) }
|
||||
}
|
||||
}
|
||||
|
||||
#[global_allocator]
|
||||
static GLOBAL: CountingAlloc = CountingAlloc;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
// ─── BEFORE: verbatim pre-round-9 logic ─────────────────────────────────────
|
||||
|
||||
#[allow(clippy::all)]
|
||||
mod before {
|
||||
use oxicloud::application::dtos::display_helpers::{
|
||||
category_for, format_file_size, icon_class_for, icon_special_class_for,
|
||||
};
|
||||
use oxicloud::application::dtos::file_dto::FileDto;
|
||||
use oxicloud::application::dtos::folder_dto::FolderDto;
|
||||
use oxicloud::domain::entities::file::File;
|
||||
|
||||
/// Old `SearchFileResultDto` shape — all-String display fields.
|
||||
pub struct OldSearchFileResultDto {
|
||||
pub id: String,
|
||||
pub name: String,
|
||||
pub path: String,
|
||||
pub size: u64,
|
||||
pub mime_type: String,
|
||||
pub folder_id: Option<String>,
|
||||
pub created_at: u64,
|
||||
pub modified_at: u64,
|
||||
pub relevance_score: u32,
|
||||
pub size_formatted: String,
|
||||
pub icon_class: String,
|
||||
pub icon_special_class: String,
|
||||
pub category: String,
|
||||
pub blob_hash: String,
|
||||
pub snippet: Option<String>,
|
||||
pub match_source: Option<String>,
|
||||
}
|
||||
|
||||
pub struct OldSearchFolderResultDto {
|
||||
pub id: String,
|
||||
pub name: String,
|
||||
pub path: String,
|
||||
pub parent_id: Option<String>,
|
||||
pub drive_id: uuid::Uuid,
|
||||
pub created_at: u64,
|
||||
pub modified_at: u64,
|
||||
pub is_root: bool,
|
||||
pub relevance_score: u32,
|
||||
}
|
||||
|
||||
// Verbatim copies of the old private helpers.
|
||||
fn get_icon_class(name: &str, mime: &str) -> String {
|
||||
icon_class_for(name, mime).to_string()
|
||||
}
|
||||
fn get_icon_special_class(name: &str, mime: &str) -> String {
|
||||
icon_special_class_for(name, mime).to_string()
|
||||
}
|
||||
fn get_category(name: &str, mime: &str) -> String {
|
||||
category_for(name, mime).to_string()
|
||||
}
|
||||
|
||||
/// Verbatim copy of the service's private `format_bytes` (unchanged by
|
||||
/// round 9; the equivalence gate asserts it still matches production).
|
||||
pub fn format_bytes(bytes: u64) -> String {
|
||||
const UNITS: &[&str] = &["B", "KB", "MB", "GB", "TB"];
|
||||
if bytes == 0 {
|
||||
return "0 B".to_string();
|
||||
}
|
||||
let exp = (bytes as f64).log(1024.0).floor() as usize;
|
||||
let exp = exp.min(UNITS.len() - 1);
|
||||
let value = bytes as f64 / 1024_f64.powi(exp as i32);
|
||||
if exp == 0 {
|
||||
format!("{} B", bytes)
|
||||
} else {
|
||||
format!("{:.1} {}", value, UNITS[exp])
|
||||
}
|
||||
}
|
||||
|
||||
/// Verbatim copy of the service's private `compute_relevance` (unchanged
|
||||
/// by round 9; the equivalence gate asserts it still matches production).
|
||||
pub fn compute_relevance(name: &str, query_lower: &str) -> u32 {
|
||||
let name_lower = name.to_lowercase();
|
||||
|
||||
if name_lower == query_lower {
|
||||
100
|
||||
} else if name_lower.starts_with(query_lower) {
|
||||
80
|
||||
} else if name_lower.contains(query_lower) {
|
||||
// Bonus for shorter names (more specific match)
|
||||
let ratio = query_lower.len() as f64 / name_lower.len() as f64;
|
||||
50 + (ratio * 20.0) as u32
|
||||
} else {
|
||||
0
|
||||
}
|
||||
}
|
||||
|
||||
/// Verbatim old `enrich_file` (borrowing, cloning, re-classifying).
|
||||
pub fn enrich_file(file: &FileDto, query_lower: &str) -> OldSearchFileResultDto {
|
||||
let relevance = if query_lower.is_empty() {
|
||||
50
|
||||
} else {
|
||||
compute_relevance(&file.name, query_lower)
|
||||
};
|
||||
|
||||
OldSearchFileResultDto {
|
||||
id: file.id.clone(),
|
||||
name: file.name.clone(),
|
||||
path: file.path.clone(),
|
||||
size: file.size,
|
||||
mime_type: file.mime_type.to_string(),
|
||||
folder_id: file.folder_id.clone(),
|
||||
created_at: file.created_at,
|
||||
modified_at: file.modified_at,
|
||||
relevance_score: relevance,
|
||||
size_formatted: format_bytes(file.size),
|
||||
icon_class: get_icon_class(&file.name, &file.mime_type),
|
||||
icon_special_class: get_icon_special_class(&file.name, &file.mime_type),
|
||||
category: get_category(&file.name, &file.mime_type),
|
||||
blob_hash: file.content_hash.clone(),
|
||||
snippet: None,
|
||||
match_source: (!query_lower.is_empty() && relevance > 0).then(|| "name".to_string()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Verbatim old `enrich_folder`.
|
||||
pub fn enrich_folder(folder: &FolderDto, query_lower: &str) -> OldSearchFolderResultDto {
|
||||
let relevance = if query_lower.is_empty() {
|
||||
50
|
||||
} else {
|
||||
compute_relevance(&folder.name, query_lower)
|
||||
};
|
||||
|
||||
OldSearchFolderResultDto {
|
||||
id: folder.id.clone(),
|
||||
name: folder.name.clone(),
|
||||
path: folder.path.clone(),
|
||||
parent_id: folder.parent_id.clone(),
|
||||
drive_id: folder.drive_id,
|
||||
created_at: folder.created_at,
|
||||
modified_at: folder.modified_at,
|
||||
is_root: folder.is_root,
|
||||
relevance_score: relevance,
|
||||
}
|
||||
}
|
||||
|
||||
/// Verbatim old NC REPORT `file_dto_from_search` body (String-field
|
||||
/// input shape) — re-runs all three classifiers per converted row.
|
||||
pub fn file_dto_from_search(fr: &OldSearchFileResultDto) -> FileDto {
|
||||
let etag = if fr.blob_hash.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
File::compute_etag(&fr.blob_hash, fr.modified_at)
|
||||
};
|
||||
FileDto {
|
||||
id: fr.id.clone(),
|
||||
name: fr.name.clone(),
|
||||
path: fr.path.clone(),
|
||||
size: fr.size,
|
||||
mime_type: fr.mime_type.clone().into(),
|
||||
folder_id: fr.folder_id.clone(),
|
||||
created_at: fr.created_at,
|
||||
modified_at: fr.modified_at,
|
||||
icon_class: icon_class_for(&fr.name, &fr.mime_type).to_string().into(),
|
||||
icon_special_class: icon_special_class_for(&fr.name, &fr.mime_type)
|
||||
.to_string()
|
||||
.into(),
|
||||
category: category_for(&fr.name, &fr.mime_type).to_string().into(),
|
||||
size_formatted: format_file_size(fr.size),
|
||||
sort_date: None,
|
||||
content_hash: fr.blob_hash.clone(),
|
||||
etag,
|
||||
created_by: None,
|
||||
updated_by: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Fixture ────────────────────────────────────────────────────────────────
|
||||
|
||||
const NAMES: [(&str, &str); 5] = [
|
||||
("report-{i}.pdf", "application/pdf"),
|
||||
("photo-{i}.jpg", "image/jpeg"),
|
||||
("notes-{i}.txt", "text/plain"),
|
||||
("track-{i}.mp3", "audio/mpeg"),
|
||||
("data-{i}.bin", "application/octet-stream"),
|
||||
];
|
||||
|
||||
fn file_dtos(n: usize) -> Vec<FileDto> {
|
||||
(0..n)
|
||||
.map(|i| {
|
||||
let (name_t, mime) = NAMES[i % NAMES.len()];
|
||||
let name = name_t.replace("{i}", &format!("{i:05}"));
|
||||
let file = oxicloud::domain::entities::file::File::from_materialized_row(
|
||||
uuid::Uuid::new_v4().to_string(),
|
||||
name,
|
||||
Some("Documents/Work"),
|
||||
4096 + i as u64,
|
||||
mime.to_string(),
|
||||
Some(uuid::Uuid::new_v4().to_string()),
|
||||
1_700_000_000,
|
||||
1_700_000_100,
|
||||
"a".repeat(64),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.expect("fixture file");
|
||||
FileDto::from(file)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn folder_dtos(n: usize) -> Vec<FolderDto> {
|
||||
(0..n)
|
||||
.map(|i| FolderDto {
|
||||
id: uuid::Uuid::new_v4().to_string(),
|
||||
name: format!("Folder {i:05}"),
|
||||
path: format!("Documents/Folder-{i:05}"),
|
||||
parent_id: Some(uuid::Uuid::new_v4().to_string()),
|
||||
drive_id: uuid::Uuid::new_v4(),
|
||||
created_at: 1_700_000_000,
|
||||
modified_at: 1_700_000_100,
|
||||
is_root: false,
|
||||
etag: format!("{i:032x}"),
|
||||
icon_class: Arc::from("fas fa-folder"),
|
||||
icon_special_class: Arc::from("folder-icon"),
|
||||
category: Arc::from("Folder"),
|
||||
created_by: None,
|
||||
updated_by: None,
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn p50(mut v: Vec<f64>) -> f64 {
|
||||
v.sort_by(|a, b| a.partial_cmp(b).unwrap());
|
||||
v[v.len() / 2]
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let n: usize = env_or("BENCH_ROWS", 10_000);
|
||||
let passes: usize = env_or("BENCH_PASSES", 50);
|
||||
let query_lower = "report";
|
||||
|
||||
// ── Equivalence gate: field-by-field identical enrichment ───────────────
|
||||
{
|
||||
let dtos = file_dtos(500);
|
||||
for dto in &dtos {
|
||||
let old = before::enrich_file(dto, query_lower);
|
||||
let new = SearchService::enrich_file_for_bench(dto.clone(), query_lower);
|
||||
let same = old.id == new.id
|
||||
&& old.name == new.name
|
||||
&& old.path == new.path
|
||||
&& old.size == new.size
|
||||
&& old.mime_type == *new.mime_type
|
||||
&& old.folder_id == new.folder_id
|
||||
&& old.created_at == new.created_at
|
||||
&& old.modified_at == new.modified_at
|
||||
&& old.relevance_score == new.relevance_score
|
||||
&& old.size_formatted == new.size_formatted
|
||||
&& old.icon_class == *new.icon_class
|
||||
&& old.icon_special_class == *new.icon_special_class
|
||||
&& old.category == *new.category
|
||||
&& old.blob_hash == new.blob_hash
|
||||
&& old.snippet == new.snippet
|
||||
&& old.match_source == new.match_source;
|
||||
if !same {
|
||||
eprintln!("EQUIVALENCE GATE FAILED (file): {} differs", old.name);
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
let folders = folder_dtos(500);
|
||||
for dto in &folders {
|
||||
let old = before::enrich_folder(dto, query_lower);
|
||||
let new = SearchService::enrich_folder_for_bench(dto.clone(), query_lower);
|
||||
let same = old.id == new.id
|
||||
&& old.name == new.name
|
||||
&& old.path == new.path
|
||||
&& old.parent_id == new.parent_id
|
||||
&& old.drive_id == new.drive_id
|
||||
&& old.created_at == new.created_at
|
||||
&& old.modified_at == new.modified_at
|
||||
&& old.is_root == new.is_root
|
||||
&& old.relevance_score == new.relevance_score;
|
||||
if !same {
|
||||
eprintln!("EQUIVALENCE GATE FAILED (folder): {} differs", old.name);
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
println!("# equivalence gate: 500 files + 500 folders field-identical — OK");
|
||||
}
|
||||
|
||||
// ── NC REPORT conversion gate: carried display fields == fresh run ──────
|
||||
{
|
||||
let dtos = file_dtos(500);
|
||||
for dto in dtos {
|
||||
let old_row = before::enrich_file(&dto, "");
|
||||
let new_row = SearchService::enrich_file_for_bench(dto, "");
|
||||
let old_conv = before::file_dto_from_search(&old_row);
|
||||
let new_conv =
|
||||
oxicloud::interfaces::nextcloud::report_handler::file_dto_from_search_for_bench(
|
||||
&new_row,
|
||||
);
|
||||
let same = old_conv.id == new_conv.id
|
||||
&& old_conv.name == new_conv.name
|
||||
&& old_conv.mime_type == new_conv.mime_type
|
||||
&& old_conv.icon_class == new_conv.icon_class
|
||||
&& old_conv.icon_special_class == new_conv.icon_special_class
|
||||
&& old_conv.category == new_conv.category
|
||||
&& old_conv.size_formatted == new_conv.size_formatted
|
||||
&& old_conv.etag == new_conv.etag
|
||||
&& old_conv.content_hash == new_conv.content_hash;
|
||||
if !same {
|
||||
eprintln!("NC CONVERSION GATE FAILED: {} differs", old_conv.name);
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
println!("# NC REPORT conversion gate: 500 rows field-identical — OK");
|
||||
}
|
||||
|
||||
// ── Section 1: enrich_file wall + allocs ────────────────────────────────
|
||||
let mut before_wall = Vec::with_capacity(passes);
|
||||
let mut after_wall = Vec::with_capacity(passes);
|
||||
let mut before_allocs = 0u64;
|
||||
let mut after_allocs = 0u64;
|
||||
|
||||
for pass in 0..passes {
|
||||
// BEFORE consumes borrowed rows: reuse one input set per pass, built
|
||||
// outside the measured window (both arms see identical inputs).
|
||||
let input = file_dtos(n);
|
||||
|
||||
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
let out: Vec<_> = input
|
||||
.iter()
|
||||
.map(|f| before::enrich_file(f, query_lower))
|
||||
.collect();
|
||||
before_wall.push(t.elapsed().as_secs_f64() * 1e9 / n as f64);
|
||||
if pass == 0 {
|
||||
before_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
|
||||
}
|
||||
black_box(&out);
|
||||
drop(out);
|
||||
|
||||
let a1 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
let out: Vec<_> = input
|
||||
.into_iter()
|
||||
.map(|f| SearchService::enrich_file_for_bench(f, query_lower))
|
||||
.collect();
|
||||
after_wall.push(t.elapsed().as_secs_f64() * 1e9 / n as f64);
|
||||
if pass == 0 {
|
||||
after_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1;
|
||||
}
|
||||
black_box(&out);
|
||||
}
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# [1] enrich_file — borrow+clone+reclassify vs consume");
|
||||
println!("# rows={n} passes={passes} (p50 of per-pass ns/row; allocs from pass 0)");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<22} | {:>10} | {:>12} | {:>12} |",
|
||||
"arm", "ns/row", "allocs", "allocs/row"
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>10.1} | {:>12} | {:>12.3} |",
|
||||
"BEFORE (borrow+clone)",
|
||||
p50(before_wall.clone()),
|
||||
before_allocs,
|
||||
before_allocs as f64 / n as f64
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>10.1} | {:>12} | {:>12.3} |",
|
||||
"AFTER (consume)",
|
||||
p50(after_wall.clone()),
|
||||
after_allocs,
|
||||
after_allocs as f64 / n as f64
|
||||
);
|
||||
let s1_ok = after_allocs < before_allocs;
|
||||
|
||||
// ── Section 2: enrich_folder ────────────────────────────────────────────
|
||||
let mut fb_wall = Vec::with_capacity(passes);
|
||||
let mut fa_wall = Vec::with_capacity(passes);
|
||||
let mut fb_allocs = 0u64;
|
||||
let mut fa_allocs = 0u64;
|
||||
for pass in 0..passes {
|
||||
let input = folder_dtos(n);
|
||||
|
||||
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
let out: Vec<_> = input
|
||||
.iter()
|
||||
.map(|f| before::enrich_folder(f, query_lower))
|
||||
.collect();
|
||||
fb_wall.push(t.elapsed().as_secs_f64() * 1e9 / n as f64);
|
||||
if pass == 0 {
|
||||
fb_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
|
||||
}
|
||||
black_box(&out);
|
||||
drop(out);
|
||||
|
||||
let a1 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
let out: Vec<_> = input
|
||||
.into_iter()
|
||||
.map(|f| SearchService::enrich_folder_for_bench(f, query_lower))
|
||||
.collect();
|
||||
fa_wall.push(t.elapsed().as_secs_f64() * 1e9 / n as f64);
|
||||
if pass == 0 {
|
||||
fa_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1;
|
||||
}
|
||||
black_box(&out);
|
||||
}
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# [2] enrich_folder — borrow+clone vs consume");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<22} | {:>10} | {:>12} | {:>12} |",
|
||||
"arm", "ns/row", "allocs", "allocs/row"
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>10.1} | {:>12} | {:>12.3} |",
|
||||
"BEFORE (borrow+clone)",
|
||||
p50(fb_wall.clone()),
|
||||
fb_allocs,
|
||||
fb_allocs as f64 / n as f64
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>10.1} | {:>12} | {:>12.3} |",
|
||||
"AFTER (consume)",
|
||||
p50(fa_wall.clone()),
|
||||
fa_allocs,
|
||||
fa_allocs as f64 / n as f64
|
||||
);
|
||||
let s2_ok = fa_allocs < fb_allocs;
|
||||
|
||||
// ── Section 3: NC REPORT conversion ─────────────────────────────────────
|
||||
let conv_n = n.min(5_000);
|
||||
let old_rows: Vec<_> = file_dtos(conv_n)
|
||||
.iter()
|
||||
.map(|f| before::enrich_file(f, ""))
|
||||
.collect();
|
||||
let new_rows: Vec<_> = file_dtos(conv_n)
|
||||
.into_iter()
|
||||
.map(|f| SearchService::enrich_file_for_bench(f, ""))
|
||||
.collect();
|
||||
|
||||
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
let out: Vec<_> = old_rows.iter().map(before::file_dto_from_search).collect();
|
||||
let conv_before_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let conv_before_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
|
||||
black_box(&out);
|
||||
drop(out);
|
||||
|
||||
let a1 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
let out: Vec<_> = new_rows
|
||||
.iter()
|
||||
.map(oxicloud::interfaces::nextcloud::report_handler::file_dto_from_search_for_bench)
|
||||
.collect();
|
||||
let conv_after_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let conv_after_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1;
|
||||
black_box(&out);
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# [3] NC REPORT search→FileDto conversion — reclassify vs carry");
|
||||
println!("# rows={conv_n}");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<22} | {:>10} | {:>12} | {:>12} |",
|
||||
"arm", "wall ms", "allocs", "allocs/row"
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>10.3} | {:>12} | {:>12.3} |",
|
||||
"BEFORE (reclassify)",
|
||||
conv_before_ms,
|
||||
conv_before_allocs,
|
||||
conv_before_allocs as f64 / conv_n as f64
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>10.3} | {:>12} | {:>12.3} |",
|
||||
"AFTER (carry Arc)",
|
||||
conv_after_ms,
|
||||
conv_after_allocs,
|
||||
conv_after_allocs as f64 / conv_n as f64
|
||||
);
|
||||
let s3_ok = conv_after_allocs < conv_before_allocs;
|
||||
|
||||
if !(s1_ok && s2_ok && s3_ok) {
|
||||
eprintln!("\nGATE FAIL: allocs not reduced (s1={s1_ok} s2={s2_ok} s3={s3_ok}) — rollback");
|
||||
std::process::exit(1);
|
||||
}
|
||||
println!("\nGATE PASS: allocs reduced in all three sections; outputs field-identical.");
|
||||
}
|
||||
@@ -0,0 +1,399 @@
|
||||
//! Round-9 storage micro-pack benchmark — four independent A/Bs, no Postgres.
|
||||
//!
|
||||
//! [1] Local chunk write — the old `try_exists` (stat) + `File::create` pair
|
||||
//! vs the new single atomic `create_new` open, at chunk-write level via
|
||||
//! the bench wrapper over the production writer. Fresh-write AND
|
||||
//! already-exists (dedup re-upload skip) arms.
|
||||
//! [2] CDC read prep — the old per-read deep clone of the cached manifest's
|
||||
//! `Vec<String>` chunk-hash list vs the new index-over-`Arc` iteration
|
||||
//! (structural replica of `DedupService::stream_chunks` before/after;
|
||||
//! the production change is exactly this data-flow).
|
||||
//! [3] Manifest cache miss herd — the old `get → SELECT → insert` shape vs
|
||||
//! the new fast-get + `try_get_with` single-flight, K concurrent cold
|
||||
//! readers on one key over a real moka cache with a counted loader
|
||||
//! (structural replica of `DedupService::manifest_cached`, sqlx swapped
|
||||
//! for a latency-injected counted loader).
|
||||
//! [4] Chunk `Content-MD5` verification hex — 16× `format!("{b:02x}")` +
|
||||
//! collect vs `common::fmt::hex_lower` (1 sized alloc).
|
||||
//!
|
||||
//! Gates: [1] AFTER wall < BEFORE wall (fresh) + identical on-disk content +
|
||||
//! identical skip semantics; [2] AFTER allocs < BEFORE allocs + identical
|
||||
//! hash sequence; [3] AFTER loader runs == 1 (BEFORE > 1) + identical value;
|
||||
//! [4] identical hex + fewer allocs.
|
||||
//!
|
||||
//! Run:
|
||||
//! cargo run --release --features bench --example bench_storage_micro
|
||||
//! Tunables (env): BENCH_CHUNKS (20000), BENCH_CHUNK_KB (4), BENCH_HERD (64),
|
||||
//! BENCH_MANIFEST_CHUNKS (4096)
|
||||
|
||||
use std::alloc::{GlobalAlloc, Layout, System};
|
||||
use std::env;
|
||||
use std::hint::black_box;
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use bytes::Bytes;
|
||||
use oxicloud::infrastructure::services::local_blob_backend::write_blob_bytes_for_bench;
|
||||
|
||||
// ─── Counting allocator ─────────────────────────────────────────────────────
|
||||
|
||||
static ALLOC_CALLS: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
struct CountingAlloc;
|
||||
|
||||
unsafe impl GlobalAlloc for CountingAlloc {
|
||||
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.alloc(layout) }
|
||||
}
|
||||
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
|
||||
unsafe { System.dealloc(ptr, layout) }
|
||||
}
|
||||
unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.realloc(ptr, layout, new_size) }
|
||||
}
|
||||
unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.alloc_zeroed(layout) }
|
||||
}
|
||||
}
|
||||
|
||||
#[global_allocator]
|
||||
static GLOBAL: CountingAlloc = CountingAlloc;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
// ─── [1] BEFORE replica: stat-then-create chunk writer (verbatim) ───────────
|
||||
|
||||
async fn write_blob_bytes_before(
|
||||
blob_path: &std::path::Path,
|
||||
data: &Bytes,
|
||||
) -> std::io::Result<Option<tokio::fs::File>> {
|
||||
use tokio::io::AsyncWriteExt;
|
||||
if tokio::fs::try_exists(blob_path).await.unwrap_or(false) {
|
||||
return Ok(None);
|
||||
}
|
||||
let mut file = tokio::fs::File::create(blob_path).await?;
|
||||
file.write_all(data).await?;
|
||||
Ok(Some(file))
|
||||
}
|
||||
|
||||
async fn section_1(chunks: usize, chunk_kb: usize) {
|
||||
let payload = Bytes::from(vec![0x5au8; chunk_kb * 1024]);
|
||||
let dir_before = tempfile::tempdir().expect("tempdir");
|
||||
let dir_after = tempfile::tempdir().expect("tempdir");
|
||||
|
||||
// Fresh writes.
|
||||
let t = Instant::now();
|
||||
for i in 0..chunks {
|
||||
let p = dir_before.path().join(format!("{i:08x}.blob"));
|
||||
write_blob_bytes_before(&p, &payload)
|
||||
.await
|
||||
.expect("before write");
|
||||
}
|
||||
let before_fresh = t.elapsed().as_secs_f64() * 1e3;
|
||||
|
||||
let t = Instant::now();
|
||||
for i in 0..chunks {
|
||||
let p = dir_after.path().join(format!("{i:08x}.blob"));
|
||||
write_blob_bytes_for_bench(&p, &payload)
|
||||
.await
|
||||
.expect("after write");
|
||||
}
|
||||
let after_fresh = t.elapsed().as_secs_f64() * 1e3;
|
||||
|
||||
// Equivalence: same file count, same bytes for a sample.
|
||||
let sample = dir_after.path().join(format!("{:08x}.blob", chunks / 2));
|
||||
let got = tokio::fs::read(&sample).await.expect("sample read");
|
||||
assert_eq!(got.len(), payload.len(), "content length mismatch");
|
||||
assert_eq!(&got[..64], &payload[..64], "content mismatch");
|
||||
|
||||
// Already-exists skip (dedup re-upload): both must return None-equivalent.
|
||||
let t = Instant::now();
|
||||
for i in 0..chunks {
|
||||
let p = dir_before.path().join(format!("{i:08x}.blob"));
|
||||
let r = write_blob_bytes_before(&p, &payload).await.expect("skip");
|
||||
assert!(r.is_none(), "BEFORE re-put must skip");
|
||||
}
|
||||
let before_skip = t.elapsed().as_secs_f64() * 1e3;
|
||||
|
||||
let t = Instant::now();
|
||||
for i in 0..chunks {
|
||||
let p = dir_after.path().join(format!("{i:08x}.blob"));
|
||||
let r = write_blob_bytes_for_bench(&p, &payload)
|
||||
.await
|
||||
.expect("skip");
|
||||
assert!(r.is_none(), "AFTER re-put must skip (AlreadyExists)");
|
||||
}
|
||||
let after_skip = t.elapsed().as_secs_f64() * 1e3;
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# [1] local chunk write — stat+create vs atomic create_new");
|
||||
println!("# chunks={chunks} x {chunk_kb} KiB");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<26} | {:>12} | {:>12} |",
|
||||
"arm", "fresh ms", "re-put ms"
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>12.1} | {:>12.1} |",
|
||||
"BEFORE (stat+create)", before_fresh, before_skip
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>12.1} | {:>12.1} |",
|
||||
"AFTER (create_new)", after_fresh, after_skip
|
||||
);
|
||||
println!(
|
||||
"\nfresh {:.2}x · re-put {:.2}x",
|
||||
before_fresh / after_fresh,
|
||||
before_skip / after_skip
|
||||
);
|
||||
if after_fresh >= before_fresh {
|
||||
eprintln!("GATE FAIL [1]: create_new not faster on fresh writes — rollback");
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// ─── [2] manifest read prep: Vec clone vs Arc-index ─────────────────────────
|
||||
|
||||
struct ManifestReplica {
|
||||
chunk_hashes: Vec<String>,
|
||||
}
|
||||
|
||||
fn section_2(manifest_chunks: usize) {
|
||||
let manifest = Arc::new(ManifestReplica {
|
||||
chunk_hashes: (0..manifest_chunks).map(|i| format!("{i:064x}")).collect(),
|
||||
});
|
||||
let reads = 200usize;
|
||||
|
||||
// BEFORE: each read clones the whole hash list out of the shared Arc
|
||||
// (the old `stream_chunks(m.chunk_hashes.clone())` call shape).
|
||||
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
let mut sum_before = 0usize;
|
||||
for _ in 0..reads {
|
||||
let hashes: Vec<String> = manifest.chunk_hashes.clone();
|
||||
for h in &hashes {
|
||||
sum_before += h.len();
|
||||
}
|
||||
black_box(&hashes);
|
||||
}
|
||||
let before_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let before_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
|
||||
|
||||
// AFTER: each read bumps the Arc and indexes (the new `stream_chunks(m)`).
|
||||
let a1 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
let mut sum_after = 0usize;
|
||||
for _ in 0..reads {
|
||||
let m = manifest.clone();
|
||||
for i in 0..m.chunk_hashes.len() {
|
||||
sum_after += m.chunk_hashes[i].len();
|
||||
}
|
||||
black_box(&m);
|
||||
}
|
||||
let after_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let after_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1;
|
||||
|
||||
assert_eq!(sum_before, sum_after, "hash sequence mismatch");
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# [2] CDC read prep — manifest Vec<String> clone vs Arc index");
|
||||
println!("# manifest={manifest_chunks} chunks, reads={reads}");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<26} | {:>10} | {:>12} | {:>12} |",
|
||||
"arm", "wall ms", "allocs", "allocs/read"
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.3} | {:>12} | {:>12.1} |",
|
||||
"BEFORE (clone Vec)",
|
||||
before_ms,
|
||||
before_allocs,
|
||||
before_allocs as f64 / reads as f64
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.3} | {:>12} | {:>12.1} |",
|
||||
"AFTER (Arc index)",
|
||||
after_ms,
|
||||
after_allocs,
|
||||
after_allocs as f64 / reads as f64
|
||||
);
|
||||
if after_allocs >= before_allocs {
|
||||
eprintln!("GATE FAIL [2]: Arc-index not fewer allocs — rollback");
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// ─── [3] manifest miss herd: get→insert vs try_get_with ─────────────────────
|
||||
|
||||
async fn section_3(herd: usize) {
|
||||
type Cache = moka::future::Cache<String, Arc<Vec<u64>>>;
|
||||
|
||||
let value = || Arc::new(vec![7u64; 1024]);
|
||||
let simulated_query = Duration::from_millis(2);
|
||||
|
||||
// BEFORE shape: check, query (2 ms), insert — every cold caller loads.
|
||||
let cache: Cache = moka::future::Cache::new(1000);
|
||||
let loads = Arc::new(AtomicU64::new(0));
|
||||
let mut set = tokio::task::JoinSet::new();
|
||||
let t = Instant::now();
|
||||
for _ in 0..herd {
|
||||
let cache = cache.clone();
|
||||
let loads = loads.clone();
|
||||
set.spawn(async move {
|
||||
if let Some(v) = cache.get("hot-file").await {
|
||||
return v;
|
||||
}
|
||||
loads.fetch_add(1, Ordering::Relaxed);
|
||||
tokio::time::sleep(simulated_query).await;
|
||||
let v = value();
|
||||
cache.insert("hot-file".to_string(), v.clone()).await;
|
||||
v
|
||||
});
|
||||
}
|
||||
let mut first: Option<Arc<Vec<u64>>> = None;
|
||||
while let Some(r) = set.join_next().await {
|
||||
let v = r.expect("join");
|
||||
if let Some(f) = &first {
|
||||
assert_eq!(f.len(), v.len());
|
||||
} else {
|
||||
first = Some(v);
|
||||
}
|
||||
}
|
||||
let before_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let before_loads = loads.load(Ordering::Relaxed);
|
||||
|
||||
// AFTER shape: fast get + try_get_with — the herd coalesces onto 1 load.
|
||||
let cache: Cache = moka::future::Cache::new(1000);
|
||||
let loads = Arc::new(AtomicU64::new(0));
|
||||
let mut set = tokio::task::JoinSet::new();
|
||||
let t = Instant::now();
|
||||
for _ in 0..herd {
|
||||
let cache = cache.clone();
|
||||
let loads = loads.clone();
|
||||
set.spawn(async move {
|
||||
if let Some(v) = cache.get("hot-file").await {
|
||||
return v;
|
||||
}
|
||||
cache
|
||||
.try_get_with("hot-file".to_string(), async move {
|
||||
loads.fetch_add(1, Ordering::Relaxed);
|
||||
tokio::time::sleep(simulated_query).await;
|
||||
Ok::<_, std::convert::Infallible>(value())
|
||||
})
|
||||
.await
|
||||
.expect("infallible")
|
||||
});
|
||||
}
|
||||
while let Some(r) = set.join_next().await {
|
||||
let v = r.expect("join");
|
||||
assert_eq!(v.len(), first.as_ref().unwrap().len());
|
||||
}
|
||||
let after_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let after_loads = loads.load(Ordering::Relaxed);
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# [3] manifest cold-miss herd — get→insert vs try_get_with");
|
||||
println!("# herd={herd} concurrent readers, 2 ms simulated manifest SELECT");
|
||||
println!("#################################################################\n");
|
||||
println!("| {:<26} | {:>10} | {:>12} |", "arm", "wall ms", "loads");
|
||||
println!(
|
||||
"| {:<26} | {:>10.1} | {:>12} |",
|
||||
"BEFORE (get→insert)", before_ms, before_loads
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.1} | {:>12} |",
|
||||
"AFTER (single-flight)", after_ms, after_loads
|
||||
);
|
||||
if after_loads != 1 {
|
||||
eprintln!("GATE FAIL [3]: single-flight ran {after_loads} loads (expected 1) — rollback");
|
||||
std::process::exit(1);
|
||||
}
|
||||
if before_loads <= 1 {
|
||||
eprintln!(
|
||||
"GATE WARN [3]: BEFORE herd only loaded {before_loads}x — herd too small to show the stampede"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// ─── [4] Content-MD5 hex ────────────────────────────────────────────────────
|
||||
|
||||
fn section_4() {
|
||||
let digests: Vec<[u8; 16]> = (0..1000u32)
|
||||
.map(|i| {
|
||||
let mut d = [0u8; 16];
|
||||
d[..4].copy_from_slice(&i.to_le_bytes());
|
||||
d
|
||||
})
|
||||
.collect();
|
||||
|
||||
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
let before: Vec<String> = digests
|
||||
.iter()
|
||||
.map(|d| d.iter().map(|b| format!("{b:02x}")).collect::<String>())
|
||||
.collect();
|
||||
let before_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let before_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
|
||||
|
||||
let a1 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
let after: Vec<String> = digests
|
||||
.iter()
|
||||
.map(|d| oxicloud::common::fmt::hex_lower(d))
|
||||
.collect();
|
||||
let after_ms = t.elapsed().as_secs_f64() * 1e3;
|
||||
let after_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1;
|
||||
|
||||
assert_eq!(before, after, "hex output mismatch");
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# [4] chunk Content-MD5 hex — per-byte format! vs hex_lower");
|
||||
println!("# digests=1000");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<26} | {:>10} | {:>12} | {:>14} |",
|
||||
"arm", "wall ms", "allocs", "allocs/digest"
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.3} | {:>12} | {:>14.2} |",
|
||||
"BEFORE (format!/byte)",
|
||||
before_ms,
|
||||
before_allocs,
|
||||
before_allocs as f64 / 1000.0
|
||||
);
|
||||
println!(
|
||||
"| {:<26} | {:>10.3} | {:>12} | {:>14.2} |",
|
||||
"AFTER (hex_lower)",
|
||||
after_ms,
|
||||
after_allocs,
|
||||
after_allocs as f64 / 1000.0
|
||||
);
|
||||
if after_allocs >= before_allocs {
|
||||
eprintln!("GATE FAIL [4]: hex_lower not fewer allocs — rollback");
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::main(flavor = "multi_thread")]
|
||||
async fn main() {
|
||||
let chunks: usize = env_or("BENCH_CHUNKS", 20_000);
|
||||
let chunk_kb: usize = env_or("BENCH_CHUNK_KB", 4);
|
||||
let herd: usize = env_or("BENCH_HERD", 64);
|
||||
let manifest_chunks: usize = env_or("BENCH_MANIFEST_CHUNKS", 4096);
|
||||
|
||||
section_1(chunks, chunk_kb).await;
|
||||
section_2(manifest_chunks);
|
||||
section_3(herd).await;
|
||||
section_4();
|
||||
|
||||
println!("\nGATE PASS: all four sections improved with identical outputs.");
|
||||
}
|
||||
@@ -13,12 +13,23 @@
|
||||
//! on any File/Folder grant write). The check still runs on every request —
|
||||
//! it is never skipped — but after the first query it resolves in-memory.
|
||||
//!
|
||||
//! Round 9 additionally decomposes the FILE decision: parent point-read
|
||||
//! (memoised) → the FOLDER cascade decision (one ltree query per folder,
|
||||
//! shared by every sibling) → direct-file-grant fallback. A shared album's
|
||||
//! COLD first view drops from one ltree UNION query per file to one ltree
|
||||
//! query per FOLDER plus cheap PK reads. The `ROUND8 cold` arm below runs
|
||||
//! the historical UNION verbatim per file for comparison.
|
||||
//!
|
||||
//! Safety gates (hard asserts, exit 1 on failure):
|
||||
//! 1. the folder-grant recipient is allowed; an outsider is denied;
|
||||
//! 2. REVOCATION — after a warm cache serves `allowed`, `clear_role` on the
|
||||
//! shared folder makes the very next check DENY (proves the grant-write
|
||||
//! invalidation flushes the cache; without it the stale `true` would
|
||||
//! still serve).
|
||||
//! still serve);
|
||||
//! 3. DIRECT-GRANT SIBLING (round 9) — a caller holding ONLY a direct
|
||||
//! grant on one file is allowed that file and denied its siblings,
|
||||
//! proving the folder-level decomposition neither shadows direct file
|
||||
//! grants nor leaks a file decision to siblings.
|
||||
//!
|
||||
//! Run (needs Postgres up; reads DATABASE_URL from .env):
|
||||
//! cargo run --release --features bench --example bench_thumbnail_cascade_cache
|
||||
@@ -29,7 +40,9 @@ use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use oxicloud::application::ports::authorization_ports::AuthorizationEngine;
|
||||
use oxicloud::domain::services::authorization::{Permission, Resource, Role, Subject};
|
||||
use oxicloud::domain::services::authorization::{
|
||||
Permission, Resource, Role, Subject, roles_implying,
|
||||
};
|
||||
use oxicloud::infrastructure::repositories::pg::{
|
||||
FileBlobReadRepository, FolderDbRepository, SubjectGroupPgRepository,
|
||||
};
|
||||
@@ -308,6 +321,46 @@ async fn main() {
|
||||
.expect("re-grant");
|
||||
}
|
||||
|
||||
// ── Safety gate 3 (round 9): direct-grant sibling isolation ──
|
||||
// The outsider gets a DIRECT grant on file[0] only (no folder/drive
|
||||
// grant): they must be allowed file[0] — the folder half of the
|
||||
// decomposition denies, the direct half matches — and denied file[1]
|
||||
// even immediately after the allowed check (no sibling leak through
|
||||
// the folder-level cache).
|
||||
{
|
||||
let engine = fresh_engine(&pool);
|
||||
engine
|
||||
.set_role(
|
||||
s.owner,
|
||||
Subject::User(s.outsider),
|
||||
Role::Viewer,
|
||||
Resource::File(s.files[0]),
|
||||
None,
|
||||
)
|
||||
.await
|
||||
.expect("direct file grant");
|
||||
if !allowed(&engine, s.outsider, s.files[0]).await {
|
||||
eprintln!(
|
||||
"SAFETY GATE FAILED: direct file grant denied — the folder-level \
|
||||
decomposition shadowed the direct-grant branch"
|
||||
);
|
||||
cleanup(&pool, &s).await;
|
||||
std::process::exit(1);
|
||||
}
|
||||
if allowed(&engine, s.outsider, s.files[1]).await {
|
||||
eprintln!(
|
||||
"SAFETY GATE FAILED: direct grant on file[0] leaked to a sibling — \
|
||||
a file decision must never authorize other files"
|
||||
);
|
||||
cleanup(&pool, &s).await;
|
||||
std::process::exit(1);
|
||||
}
|
||||
engine
|
||||
.clear_role(Subject::User(s.outsider), Resource::File(s.files[0]))
|
||||
.await
|
||||
.expect("clear direct grant");
|
||||
}
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# shared-album thumbnail authz: folder-cascade query/thumb vs cache");
|
||||
println!("# thumbs={thumbs} (recipient holds a folder grant, no drive membership)");
|
||||
@@ -331,8 +384,66 @@ async fn main() {
|
||||
);
|
||||
}
|
||||
|
||||
// AFTER cold: one persistent engine — the first grid view queries once per
|
||||
// distinct file (cache misses populate).
|
||||
// ROUND8 cold: the historical per-file UNION (direct grant ∨ ltree
|
||||
// ancestor join) run verbatim once per file — what a cold first view
|
||||
// cost before the round-9 folder-level decomposition.
|
||||
{
|
||||
let subject_types: Vec<&str> = vec!["user", "group"];
|
||||
let subject_ids = vec![s.recipient];
|
||||
let roles: Vec<&str> = roles_implying(Permission::Read)
|
||||
.iter()
|
||||
.map(|r| r.as_str())
|
||||
.collect();
|
||||
let t = Instant::now();
|
||||
for &f in &s.files {
|
||||
let exists: Option<i32> = sqlx::query_scalar(
|
||||
r#"
|
||||
SELECT 1
|
||||
FROM (
|
||||
SELECT 1
|
||||
FROM storage.role_grants
|
||||
WHERE subject_type = ANY($1)
|
||||
AND subject_id = ANY($2)
|
||||
AND role = ANY($3::storage.grant_role[])
|
||||
AND resource_type = 'file' AND resource_id = $4
|
||||
AND (expires_at IS NULL OR expires_at > NOW())
|
||||
UNION ALL
|
||||
SELECT 1
|
||||
FROM storage.role_grants g
|
||||
JOIN storage.folders gf ON gf.id = g.resource_id
|
||||
JOIN storage.files target_f ON target_f.id = $4
|
||||
WHERE g.subject_type = ANY($1)
|
||||
AND g.subject_id = ANY($2)
|
||||
AND g.role = ANY($3::storage.grant_role[])
|
||||
AND g.resource_type = 'folder'
|
||||
AND (g.expires_at IS NULL OR g.expires_at > NOW())
|
||||
AND target_f.folder_id IS NOT NULL
|
||||
AND gf.lpath @> (SELECT lpath FROM storage.folders
|
||||
WHERE id = target_f.folder_id)
|
||||
) any_match
|
||||
LIMIT 1
|
||||
"#,
|
||||
)
|
||||
.bind(&subject_types)
|
||||
.bind(&subject_ids)
|
||||
.bind(&roles)
|
||||
.bind(f)
|
||||
.fetch_optional(pool.as_ref())
|
||||
.await
|
||||
.expect("round8 union query");
|
||||
assert!(exists.is_some(), "ROUND8 arm: recipient must be allowed");
|
||||
}
|
||||
let el = t.elapsed();
|
||||
println!(
|
||||
"| {:<28} | {:>10.2} | {:>12.2} |",
|
||||
"ROUND8 cold (union/file)",
|
||||
el.as_secs_f64() * 1e3,
|
||||
el.as_secs_f64() * 1e6 / thumbs as f64
|
||||
);
|
||||
}
|
||||
|
||||
// AFTER cold: one persistent engine — the first grid view resolves each
|
||||
// file's parent (PK read) and shares ONE folder-cascade decision.
|
||||
let engine = fresh_engine(&pool);
|
||||
{
|
||||
let t = Instant::now();
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
/**
|
||||
* Benchmark gate for the O(1) contact index behind `resolveLabel` /
|
||||
* `resolveRecipient` (recipients.ts).
|
||||
*
|
||||
* Audit finding: both resolvers ran `contactCache.find((x) => x.id === id)`
|
||||
* — a linear scan over the WHOLE system address book — once per rendered
|
||||
* grant row / lane header on /shared, and the page re-renders on every
|
||||
* infinite-scroll page and role change. Cost per frame: O(rows × directory
|
||||
* size) — ~150k comparisons for 30 rows in a 5 000-user org. The fix builds
|
||||
* a `Map<id, Contact>` once per cache identity (exactly like the existing
|
||||
* `groupCache`) and looks up O(1).
|
||||
*
|
||||
* Gates: (1) labels identical to the linear scan for present AND absent
|
||||
* ids; (2) comparison count collapses from rows×C to ~C (one index build);
|
||||
* (3) resolving a full page against a 5 000-contact directory is ≥10x
|
||||
* faster with the index.
|
||||
*/
|
||||
|
||||
interface Contact {
|
||||
id: string;
|
||||
full_name?: string;
|
||||
email?: string;
|
||||
}
|
||||
|
||||
function contactLabel(c: Contact): { label: string; email?: string } {
|
||||
return { label: c.full_name || c.email || c.id, email: c.email };
|
||||
}
|
||||
|
||||
function directory(n: number): Contact[] {
|
||||
return Array.from({ length: n }, (_, i) => ({
|
||||
id: `user-${i}`,
|
||||
full_name: `User Number ${i}`,
|
||||
email: `user${i}@example.com`
|
||||
}));
|
||||
}
|
||||
|
||||
/** BEFORE — verbatim resolver shape: linear `.find` per call. */
|
||||
function makeBefore(cache: Contact[], counter: { cmp: number }) {
|
||||
return (id: string): string => {
|
||||
let found: Contact | undefined;
|
||||
for (const x of cache) {
|
||||
counter.cmp++;
|
||||
if (x.id === id) {
|
||||
found = x;
|
||||
break;
|
||||
}
|
||||
}
|
||||
return found ? contactLabel(found).label : id;
|
||||
};
|
||||
}
|
||||
|
||||
/** AFTER — the shipped shape: identity-memoized Map index, O(1) get. */
|
||||
function makeAfter(cache: Contact[], counter: { cmp: number }) {
|
||||
let contactById: Map<string, Contact> | null = null;
|
||||
let source: Contact[] | null = null;
|
||||
const index = () => {
|
||||
if (!contactById || source !== cache) {
|
||||
contactById = new Map(
|
||||
cache.map((c) => {
|
||||
counter.cmp++;
|
||||
return [c.id, c] as const;
|
||||
})
|
||||
);
|
||||
source = cache;
|
||||
}
|
||||
return contactById;
|
||||
};
|
||||
return (id: string): string => {
|
||||
const c = index().get(id);
|
||||
return c ? contactLabel(c).label : id;
|
||||
};
|
||||
}
|
||||
|
||||
describe('resolveLabel contact index (benchmark gate)', () => {
|
||||
const C = 5_000;
|
||||
const contacts = directory(C);
|
||||
// A /shared page: 30 rows, most present, some unknown (revoked users).
|
||||
const rowIds = [
|
||||
...Array.from({ length: 26 }, (_, i) => `user-${i * 137}`),
|
||||
'ghost-1',
|
||||
'ghost-2',
|
||||
'user-4999',
|
||||
'ghost-3'
|
||||
];
|
||||
|
||||
it('labels identical to the linear scan for present and absent ids', () => {
|
||||
const before = makeBefore(contacts, { cmp: 0 });
|
||||
const after = makeAfter(contacts, { cmp: 0 });
|
||||
for (const id of rowIds) {
|
||||
expect(after(id), id).toBe(before(id));
|
||||
}
|
||||
// Absent ids fall back to the raw id in both.
|
||||
expect(after('ghost-1')).toBe('ghost-1');
|
||||
});
|
||||
|
||||
it('comparison count collapses from rows×C to one index build (~C)', () => {
|
||||
const beforeCounter = { cmp: 0 };
|
||||
const before = makeBefore(contacts, beforeCounter);
|
||||
for (const id of rowIds) before(id);
|
||||
// Linear scans: each present id walks ~id-position entries, absent
|
||||
// ids walk the full directory.
|
||||
expect(beforeCounter.cmp).toBeGreaterThan(C * 3);
|
||||
|
||||
const afterCounter = { cmp: 0 };
|
||||
const after = makeAfter(contacts, afterCounter);
|
||||
for (const id of rowIds) after(id);
|
||||
// One index build (C inserts), zero comparisons per lookup after.
|
||||
expect(afterCounter.cmp).toBe(C);
|
||||
|
||||
// A SECOND render frame re-uses the index: zero additional work.
|
||||
for (const id of rowIds) after(id);
|
||||
expect(afterCounter.cmp).toBe(C);
|
||||
});
|
||||
|
||||
it('resolving a page against a 5k directory is ≥10x faster with the index', () => {
|
||||
const frames = 50;
|
||||
|
||||
const before = makeBefore(contacts, { cmp: 0 });
|
||||
const t0 = performance.now();
|
||||
for (let f = 0; f < frames; f++) {
|
||||
for (const id of rowIds) before(id);
|
||||
}
|
||||
const beforeMs = performance.now() - t0;
|
||||
|
||||
const after = makeAfter(contacts, { cmp: 0 });
|
||||
const t1 = performance.now();
|
||||
for (let f = 0; f < frames; f++) {
|
||||
for (const id of rowIds) after(id);
|
||||
}
|
||||
const afterMs = performance.now() - t1;
|
||||
|
||||
console.log(
|
||||
`resolveLabel ${frames} frames × ${rowIds.length} rows @ C=${C}: ` +
|
||||
`before ${beforeMs.toFixed(1)} ms, after ${afterMs.toFixed(1)} ms ` +
|
||||
`(${(beforeMs / afterMs).toFixed(1)}x)`
|
||||
);
|
||||
expect(afterMs).toBeLessThan(beforeMs / 10);
|
||||
});
|
||||
});
|
||||
@@ -134,10 +134,26 @@ export async function ensureResolvers(): Promise<void> {
|
||||
await Promise.all([systemContacts(), loadGroups()]);
|
||||
}
|
||||
|
||||
// O(1) id→contact index over `contactCache`, built once per cache identity.
|
||||
// `resolveLabel`/`resolveRecipient` run per rendered grant row on /shared —
|
||||
// the previous `contactCache.find(...)` linear scan made each render frame
|
||||
// O(rows × directory size).
|
||||
let contactById: Map<string, Contact> | null = null;
|
||||
let contactByIdSource: Contact[] | null = null;
|
||||
|
||||
function contactIndex(): Map<string, Contact> | null {
|
||||
if (!contactCache) return null;
|
||||
if (!contactById || contactByIdSource !== contactCache) {
|
||||
contactById = new Map(contactCache.map((c) => [c.id, c]));
|
||||
contactByIdSource = contactCache;
|
||||
}
|
||||
return contactById;
|
||||
}
|
||||
|
||||
/** Resolve a subject id to a display label using the preloaded caches. */
|
||||
export function resolveLabel(type: 'user' | 'group', id: string): string {
|
||||
if (type === 'group') return groupCache?.get(id) ?? id;
|
||||
const c = contactCache?.find((x) => x.id === id);
|
||||
const c = contactIndex()?.get(id);
|
||||
return c ? contactLabel(c).label : id;
|
||||
}
|
||||
|
||||
@@ -146,7 +162,7 @@ export function resolveRecipient(type: 'user' | 'group', id: string): Recipient
|
||||
if (type === 'group') {
|
||||
return { type: 'group', id, label: groupCache?.get(id) ?? id };
|
||||
}
|
||||
const c = contactCache?.find((x) => x.id === id);
|
||||
const c = contactIndex()?.get(id);
|
||||
if (!c) return { type: 'user', id, label: id };
|
||||
const { label, email } = contactLabel(c);
|
||||
return { type: 'user', id, label, sublabel: email };
|
||||
|
||||
@@ -256,6 +256,11 @@
|
||||
|
||||
// Drop selection ids that are no longer present after a reload.
|
||||
$effect(() => {
|
||||
// With nothing selected (the common case) every infinite-scroll page
|
||||
// re-fired this effect and built a throwaway O(N) id Set for a loop
|
||||
// that never runs — skip straight out. `selected.size` is reactive,
|
||||
// so the effect re-fires when a selection appears.
|
||||
if (selected.size === 0) return;
|
||||
const ids = new Set(items.map((i) => i.id));
|
||||
let changed = false;
|
||||
for (const id of selected) {
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
/**
|
||||
* Benchmark gates for two per-page derive cleanups (round 9):
|
||||
*
|
||||
* [1] ResourceList's selection-prune `$effect` built an O(N) id `Set` on
|
||||
* EVERY `items` change (every infinite-scroll page) even when nothing
|
||||
* was selected — the loop it feeds never runs in that case. The shipped
|
||||
* guard (`if (selected.size === 0) return`) makes the empty-selection
|
||||
* page append free while keeping the pruned result byte-identical when
|
||||
* a selection exists.
|
||||
*
|
||||
* [2] The photos timeline derive called `window.matchMedia(...)` on every
|
||||
* recompute (every 60-photo page append) for a boolean that changes
|
||||
* only on viewport-class crossings. The shipped code hoists it into
|
||||
* state fed by a single MediaQueryList `change` listener.
|
||||
*
|
||||
* Both are modeled as pure replicas of the effect/derive bodies (no jsdom
|
||||
* mounting needed) with instrumentation counters, mirroring the shipped
|
||||
* control flow exactly.
|
||||
*/
|
||||
|
||||
interface Item {
|
||||
id: string;
|
||||
}
|
||||
|
||||
const page = (start: number, n: number): Item[] =>
|
||||
Array.from({ length: n }, (_, i) => ({ id: `it-${start + i}` }));
|
||||
|
||||
/** BEFORE — verbatim effect body: unconditional Set build. */
|
||||
function pruneBefore(items: Item[], selected: Set<string>, counter: { setBuilds: number }) {
|
||||
counter.setBuilds++;
|
||||
const ids = new Set(items.map((i) => i.id));
|
||||
for (const id of [...selected]) {
|
||||
if (!ids.has(id)) selected.delete(id);
|
||||
}
|
||||
}
|
||||
|
||||
/** AFTER — the shipped body: skip entirely while nothing is selected. */
|
||||
function pruneAfter(items: Item[], selected: Set<string>, counter: { setBuilds: number }) {
|
||||
if (selected.size === 0) return;
|
||||
counter.setBuilds++;
|
||||
const ids = new Set(items.map((i) => i.id));
|
||||
for (const id of [...selected]) {
|
||||
if (!ids.has(id)) selected.delete(id);
|
||||
}
|
||||
}
|
||||
|
||||
describe('selection-prune guard (benchmark gate)', () => {
|
||||
it('empty selection: zero Set builds across a 100-page drain (was 100)', () => {
|
||||
const beforeCounter = { setBuilds: 0 };
|
||||
const afterCounter = { setBuilds: 0 };
|
||||
let items: Item[] = [];
|
||||
for (let p = 0; p < 100; p++) {
|
||||
items = [...items, ...page(p * 50, 50)];
|
||||
pruneBefore(items, new Set(), beforeCounter);
|
||||
pruneAfter(items, new Set(), afterCounter);
|
||||
}
|
||||
expect(beforeCounter.setBuilds).toBe(100);
|
||||
expect(afterCounter.setBuilds).toBe(0);
|
||||
});
|
||||
|
||||
it('active selection: pruned set identical to the unguarded version', () => {
|
||||
const items = page(0, 200);
|
||||
// Selection holds survivors + ids that vanished on reload.
|
||||
const seed = ['it-3', 'it-77', 'gone-1', 'it-150', 'gone-2'];
|
||||
const a = new Set(seed);
|
||||
const b = new Set(seed);
|
||||
pruneBefore(items, a, { setBuilds: 0 });
|
||||
pruneAfter(items, b, { setBuilds: 0 });
|
||||
expect([...b].sort()).toEqual([...a].sort());
|
||||
expect(b.has('gone-1')).toBe(false);
|
||||
expect(b.has('it-3')).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
// ── [2] matchMedia hoist ────────────────────────────────────────────────────
|
||||
|
||||
interface MqlStub {
|
||||
matches: boolean;
|
||||
listeners: ((e: { matches: boolean }) => void)[];
|
||||
}
|
||||
|
||||
function makeMatchMedia(counter: { calls: number }, stub: MqlStub) {
|
||||
return () => {
|
||||
counter.calls++;
|
||||
return {
|
||||
get matches() {
|
||||
return stub.matches;
|
||||
},
|
||||
addEventListener: (_: 'change', fn: (e: { matches: boolean }) => void) => {
|
||||
stub.listeners.push(fn);
|
||||
},
|
||||
removeEventListener: () => {}
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
describe('photos matchMedia hoist (benchmark gate)', () => {
|
||||
it('P recomputes: 1 matchMedia call instead of P, identical booleans', () => {
|
||||
const P = 50;
|
||||
const stub: MqlStub = { matches: false, listeners: [] };
|
||||
|
||||
// BEFORE — the derive body queries per recompute.
|
||||
const beforeCounter = { calls: 0 };
|
||||
const mmBefore = makeMatchMedia(beforeCounter, stub);
|
||||
const beforeValues: boolean[] = [];
|
||||
for (let i = 0; i < P; i++) {
|
||||
beforeValues.push(mmBefore().matches);
|
||||
}
|
||||
expect(beforeCounter.calls).toBe(P);
|
||||
|
||||
// AFTER — one query + listener; recomputes read the state boolean.
|
||||
const afterCounter = { calls: 0 };
|
||||
const mmAfter = makeMatchMedia(afterCounter, stub);
|
||||
const mql = mmAfter();
|
||||
let isMobile = mql.matches;
|
||||
mql.addEventListener('change', (e) => {
|
||||
isMobile = e.matches;
|
||||
});
|
||||
const afterValues: boolean[] = [];
|
||||
for (let i = 0; i < P; i++) {
|
||||
afterValues.push(isMobile);
|
||||
}
|
||||
expect(afterCounter.calls).toBe(1);
|
||||
expect(afterValues).toEqual(beforeValues);
|
||||
|
||||
// A viewport-class crossing propagates through the listener.
|
||||
stub.matches = true;
|
||||
for (const fn of stub.listeners) fn({ matches: true });
|
||||
expect(isMobile).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -94,15 +94,27 @@
|
||||
// deps re-fire without an actual append, `sync` sees a non-growing list and
|
||||
// safely full-rebuilds — same output as the pure `buildPhotoRows`.
|
||||
const timeline = new PhotoTimeline();
|
||||
// `mobile` as state fed by one MediaQueryList listener: the derive below
|
||||
// re-runs on every page append, and `window.matchMedia(...)` inside it was
|
||||
// a per-recompute style/layout read that only changes on viewport-class
|
||||
// crossings — now those crossings push the boolean instead.
|
||||
let isMobile = $state(false);
|
||||
$effect(() => {
|
||||
if (typeof window === 'undefined' || typeof window.matchMedia !== 'function') return;
|
||||
const mql = window.matchMedia('(max-width: 768px)');
|
||||
isMobile = mql.matches;
|
||||
const onchange = (e: MediaQueryListEvent) => {
|
||||
isMobile = e.matches;
|
||||
};
|
||||
mql.addEventListener('change', onchange);
|
||||
return () => mql.removeEventListener('change', onchange);
|
||||
});
|
||||
const photoRows = $derived.by<PhotoRow[]>(() =>
|
||||
timeline.sync(visibleItems, {
|
||||
groupMode,
|
||||
layoutMode,
|
||||
width: gridWidth,
|
||||
mobile:
|
||||
typeof window !== 'undefined' &&
|
||||
typeof window.matchMedia === 'function' &&
|
||||
window.matchMedia('(max-width: 768px)').matches,
|
||||
mobile: isMobile,
|
||||
timestampOf: photoTimestamp,
|
||||
labelOf: bucketLabel
|
||||
})
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::sync::Arc;
|
||||
use utoipa::ToSchema;
|
||||
|
||||
/**
|
||||
@@ -109,8 +110,10 @@ pub struct SearchFileResultDto {
|
||||
pub path: String,
|
||||
/// Size in bytes
|
||||
pub size: u64,
|
||||
/// MIME type
|
||||
pub mime_type: String,
|
||||
/// MIME type — `Arc<str>` so enrichment reuses `FileDto`'s interned
|
||||
/// value (an atomic increment) instead of allocating per result row.
|
||||
#[schema(value_type = String)]
|
||||
pub mime_type: Arc<str>,
|
||||
/// Parent folder ID
|
||||
pub folder_id: Option<String>,
|
||||
/// Creation timestamp
|
||||
@@ -122,11 +125,14 @@ pub struct SearchFileResultDto {
|
||||
/// Human-readable file size (e.g., "2.5 MB")
|
||||
pub size_formatted: String,
|
||||
/// CSS icon class for the file type (e.g., "fas fa-file-pdf")
|
||||
pub icon_class: String,
|
||||
#[schema(value_type = String)]
|
||||
pub icon_class: Arc<str>,
|
||||
/// Extra CSS class for icon styling (e.g., "pdf-icon", "code-icon js-icon")
|
||||
pub icon_special_class: String,
|
||||
#[schema(value_type = String)]
|
||||
pub icon_special_class: Arc<str>,
|
||||
/// Content category: "document", "image", "video", "audio", "archive", "code", "other"
|
||||
pub category: String,
|
||||
#[schema(value_type = String)]
|
||||
pub category: Arc<str>,
|
||||
/// Raw BLAKE3 content hash. Feeds `FileDto::content_hash` and
|
||||
/// `File::compute_etag` when search results are converted to
|
||||
/// `FileDto` (NC REPORT/SEARCH response). Defaults to `String::new()`
|
||||
@@ -267,9 +273,11 @@ pub struct SearchSuggestionItem {
|
||||
/// Path for context
|
||||
pub path: String,
|
||||
/// CSS icon class
|
||||
pub icon_class: String,
|
||||
#[schema(value_type = String)]
|
||||
pub icon_class: Arc<str>,
|
||||
/// Extra CSS class for icon styling
|
||||
pub icon_special_class: String,
|
||||
#[schema(value_type = String)]
|
||||
pub icon_special_class: Arc<str>,
|
||||
/// Relevance score
|
||||
pub relevance_score: u32,
|
||||
}
|
||||
|
||||
@@ -2,9 +2,7 @@ use std::cmp::Reverse;
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use crate::application::dtos::display_helpers::{
|
||||
category_for, icon_class_for, icon_special_class_for,
|
||||
};
|
||||
use crate::application::dtos::display_helpers::intern_display;
|
||||
use crate::application::dtos::file_dto::FileDto;
|
||||
use crate::application::dtos::folder_dto::FolderDto;
|
||||
use crate::application::dtos::search_dto::{
|
||||
@@ -209,23 +207,6 @@ fn format_bytes(bytes: u64) -> String {
|
||||
}
|
||||
}
|
||||
|
||||
/// Get Font Awesome icon class for a file based on extension and MIME type.
|
||||
/// Delegates to the centralised `display_helpers` so every API surface is
|
||||
/// consistent.
|
||||
fn get_icon_class(name: &str, mime: &str) -> String {
|
||||
icon_class_for(name, mime).to_string()
|
||||
}
|
||||
|
||||
/// Get CSS special class for icon styling.
|
||||
fn get_icon_special_class(name: &str, mime: &str) -> String {
|
||||
icon_special_class_for(name, mime).to_string()
|
||||
}
|
||||
|
||||
/// Get category label from centralised helpers.
|
||||
fn get_category(name: &str, mime: &str) -> String {
|
||||
category_for(name, mime).to_string()
|
||||
}
|
||||
|
||||
// ─── SearchService implementation ───────────────────────────────────────
|
||||
|
||||
impl SearchService {
|
||||
@@ -267,8 +248,14 @@ impl SearchService {
|
||||
|
||||
/// Enrich a FileDto → SearchFileResultDto with server-computed metadata.
|
||||
///
|
||||
/// Consumes the DTO: every `String` moves and the interned display
|
||||
/// fields (`mime_type`/`icon_class`/`icon_special_class`/`category`,
|
||||
/// already computed once in `FileDto::from`) transfer as refcount
|
||||
/// bumps — the old borrow-based version cloned all of them AND re-ran
|
||||
/// the three display classifiers per result row.
|
||||
///
|
||||
/// `query_lower` must already be lowercased (empty string when no query).
|
||||
fn enrich_file(file: &FileDto, query_lower: &str) -> SearchFileResultDto {
|
||||
fn enrich_file(file: FileDto, query_lower: &str) -> SearchFileResultDto {
|
||||
let relevance = if query_lower.is_empty() {
|
||||
50
|
||||
} else {
|
||||
@@ -276,23 +263,23 @@ impl SearchService {
|
||||
};
|
||||
|
||||
SearchFileResultDto {
|
||||
id: file.id.clone(),
|
||||
name: file.name.clone(),
|
||||
path: file.path.clone(),
|
||||
id: file.id,
|
||||
name: file.name,
|
||||
path: file.path,
|
||||
size: file.size,
|
||||
mime_type: file.mime_type.to_string(),
|
||||
folder_id: file.folder_id.clone(),
|
||||
mime_type: file.mime_type,
|
||||
folder_id: file.folder_id,
|
||||
created_at: file.created_at,
|
||||
modified_at: file.modified_at,
|
||||
relevance_score: relevance,
|
||||
size_formatted: format_bytes(file.size),
|
||||
icon_class: get_icon_class(&file.name, &file.mime_type),
|
||||
icon_special_class: get_icon_special_class(&file.name, &file.mime_type),
|
||||
category: get_category(&file.name, &file.mime_type),
|
||||
icon_class: file.icon_class,
|
||||
icon_special_class: file.icon_special_class,
|
||||
category: file.category,
|
||||
// Carry the content hash through so REPORT/SEARCH
|
||||
// responses on the NC surface can emit the same ETag
|
||||
// (`File::compute_etag`) as PROPFIND/GET would.
|
||||
blob_hash: file.content_hash.clone(),
|
||||
blob_hash: file.content_hash,
|
||||
snippet: None,
|
||||
match_source: (!query_lower.is_empty() && relevance > 0).then(|| "name".to_string()),
|
||||
}
|
||||
@@ -300,8 +287,10 @@ impl SearchService {
|
||||
|
||||
/// Enrich a FolderDto → SearchFolderResultDto with server-computed metadata.
|
||||
///
|
||||
/// Consumes the DTO so the owned strings move instead of cloning.
|
||||
///
|
||||
/// `query_lower` must already be lowercased (empty string when no query).
|
||||
fn enrich_folder(folder: &FolderDto, query_lower: &str) -> SearchFolderResultDto {
|
||||
fn enrich_folder(folder: FolderDto, query_lower: &str) -> SearchFolderResultDto {
|
||||
let relevance = if query_lower.is_empty() {
|
||||
50
|
||||
} else {
|
||||
@@ -309,10 +298,10 @@ impl SearchService {
|
||||
};
|
||||
|
||||
SearchFolderResultDto {
|
||||
id: folder.id.clone(),
|
||||
name: folder.name.clone(),
|
||||
path: folder.path.clone(),
|
||||
parent_id: folder.parent_id.clone(),
|
||||
id: folder.id,
|
||||
name: folder.name,
|
||||
path: folder.path,
|
||||
parent_id: folder.parent_id,
|
||||
drive_id: folder.drive_id,
|
||||
created_at: folder.created_at,
|
||||
modified_at: folder.modified_at,
|
||||
@@ -481,9 +470,10 @@ impl SearchService {
|
||||
let Some(hit) = by_id.get(dto.id.as_str()) else {
|
||||
continue;
|
||||
};
|
||||
let mut enriched = Self::enrich_file(&dto, "");
|
||||
enriched.relevance_score = content_relevance(hit.score, max_score);
|
||||
enriched.snippet = hit.snippet.clone();
|
||||
let (score, snippet) = (hit.score, hit.snippet.clone());
|
||||
let mut enriched = Self::enrich_file(dto, "");
|
||||
enriched.relevance_score = content_relevance(score, max_score);
|
||||
enriched.snippet = snippet;
|
||||
enriched.match_source = Some("content".to_string());
|
||||
enriched_files.push(enriched);
|
||||
added += 1;
|
||||
@@ -536,15 +526,15 @@ impl SearchService {
|
||||
for file in files {
|
||||
let file_dto = FileDto::from(file);
|
||||
let score = compute_relevance(&file_dto.name, &query_lower);
|
||||
let icon_class = get_icon_class(&file_dto.name, &file_dto.mime_type);
|
||||
let icon_special_class = get_icon_special_class(&file_dto.name, &file_dto.mime_type);
|
||||
suggestions.push(SearchSuggestionItem {
|
||||
name: file_dto.name,
|
||||
item_type: "file".to_string(),
|
||||
id: file_dto.id,
|
||||
path: file_dto.path,
|
||||
icon_class,
|
||||
icon_special_class,
|
||||
// Interned in `FileDto::from` — reuse instead of re-running
|
||||
// the display classifiers per keystroke suggestion.
|
||||
icon_class: file_dto.icon_class,
|
||||
icon_special_class: file_dto.icon_special_class,
|
||||
relevance_score: score,
|
||||
});
|
||||
}
|
||||
@@ -557,8 +547,8 @@ impl SearchService {
|
||||
item_type: "folder".to_string(),
|
||||
id: folder_dto.id,
|
||||
path: folder_dto.path,
|
||||
icon_class: "fas fa-folder".to_string(),
|
||||
icon_special_class: "folder-icon".to_string(),
|
||||
icon_class: intern_display("fas fa-folder"),
|
||||
icon_special_class: intern_display("folder-icon"),
|
||||
relevance_score: score,
|
||||
});
|
||||
}
|
||||
@@ -575,6 +565,22 @@ impl SearchService {
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Bench-only public wrappers (feature = "bench") ──────────────────────
|
||||
|
||||
#[cfg(feature = "bench")]
|
||||
impl SearchService {
|
||||
/// Public wrapper over the private `enrich_file` so
|
||||
/// `examples/bench_search_enrich.rs` can measure it.
|
||||
pub fn enrich_file_for_bench(file: FileDto, query_lower: &str) -> SearchFileResultDto {
|
||||
Self::enrich_file(file, query_lower)
|
||||
}
|
||||
|
||||
/// Public wrapper over the private `enrich_folder` for the same bench.
|
||||
pub fn enrich_folder_for_bench(folder: FolderDto, query_lower: &str) -> SearchFolderResultDto {
|
||||
Self::enrich_folder(folder, query_lower)
|
||||
}
|
||||
}
|
||||
|
||||
// ─── SearchUseCase trait implementation ──────────────────────────────────
|
||||
|
||||
impl SearchUseCase for SearchService {
|
||||
@@ -627,11 +633,11 @@ impl SearchUseCase for SearchService {
|
||||
.search_files_paginated(criteria.folder_id.as_deref(), &criteria, user_id)
|
||||
.await?;
|
||||
|
||||
// Convert to DTOs and enrich with metadata
|
||||
let file_dtos: Vec<FileDto> = files.into_iter().map(FileDto::from).collect();
|
||||
let mut enriched_files: Vec<SearchFileResultDto> = file_dtos
|
||||
.iter()
|
||||
.map(|f| Self::enrich_file(f, &query_lower))
|
||||
// Convert to DTOs and enrich with metadata — one fused
|
||||
// pass, no intermediate Vec<FileDto> materialization.
|
||||
let mut enriched_files: Vec<SearchFileResultDto> = files
|
||||
.into_iter()
|
||||
.map(|f| Self::enrich_file(FileDto::from(f), &query_lower))
|
||||
.collect();
|
||||
|
||||
// Get folders for this folder (non-recursive, filtered in SQL)
|
||||
@@ -645,13 +651,10 @@ impl SearchUseCase for SearchService {
|
||||
)
|
||||
.await?;
|
||||
|
||||
let filtered_folders: Vec<FolderDto> =
|
||||
folders.into_iter().map(FolderDto::from).collect();
|
||||
|
||||
// For folders, apply sorting and pagination in memory (usually fewer folders)
|
||||
let mut enriched_folders: Vec<SearchFolderResultDto> = filtered_folders
|
||||
.iter()
|
||||
.map(|f| Self::enrich_folder(f, &query_lower))
|
||||
let mut enriched_folders: Vec<SearchFolderResultDto> = folders
|
||||
.into_iter()
|
||||
.map(|f| Self::enrich_folder(FolderDto::from(f), &query_lower))
|
||||
.collect();
|
||||
|
||||
// Sort folders (cached_key avoids O(N log N) temporary String allocations)
|
||||
@@ -732,17 +735,15 @@ impl SearchUseCase for SearchService {
|
||||
.await?;
|
||||
|
||||
// ── Convert to DTOs and enrich with server-computed metadata ──
|
||||
let file_dtos: Vec<FileDto> = found_files.into_iter().map(FileDto::from).collect();
|
||||
let mut enriched_files: Vec<SearchFileResultDto> = file_dtos
|
||||
.iter()
|
||||
.map(|f| Self::enrich_file(f, &query_lower))
|
||||
// Fused single pass: no intermediate DTO Vec materialization.
|
||||
let mut enriched_files: Vec<SearchFileResultDto> = found_files
|
||||
.into_iter()
|
||||
.map(|f| Self::enrich_file(FileDto::from(f), &query_lower))
|
||||
.collect();
|
||||
|
||||
let folder_dtos: Vec<FolderDto> =
|
||||
found_folders.into_iter().map(FolderDto::from).collect();
|
||||
let mut enriched_folders: Vec<SearchFolderResultDto> = folder_dtos
|
||||
.iter()
|
||||
.map(|f| Self::enrich_folder(f, &query_lower))
|
||||
let mut enriched_folders: Vec<SearchFolderResultDto> = found_folders
|
||||
.into_iter()
|
||||
.map(|f| Self::enrich_folder(FolderDto::from(f), &query_lower))
|
||||
.collect();
|
||||
|
||||
// ── Sort folders (cached_key avoids O(N log N) temporary String allocations) ──
|
||||
@@ -893,15 +894,15 @@ mod tests {
|
||||
name: name.to_string(),
|
||||
path: format!("/{name}"),
|
||||
size,
|
||||
mime_type: "text/plain".to_string(),
|
||||
mime_type: "text/plain".into(),
|
||||
folder_id: None,
|
||||
created_at: 0,
|
||||
modified_at,
|
||||
relevance_score: relevance,
|
||||
size_formatted: String::new(),
|
||||
icon_class: String::new(),
|
||||
icon_special_class: String::new(),
|
||||
category: String::new(),
|
||||
icon_class: "".into(),
|
||||
icon_special_class: "".into(),
|
||||
category: "".into(),
|
||||
blob_hash: String::new(),
|
||||
snippet: None,
|
||||
match_source: None,
|
||||
|
||||
@@ -498,21 +498,26 @@ impl DriveRepository for DrivePgRepository {
|
||||
// the trash. Trashed items don't count — owners can delete a
|
||||
// drive even when its trash bin still holds rows; the trash GC
|
||||
// will clean those up after the standard retention window.
|
||||
let count: (i64,) = sqlx::query_as(
|
||||
//
|
||||
// EXISTS instead of COUNT(*): only emptiness is tested, so the
|
||||
// planner stops at the first matching row — a populated drive
|
||||
// answers from one index probe instead of aggregating every
|
||||
// live file + folder it contains.
|
||||
let occupied: (bool,) = sqlx::query_as(
|
||||
r#"
|
||||
SELECT (
|
||||
(SELECT COUNT(*) FROM storage.folders
|
||||
WHERE drive_id = $1 AND parent_id IS NOT NULL AND NOT is_trashed)
|
||||
+ (SELECT COUNT(*) FROM storage.files
|
||||
WHERE drive_id = $1 AND NOT is_trashed)
|
||||
)
|
||||
SELECT EXISTS(
|
||||
SELECT 1 FROM storage.folders
|
||||
WHERE drive_id = $1 AND parent_id IS NOT NULL AND NOT is_trashed)
|
||||
OR EXISTS(
|
||||
SELECT 1 FROM storage.files
|
||||
WHERE drive_id = $1 AND NOT is_trashed)
|
||||
"#,
|
||||
)
|
||||
.bind(drive_id)
|
||||
.fetch_one(self.pool.as_ref())
|
||||
.await
|
||||
.map_err(|e| Self::map_sqlx_err("is_empty", e))?;
|
||||
Ok(count.0 == 0)
|
||||
Ok(!occupied.0)
|
||||
}
|
||||
|
||||
async fn delete_atomic(&self, drive_id: Uuid) -> Result<(), DriveRepositoryError> {
|
||||
|
||||
@@ -32,11 +32,15 @@ use crate::domain::services::path_service::StoragePath;
|
||||
/// Post-D7-step-6: `storage.folders.user_id` dropped, so the tuple
|
||||
/// no longer carries it. The domain entity's `user_id` field is
|
||||
/// populated with `None` at `row_to_folder` construction.
|
||||
/// `id` / `parent_id` decode as binary `Uuid` (16 bytes on the wire vs 36
|
||||
/// as `::text`, and the server skips the cast); `row_to_folder` renders
|
||||
/// them to `String` once app-side — the round-6 `row_to_file` shape
|
||||
/// (benches/ROUND6.md §10) applied to the folder listings.
|
||||
type FolderRow = (
|
||||
Uuid,
|
||||
String,
|
||||
String,
|
||||
String,
|
||||
Option<String>,
|
||||
Option<Uuid>,
|
||||
Uuid,
|
||||
i64,
|
||||
i64,
|
||||
@@ -49,10 +53,10 @@ type FolderRow = (
|
||||
/// the last element after the §14 provenance columns). Same
|
||||
/// column set as [`FolderRow`] plus the trailing count.
|
||||
type FolderRowPaginated = (
|
||||
Uuid,
|
||||
String,
|
||||
String,
|
||||
String,
|
||||
Option<String>,
|
||||
Option<Uuid>,
|
||||
Uuid,
|
||||
i64,
|
||||
i64,
|
||||
@@ -131,10 +135,10 @@ impl FolderDbRepository {
|
||||
/// `Option<Uuid>` because the FK is `ON DELETE SET NULL`.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn row_to_folder(
|
||||
id: String,
|
||||
id: Uuid,
|
||||
name: String,
|
||||
path: String,
|
||||
parent_id: Option<String>,
|
||||
parent_id: Option<Uuid>,
|
||||
drive_id: Uuid,
|
||||
created_at: i64,
|
||||
modified_at: i64,
|
||||
@@ -143,10 +147,10 @@ impl FolderDbRepository {
|
||||
updated_by: Option<Uuid>,
|
||||
) -> Result<Folder, DomainError> {
|
||||
Folder::from_materialized_row(
|
||||
id,
|
||||
id.to_string(),
|
||||
name,
|
||||
path,
|
||||
parent_id,
|
||||
parent_id.map(|u| u.to_string()),
|
||||
drive_id,
|
||||
created_at as u64,
|
||||
modified_at as u64,
|
||||
@@ -170,7 +174,7 @@ impl FolderDbRepository {
|
||||
|
||||
let rows = sqlx::query_as::<_, FolderRow>(
|
||||
r#"
|
||||
SELECT id::text, name, path, parent_id::text, drive_id,
|
||||
SELECT id, name, path, parent_id, drive_id,
|
||||
EXTRACT(EPOCH FROM created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM updated_at)::bigint,
|
||||
EXTRACT(EPOCH FROM tree_modified_at)::bigint,
|
||||
@@ -235,12 +239,25 @@ impl FolderRepository for FolderDbRepository {
|
||||
//
|
||||
// RETURNING surfaces the two provenance columns so the built
|
||||
// entity / DTO carries fresh values without a re-read.
|
||||
let row = sqlx::query_as::<_, (String, String, i64, i64, i64, Option<Uuid>, Option<Uuid>)>(
|
||||
let row = sqlx::query_as::<
|
||||
_,
|
||||
(
|
||||
Uuid,
|
||||
Option<Uuid>,
|
||||
String,
|
||||
i64,
|
||||
i64,
|
||||
i64,
|
||||
Option<Uuid>,
|
||||
Option<Uuid>,
|
||||
),
|
||||
>(
|
||||
r#"
|
||||
INSERT INTO storage.folders
|
||||
(name, parent_id, drive_id, created_by, updated_by)
|
||||
VALUES ($1, $2::uuid, $3, $4, $4)
|
||||
RETURNING id::text,
|
||||
RETURNING id,
|
||||
parent_id,
|
||||
path,
|
||||
EXTRACT(EPOCH FROM created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM updated_at)::bigint,
|
||||
@@ -268,16 +285,16 @@ impl FolderRepository for FolderDbRepository {
|
||||
})?;
|
||||
|
||||
Self::row_to_folder(
|
||||
row.0, name, row.1, parent_id, drive_id, row.2, row.3, row.4,
|
||||
row.0, name, row.2, row.1, drive_id, row.3, row.4, row.5,
|
||||
// Fresh from RETURNING — caller_id was bound to both columns.
|
||||
row.5, row.6,
|
||||
row.6, row.7,
|
||||
)
|
||||
}
|
||||
|
||||
async fn get_folder(&self, id: &str) -> Result<Folder, DomainError> {
|
||||
let row = sqlx::query_as::<_, FolderRow>(
|
||||
r#"
|
||||
SELECT id::text, name, path, parent_id::text, drive_id,
|
||||
SELECT id, name, path, parent_id, drive_id,
|
||||
EXTRACT(EPOCH FROM created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM updated_at)::bigint,
|
||||
EXTRACT(EPOCH FROM tree_modified_at)::bigint,
|
||||
@@ -319,7 +336,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
// wrapper scoping post-D0).
|
||||
let row = sqlx::query_as::<_, FolderRow>(
|
||||
r#"
|
||||
SELECT id::text, name, path, parent_id::text, drive_id,
|
||||
SELECT id, name, path, parent_id, drive_id,
|
||||
EXTRACT(EPOCH FROM created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM updated_at)::bigint,
|
||||
EXTRACT(EPOCH FROM tree_modified_at)::bigint,
|
||||
@@ -345,7 +362,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
let rows: Vec<FolderRow> = if let Some(pid) = parent_id {
|
||||
sqlx::query_as(
|
||||
r#"
|
||||
SELECT id::text, name, path, parent_id::text, drive_id,
|
||||
SELECT id, name, path, parent_id, drive_id,
|
||||
EXTRACT(EPOCH FROM created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM updated_at)::bigint,
|
||||
EXTRACT(EPOCH FROM tree_modified_at)::bigint,
|
||||
@@ -361,7 +378,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
} else {
|
||||
sqlx::query_as(
|
||||
r#"
|
||||
SELECT id::text, name, path, parent_id::text, drive_id,
|
||||
SELECT id, name, path, parent_id, drive_id,
|
||||
EXTRACT(EPOCH FROM created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM updated_at)::bigint,
|
||||
EXTRACT(EPOCH FROM tree_modified_at)::bigint,
|
||||
@@ -404,7 +421,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
// top of `folder_repository.rs`. Frontend cross-references
|
||||
// `/api/drives::caller_role` via `folder.drive_id`.
|
||||
let sql = format!(
|
||||
"SELECT fo.id::text, fo.name, fo.path, fo.parent_id::text, \
|
||||
"SELECT fo.id, fo.name, fo.path, fo.parent_id, \
|
||||
fo.drive_id, \
|
||||
EXTRACT(EPOCH FROM fo.created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM fo.updated_at)::bigint, \
|
||||
@@ -445,7 +462,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
let rows: Vec<FolderRowPaginated> = if let Some(pid) = parent_id {
|
||||
sqlx::query_as(
|
||||
r#"
|
||||
SELECT id::text, name, path, parent_id::text, drive_id,
|
||||
SELECT id, name, path, parent_id, drive_id,
|
||||
EXTRACT(EPOCH FROM created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM updated_at)::bigint,
|
||||
EXTRACT(EPOCH FROM tree_modified_at)::bigint,
|
||||
@@ -465,7 +482,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
} else {
|
||||
sqlx::query_as(
|
||||
r#"
|
||||
SELECT id::text, name, path, parent_id::text, drive_id,
|
||||
SELECT id, name, path, parent_id, drive_id,
|
||||
EXTRACT(EPOCH FROM created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM updated_at)::bigint,
|
||||
EXTRACT(EPOCH FROM tree_modified_at)::bigint,
|
||||
@@ -529,7 +546,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
"AND $3::text IS NULL"
|
||||
};
|
||||
let sql = format!(
|
||||
"SELECT id::text, name, path, parent_id::text, drive_id, \
|
||||
"SELECT id, name, path, parent_id, drive_id, \
|
||||
EXTRACT(EPOCH FROM created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM updated_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM tree_modified_at)::bigint, \
|
||||
@@ -567,7 +584,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
include_total: bool,
|
||||
) -> Result<(Vec<Folder>, Option<usize>), DomainError> {
|
||||
let sql = format!(
|
||||
"SELECT fo.id::text, fo.name, fo.path, fo.parent_id::text, \
|
||||
"SELECT fo.id, fo.name, fo.path, fo.parent_id, \
|
||||
fo.drive_id, \
|
||||
EXTRACT(EPOCH FROM fo.created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM fo.updated_at)::bigint, \
|
||||
@@ -629,7 +646,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
UPDATE storage.folders
|
||||
SET name = $1, updated_at = NOW(), updated_by = $3
|
||||
WHERE id = $2::uuid AND NOT is_trashed
|
||||
RETURNING id::text, name, path, parent_id::text, drive_id,
|
||||
RETURNING id, name, path, parent_id, drive_id,
|
||||
EXTRACT(EPOCH FROM created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM updated_at)::bigint,
|
||||
EXTRACT(EPOCH FROM tree_modified_at)::bigint,
|
||||
@@ -690,7 +707,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
updated_at = NOW(),
|
||||
updated_by = $3
|
||||
WHERE f.id = $2::uuid AND NOT f.is_trashed
|
||||
RETURNING f.id::text, f.name, f.path, f.parent_id::text, f.drive_id,
|
||||
RETURNING f.id, f.name, f.path, f.parent_id, f.drive_id,
|
||||
EXTRACT(EPOCH FROM f.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM f.updated_at)::bigint,
|
||||
EXTRACT(EPOCH FROM f.tree_modified_at)::bigint,
|
||||
@@ -989,7 +1006,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
/// Ordered by `fo.path` so callers can iterate in directory order.
|
||||
#[allow(clippy::type_complexity)]
|
||||
async fn list_subtree_folders(&self, folder_id: &str) -> Result<Vec<Folder>, DomainError> {
|
||||
let sql = "SELECT fo.id::text, fo.name, fo.path, fo.parent_id::text, \
|
||||
let sql = "SELECT fo.id, fo.name, fo.path, fo.parent_id, \
|
||||
fo.drive_id, \
|
||||
EXTRACT(EPOCH FROM fo.created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM fo.updated_at)::bigint, \
|
||||
@@ -1056,7 +1073,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
if recursive {
|
||||
// Recursive, no folder scope → ALL folders in caller's readable drives
|
||||
let sql = format!(
|
||||
"SELECT fo.id::text, fo.name, fo.path, fo.parent_id::text, \
|
||||
"SELECT fo.id, fo.name, fo.path, fo.parent_id, \
|
||||
fo.drive_id, \
|
||||
EXTRACT(EPOCH FROM fo.created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM fo.updated_at)::bigint, \
|
||||
@@ -1095,7 +1112,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
// the caller can read (parent_id already establishes the subtree).
|
||||
let sql = if parent_id.is_some() {
|
||||
format!(
|
||||
"SELECT fo.id::text, fo.name, fo.path, fo.parent_id::text, \
|
||||
"SELECT fo.id, fo.name, fo.path, fo.parent_id, \
|
||||
fo.drive_id, \
|
||||
EXTRACT(EPOCH FROM fo.created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM fo.updated_at)::bigint, \
|
||||
@@ -1115,7 +1132,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
_ => "",
|
||||
};
|
||||
format!(
|
||||
"SELECT fo.id::text, fo.name, fo.path, fo.parent_id::text, \
|
||||
"SELECT fo.id, fo.name, fo.path, fo.parent_id, \
|
||||
fo.drive_id, \
|
||||
EXTRACT(EPOCH FROM fo.created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM fo.updated_at)::bigint, \
|
||||
@@ -1188,7 +1205,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
};
|
||||
|
||||
let sql = format!(
|
||||
"SELECT fo.id::text, fo.name, fo.path, fo.parent_id::text, \
|
||||
"SELECT fo.id, fo.name, fo.path, fo.parent_id, \
|
||||
fo.drive_id, \
|
||||
EXTRACT(EPOCH FROM fo.created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM fo.updated_at)::bigint, \
|
||||
@@ -1245,7 +1262,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
let rows: Vec<FolderRow> = if let Some(pid) = parent_id {
|
||||
sqlx::query_as(&format!(
|
||||
r#"
|
||||
SELECT fo.id::text, fo.name, fo.path, fo.parent_id::text, fo.drive_id,
|
||||
SELECT fo.id, fo.name, fo.path, fo.parent_id, fo.drive_id,
|
||||
EXTRACT(EPOCH FROM fo.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fo.updated_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fo.tree_modified_at)::bigint,
|
||||
@@ -1274,7 +1291,7 @@ impl FolderRepository for FolderDbRepository {
|
||||
} else {
|
||||
sqlx::query_as(&format!(
|
||||
r#"
|
||||
SELECT fo.id::text, fo.name, fo.path, fo.parent_id::text, fo.drive_id,
|
||||
SELECT fo.id, fo.name, fo.path, fo.parent_id, fo.drive_id,
|
||||
EXTRACT(EPOCH FROM fo.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fo.updated_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fo.tree_modified_at)::bigint,
|
||||
|
||||
@@ -187,22 +187,48 @@ impl BlobStorageBackend for CachedBlobBackend {
|
||||
};
|
||||
Box::pin(async move {
|
||||
let size = inner.put_blob_from_bytes(&hash, data.clone()).await?;
|
||||
// Also cache locally (best-effort): write bytes to cache path
|
||||
let dest = self_ref.cached_path(&hash);
|
||||
if let Some(parent) = dest.parent() {
|
||||
let _ = fs::create_dir_all(parent).await;
|
||||
}
|
||||
let _ = fs::write(&dest, &data).await;
|
||||
let data_len = data.len() as u64;
|
||||
let mut idx = self_ref.index.lock().await;
|
||||
if let Some(old) = idx.put(hash, CacheEntry { size: data_len }) {
|
||||
self_ref.current_size.fetch_sub(old.size, Ordering::Relaxed);
|
||||
}
|
||||
self_ref.current_size.fetch_add(data_len, Ordering::Relaxed);
|
||||
self_ref.cache_bytes_write_through(hash, &data).await;
|
||||
Ok(size)
|
||||
})
|
||||
}
|
||||
|
||||
// Without this override the trait default would re-route the CDC chunk
|
||||
// write through `put_blob_from_bytes` above, whose inner (synced) call
|
||||
// pays the remote exists-probe per chunk. The local write-through cache
|
||||
// population is kept identical — post-upload readers (thumbnail/EXIF/
|
||||
// face hooks) hit the cache instead of re-fetching from the remote.
|
||||
fn put_blob_from_bytes_unsynced(
|
||||
&self,
|
||||
hash: &str,
|
||||
data: Bytes,
|
||||
) -> Pin<Box<dyn std::future::Future<Output = Result<u64, DomainError>> + Send + '_>> {
|
||||
let inner = self.inner.clone();
|
||||
let hash = hash.to_string();
|
||||
let self_ref = CachedRef {
|
||||
cache_dir: self.cache_dir.clone(),
|
||||
max_cache_bytes: self.max_cache_bytes,
|
||||
index: self.index.clone(),
|
||||
current_size: self.current_size.clone(),
|
||||
inflight: self.inflight.clone(),
|
||||
};
|
||||
Box::pin(async move {
|
||||
let size = inner
|
||||
.put_blob_from_bytes_unsynced(&hash, data.clone())
|
||||
.await?;
|
||||
self_ref.cache_bytes_write_through(hash, &data).await;
|
||||
Ok(size)
|
||||
})
|
||||
}
|
||||
|
||||
// The durability barrier must reach the backend that buffered the
|
||||
// unsynced writes; the local cache copy is disposable and needs none.
|
||||
fn sync_blobs(
|
||||
&self,
|
||||
hashes: &[String],
|
||||
) -> Pin<Box<dyn std::future::Future<Output = Result<(), DomainError>> + Send + '_>> {
|
||||
self.inner.sync_blobs(hashes)
|
||||
}
|
||||
|
||||
fn get_blob_stream(
|
||||
&self,
|
||||
hash: &str,
|
||||
@@ -437,6 +463,24 @@ impl CachedRef {
|
||||
self.cache_dir.join(prefix).join(format!("{hash}.blob"))
|
||||
}
|
||||
|
||||
/// Best-effort write-through cache population shared by both blob-bytes
|
||||
/// PUT paths. Deliberately no eviction sweep here — the byte budget is
|
||||
/// enforced on read-miss inserts (`insert_into_cache_static`), matching
|
||||
/// the historical write-path behavior.
|
||||
async fn cache_bytes_write_through(&self, hash: String, data: &Bytes) {
|
||||
let dest = self.cached_path(&hash);
|
||||
if let Some(parent) = dest.parent() {
|
||||
let _ = fs::create_dir_all(parent).await;
|
||||
}
|
||||
let _ = fs::write(&dest, data).await;
|
||||
let data_len = data.len() as u64;
|
||||
let mut idx = self.index.lock().await;
|
||||
if let Some(old) = idx.put(hash, CacheEntry { size: data_len }) {
|
||||
self.current_size.fetch_sub(old.size, Ordering::Relaxed);
|
||||
}
|
||||
self.current_size.fetch_add(data_len, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// Single-flight wrapper around [`Self::fetch_and_cache_static`]: the
|
||||
/// first caller for a hash becomes the leader and downloads; concurrent
|
||||
/// callers queue on the per-hash gate, then re-check the cache and serve
|
||||
|
||||
@@ -834,8 +834,7 @@ impl ChunkedUploadService {
|
||||
let data_clone = data.clone(); // Bytes::clone is O(1) — just an Arc increment
|
||||
let actual_checksum = tokio::task::spawn_blocking(move || {
|
||||
use md5::{Digest, Md5};
|
||||
let hash = Md5::digest(&data_clone);
|
||||
hash.iter().map(|b| format!("{b:02x}")).collect::<String>()
|
||||
crate::common::fmt::hex_lower(&Md5::digest(&data_clone))
|
||||
})
|
||||
.await
|
||||
.map_err(|e| format!("MD5 checksum task failed: {e}"))?;
|
||||
|
||||
@@ -1747,18 +1747,24 @@ impl DedupService {
|
||||
/// remote object stores where overlapping fetches hide per-chunk latency).
|
||||
/// Shared by [`Self::read_blob_stream`] and [`Self::read_blob_bytes`] so both
|
||||
/// build the chunk stream identically from a manifest's `chunk_hashes`.
|
||||
/// Takes the shared manifest `Arc` and iterates its hashes by index —
|
||||
/// the old `Vec<String>` signature forced every read to deep-clone the
|
||||
/// whole hash list out of the cached manifest before the first byte
|
||||
/// (N ~64-B String allocs per read of an N-chunk file); the per-chunk
|
||||
/// `Arc` bump here is a single atomic increment.
|
||||
fn stream_chunks(
|
||||
&self,
|
||||
chunk_hashes: Vec<String>,
|
||||
manifest: Arc<ChunkManifest>,
|
||||
) -> Pin<Box<dyn Stream<Item = Result<Bytes, std::io::Error>> + Send>> {
|
||||
let prefetch = self.backend.read_prefetch().max(1);
|
||||
let backend = self.backend.clone();
|
||||
let chunk_stream = stream::iter(chunk_hashes)
|
||||
.map(move |chunk_hash| {
|
||||
let chunk_stream = stream::iter(0..manifest.chunk_hashes.len())
|
||||
.map(move |i| {
|
||||
let backend = backend.clone();
|
||||
let manifest = manifest.clone();
|
||||
async move {
|
||||
backend
|
||||
.get_blob_stream(&chunk_hash)
|
||||
.get_blob_stream(&manifest.chunk_hashes[i])
|
||||
.await
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))
|
||||
}
|
||||
@@ -1771,31 +1777,56 @@ impl DedupService {
|
||||
/// Cached manifest fetch for the read path (see the `manifest_cache`
|
||||
/// field docs). `None` = legacy whole-file blob — never cached, so a
|
||||
/// background rechunk that creates a manifest is honoured immediately.
|
||||
///
|
||||
/// Misses are single-flighted through `try_get_with`: K concurrent cold
|
||||
/// readers of one newly-hot file (e.g. parallel Range probes on a big
|
||||
/// video) coalesce onto ONE manifest SELECT instead of K. The
|
||||
/// positive-only contract is preserved by routing "no manifest row" and
|
||||
/// DB failures through the loader's error channel, which moka never
|
||||
/// caches. The zero-alloc `get` fast path stays in front so warm reads
|
||||
/// don't pay the owned-key clone `try_get_with` requires.
|
||||
async fn manifest_cached(&self, hash: &str) -> Result<Option<Arc<ChunkManifest>>, DomainError> {
|
||||
if let Some(m) = self.manifest_cache.get(hash).await {
|
||||
return Ok(Some(m));
|
||||
}
|
||||
let row = sqlx::query_as::<_, (Vec<String>, Vec<i64>, i64)>(
|
||||
"SELECT chunk_hashes, chunk_sizes, total_size
|
||||
FROM storage.chunk_manifests WHERE file_hash = $1",
|
||||
)
|
||||
.bind(hash)
|
||||
.fetch_optional(self.pool.as_ref())
|
||||
.await
|
||||
.map_err(|e| DomainError::internal_error("Dedup", format!("Manifest lookup: {}", e)))?;
|
||||
match row {
|
||||
Some((chunk_hashes, chunk_sizes, total_size)) => {
|
||||
let m = Arc::new(ChunkManifest {
|
||||
chunk_hashes,
|
||||
chunk_sizes,
|
||||
total_size,
|
||||
});
|
||||
self.manifest_cache
|
||||
.insert(hash.to_string(), m.clone())
|
||||
.await;
|
||||
Ok(Some(m))
|
||||
}
|
||||
None => Ok(None),
|
||||
|
||||
enum MissKind {
|
||||
Legacy,
|
||||
Db(String),
|
||||
}
|
||||
|
||||
let pool = self.pool.clone();
|
||||
let query_hash = hash.to_string();
|
||||
let result = self
|
||||
.manifest_cache
|
||||
.try_get_with(hash.to_string(), async move {
|
||||
let row = sqlx::query_as::<_, (Vec<String>, Vec<i64>, i64)>(
|
||||
"SELECT chunk_hashes, chunk_sizes, total_size
|
||||
FROM storage.chunk_manifests WHERE file_hash = $1",
|
||||
)
|
||||
.bind(&query_hash)
|
||||
.fetch_optional(pool.as_ref())
|
||||
.await
|
||||
.map_err(|e| MissKind::Db(e.to_string()))?;
|
||||
match row {
|
||||
Some((chunk_hashes, chunk_sizes, total_size)) => Ok(Arc::new(ChunkManifest {
|
||||
chunk_hashes,
|
||||
chunk_sizes,
|
||||
total_size,
|
||||
})),
|
||||
None => Err(MissKind::Legacy),
|
||||
}
|
||||
})
|
||||
.await;
|
||||
match result {
|
||||
Ok(m) => Ok(Some(m)),
|
||||
Err(miss) => match &*miss {
|
||||
MissKind::Legacy => Ok(None),
|
||||
MissKind::Db(msg) => Err(DomainError::internal_error(
|
||||
"Dedup",
|
||||
format!("Manifest lookup: {}", msg),
|
||||
)),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1810,7 +1841,7 @@ impl DedupService {
|
||||
) -> Result<Pin<Box<dyn Stream<Item = Result<Bytes, std::io::Error>> + Send>>, DomainError>
|
||||
{
|
||||
match self.manifest_cached(hash).await? {
|
||||
Some(m) => Ok(self.stream_chunks(m.chunk_hashes.clone())),
|
||||
Some(m) => Ok(self.stream_chunks(m)),
|
||||
// Legacy whole-file blob
|
||||
None => self.backend.get_blob_stream(hash).await,
|
||||
}
|
||||
@@ -1829,10 +1860,10 @@ impl DedupService {
|
||||
/// every full-blob read (e.g. 2N queries for an N-image gallery cold load).
|
||||
pub async fn read_blob_bytes(&self, hash: &str) -> Result<Bytes, DomainError> {
|
||||
let (mut stream, expected_size) = match self.manifest_cached(hash).await? {
|
||||
Some(m) => (
|
||||
self.stream_chunks(m.chunk_hashes.clone()),
|
||||
m.total_size.max(0) as usize,
|
||||
),
|
||||
Some(m) => {
|
||||
let expected = m.total_size.max(0) as usize;
|
||||
(self.stream_chunks(m), expected)
|
||||
}
|
||||
None => {
|
||||
// Legacy whole-file blob: size + stream straight from the backend.
|
||||
let size = self.backend.blob_size(hash).await? as usize;
|
||||
@@ -1863,16 +1894,17 @@ impl DedupService {
|
||||
) -> Result<Pin<Box<dyn Stream<Item = Result<Bytes, std::io::Error>> + Send>>, DomainError>
|
||||
{
|
||||
if let Some(m) = self.manifest_cached(hash).await? {
|
||||
let (chunk_hashes, chunk_sizes, total_size) =
|
||||
(&m.chunk_hashes, &m.chunk_sizes, m.total_size);
|
||||
let end = end.unwrap_or(total_size as u64);
|
||||
let end = end.unwrap_or(m.total_size as u64);
|
||||
|
||||
// Calculate which chunks overlap [start, end)
|
||||
// Calculate which chunks overlap [start, end). Chunks are
|
||||
// addressed by manifest INDEX (the hash is read through the
|
||||
// shared `Arc` at fetch time) — a `bytes=0-` probe of an
|
||||
// N-chunk video used to clone all N hash Strings here.
|
||||
let mut offset: u64 = 0;
|
||||
// (chunk_hash, range_start_within_chunk, range_end_within_chunk)
|
||||
let mut selected: Vec<(String, u64, Option<u64>)> = Vec::new();
|
||||
// (chunk_index, range_start_within_chunk, range_end_within_chunk)
|
||||
let mut selected: Vec<(usize, u64, Option<u64>)> = Vec::new();
|
||||
|
||||
for (i, &chunk_size) in chunk_sizes.iter().enumerate() {
|
||||
for (i, &chunk_size) in m.chunk_sizes.iter().enumerate() {
|
||||
let chunk_size = chunk_size as u64;
|
||||
let chunk_end = offset + chunk_size;
|
||||
|
||||
@@ -1883,7 +1915,7 @@ impl DedupService {
|
||||
} else {
|
||||
None
|
||||
};
|
||||
selected.push((chunk_hashes[i].clone(), range_start, range_end));
|
||||
selected.push((i, range_start, range_end));
|
||||
}
|
||||
|
||||
offset += chunk_size;
|
||||
@@ -1897,11 +1929,16 @@ impl DedupService {
|
||||
let prefetch = self.backend.read_prefetch().max(1);
|
||||
let backend = self.backend.clone();
|
||||
let chunk_stream = stream::iter(selected)
|
||||
.map(move |(chunk_hash, range_start, range_end)| {
|
||||
.map(move |(i, range_start, range_end)| {
|
||||
let backend = backend.clone();
|
||||
let manifest = m.clone();
|
||||
async move {
|
||||
backend
|
||||
.get_blob_range_stream(&chunk_hash, range_start, range_end)
|
||||
.get_blob_range_stream(
|
||||
&manifest.chunk_hashes[i],
|
||||
range_start,
|
||||
range_end,
|
||||
)
|
||||
.await
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))
|
||||
}
|
||||
|
||||
@@ -128,18 +128,41 @@ async fn fsync_paths_parallel(paths: Vec<PathBuf>, strict: bool) -> Result<(), D
|
||||
/// (fsync now vs. deferred batch sync), or `None` when the blob already
|
||||
/// existed (idempotent skip — content-addressed, so identical by definition).
|
||||
async fn write_blob_bytes(blob_path: &Path, data: &Bytes) -> Result<Option<File>, DomainError> {
|
||||
if fs::try_exists(blob_path).await.unwrap_or(false) {
|
||||
return Ok(None);
|
||||
}
|
||||
let mut file = fs::File::create(blob_path).await.map_err(|e| {
|
||||
DomainError::internal_error("Blob", format!("Failed to create blob file: {}", e))
|
||||
})?;
|
||||
// One atomic O_CREAT|O_EXCL open replaces the old stat-then-create pair:
|
||||
// `AlreadyExists` IS the idempotent skip (content-addressed names mean an
|
||||
// existing file has identical content), saving a syscall + a blocking-pool
|
||||
// dispatch on every new chunk of every upload.
|
||||
let mut file = match fs::File::options()
|
||||
.write(true)
|
||||
.create_new(true)
|
||||
.open(blob_path)
|
||||
.await
|
||||
{
|
||||
Ok(f) => f,
|
||||
Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => return Ok(None),
|
||||
Err(e) => {
|
||||
return Err(DomainError::internal_error(
|
||||
"Blob",
|
||||
format!("Failed to create blob file: {}", e),
|
||||
));
|
||||
}
|
||||
};
|
||||
file.write_all(data).await.map_err(|e| {
|
||||
DomainError::internal_error("Blob", format!("Failed to write blob from bytes: {}", e))
|
||||
})?;
|
||||
Ok(Some(file))
|
||||
}
|
||||
|
||||
/// Bench-only public wrapper (feature = "bench") over the private chunk
|
||||
/// writer so `examples/bench_storage_micro.rs` can A/B the open strategy.
|
||||
#[cfg(feature = "bench")]
|
||||
pub async fn write_blob_bytes_for_bench(
|
||||
blob_path: &Path,
|
||||
data: &Bytes,
|
||||
) -> Result<Option<File>, DomainError> {
|
||||
write_blob_bytes(blob_path, data).await
|
||||
}
|
||||
|
||||
/// Compile-time lookup table for the 256 two-digit lowercase hex prefixes ("00"…"ff").
|
||||
static HEX_PREFIXES: [&str; 256] = [
|
||||
"00", "01", "02", "03", "04", "05", "06", "07", "08", "09", "0a", "0b", "0c", "0d", "0e", "0f",
|
||||
|
||||
@@ -116,6 +116,15 @@ const CASCADE_GRANT_CACHE_CAPACITY: u64 = 100_000;
|
||||
/// invalidation tree". Short enough that any such change takes effect in <1 min.
|
||||
const CASCADE_GRANT_CACHE_TTL: Duration = Duration::from_secs(30);
|
||||
|
||||
/// `file_parent_cache` bound/TTL: `file_id → Option<folder_id>` point rows
|
||||
/// (~50 B each) resolved on the file-cascade path so an N-file album pays
|
||||
/// ONE folder-cascade query instead of N (ROUND9). Parentage changes only
|
||||
/// on move — an indirect path the cascade cache already self-heals via TTL,
|
||||
/// so the same 30 s window applies (grant writes don't alter parentage and
|
||||
/// need no flush here).
|
||||
const FILE_PARENT_CACHE_CAPACITY: u64 = 100_000;
|
||||
const FILE_PARENT_CACHE_TTL: Duration = Duration::from_secs(30);
|
||||
|
||||
pub struct PgAclEngine {
|
||||
pool: Arc<PgPool>,
|
||||
folder_repo: Arc<FolderDbRepository>,
|
||||
@@ -202,6 +211,13 @@ pub struct PgAclEngine {
|
||||
/// only positively-or-negatively for at most the TTL. A revoke via
|
||||
/// `clear_role` flushes immediately; anything missed self-heals in ≤30 s.
|
||||
cascade_grant_cache: Cache<(Subject, Resource, Permission), bool>,
|
||||
/// `file_id → Option<parent folder_id>` memo for the file-cascade
|
||||
/// decomposition (see `cascade_grant_cached`): resolving the parent lets
|
||||
/// a whole folder's files share ONE folder-cascade decision, so a shared
|
||||
/// album's first view runs one ltree query instead of one per file.
|
||||
/// Grant writes don't affect parentage — only the TTL applies (moves are
|
||||
/// an indirect path, same self-heal contract as `cascade_grant_cache`).
|
||||
file_parent_cache: Cache<Uuid, Option<Uuid>>,
|
||||
}
|
||||
|
||||
impl PgAclEngine {
|
||||
@@ -243,6 +259,10 @@ impl PgAclEngine {
|
||||
.max_capacity(CASCADE_GRANT_CACHE_CAPACITY)
|
||||
.time_to_live(CASCADE_GRANT_CACHE_TTL)
|
||||
.build(),
|
||||
file_parent_cache: Cache::builder()
|
||||
.max_capacity(FILE_PARENT_CACHE_CAPACITY)
|
||||
.time_to_live(FILE_PARENT_CACHE_TTL)
|
||||
.build(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -317,6 +337,10 @@ impl PgAclEngine {
|
||||
.max_capacity(1)
|
||||
.time_to_live(Duration::from_secs(1))
|
||||
.build(),
|
||||
file_parent_cache: Cache::builder()
|
||||
.max_capacity(1)
|
||||
.time_to_live(Duration::from_secs(1))
|
||||
.build(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -718,11 +742,11 @@ impl PgAclEngine {
|
||||
Ok(exists.is_some())
|
||||
}
|
||||
|
||||
/// Cascading check for files: either a direct file grant OR a grant on
|
||||
/// any ancestor folder of the file's containing folder. See
|
||||
/// `folder_cascade_grant_exists` for the meaning of `subject_types` /
|
||||
/// `subject_ids` and the D-Prep role-array migration.
|
||||
async fn file_cascade_grant_exists(
|
||||
/// Direct file grant only — the first branch of the historical file
|
||||
/// cascade UNION, split out so `cascade_grant_cached` can amortize the
|
||||
/// ancestor-folder branch per FOLDER (see the `Resource::File` arm).
|
||||
/// A plain indexed `role_grants` point lookup, no ltree join.
|
||||
async fn file_direct_grant_exists(
|
||||
&self,
|
||||
subject_types: &[&str],
|
||||
subject_ids: &[Uuid],
|
||||
@@ -735,30 +759,12 @@ impl PgAclEngine {
|
||||
let exists: Option<i32> = sqlx::query_scalar(
|
||||
r#"
|
||||
SELECT 1
|
||||
FROM (
|
||||
-- direct file grant
|
||||
SELECT 1
|
||||
FROM storage.role_grants
|
||||
WHERE subject_type = ANY($1)
|
||||
AND subject_id = ANY($2)
|
||||
AND role = ANY($3::storage.grant_role[])
|
||||
AND resource_type = 'file' AND resource_id = $4
|
||||
AND (expires_at IS NULL OR expires_at > NOW())
|
||||
UNION ALL
|
||||
-- cascading from any ancestor folder of the file's containing folder
|
||||
SELECT 1
|
||||
FROM storage.role_grants g
|
||||
JOIN storage.folders gf ON gf.id = g.resource_id
|
||||
JOIN storage.files target_f ON target_f.id = $4
|
||||
WHERE g.subject_type = ANY($1)
|
||||
AND g.subject_id = ANY($2)
|
||||
AND g.role = ANY($3::storage.grant_role[])
|
||||
AND g.resource_type = 'folder'
|
||||
AND (g.expires_at IS NULL OR g.expires_at > NOW())
|
||||
AND target_f.folder_id IS NOT NULL
|
||||
AND gf.lpath @> (SELECT lpath FROM storage.folders
|
||||
WHERE id = target_f.folder_id)
|
||||
) any_match
|
||||
FROM storage.role_grants
|
||||
WHERE subject_type = ANY($1)
|
||||
AND subject_id = ANY($2)
|
||||
AND role = ANY($3::storage.grant_role[])
|
||||
AND resource_type = 'file' AND resource_id = $4
|
||||
AND (expires_at IS NULL OR expires_at > NOW())
|
||||
LIMIT 1
|
||||
"#,
|
||||
)
|
||||
@@ -768,11 +774,36 @@ impl PgAclEngine {
|
||||
.bind(file_id)
|
||||
.fetch_optional(self.pool.as_ref())
|
||||
.await
|
||||
.map_err(|e| DomainError::internal_error("PgAcl", format!("file cascade: {e}")))?;
|
||||
.map_err(|e| DomainError::internal_error("PgAcl", format!("file direct grant: {e}")))?;
|
||||
|
||||
Ok(exists.is_some())
|
||||
}
|
||||
|
||||
/// Memoised `file_id → Option<parent folder_id>` point read backing the
|
||||
/// file-cascade decomposition. `None` covers both a missing row and a
|
||||
/// NULL `folder_id` — in either case only the direct-file-grant branch
|
||||
/// can match (mirroring the historical UNION's `folder_id IS NOT NULL`
|
||||
/// guard).
|
||||
async fn file_parent_folder_cached(
|
||||
&self,
|
||||
file_id: Uuid,
|
||||
counters: &QueryCounters,
|
||||
) -> Result<Option<Uuid>, DomainError> {
|
||||
if let Some(parent) = self.file_parent_cache.get(&file_id).await {
|
||||
return Ok(parent);
|
||||
}
|
||||
counters.sql_queries.fetch_add(1, Ordering::Relaxed);
|
||||
let parent: Option<Option<Uuid>> =
|
||||
sqlx::query_scalar("SELECT folder_id FROM storage.files WHERE id = $1")
|
||||
.bind(file_id)
|
||||
.fetch_optional(self.pool.as_ref())
|
||||
.await
|
||||
.map_err(|e| DomainError::internal_error("PgAcl", format!("file parent: {e}")))?;
|
||||
let parent = parent.flatten();
|
||||
self.file_parent_cache.insert(file_id, parent).await;
|
||||
Ok(parent)
|
||||
}
|
||||
|
||||
/// Cache-aware wrapper over the File/Folder grant cascade. Serves the
|
||||
/// memoised `(subject, resource, permission)` decision when warm; on a
|
||||
/// miss it expands the subject set (itself cached) and runs the matching
|
||||
@@ -780,10 +811,23 @@ impl PgAclEngine {
|
||||
/// precheck fails, so it never caches a decision a drive grant would have
|
||||
/// satisfied — a later drive grant short-circuits above this cache.
|
||||
///
|
||||
/// **File decomposition (ROUND9).** The historical file query was one
|
||||
/// UNION: `direct file grant ∨ grant on any ancestor of the parent
|
||||
/// folder` — one ltree join per file, so a shared N-photo album's FIRST
|
||||
/// view ran N near-identical ancestor queries (round 8 memoised only the
|
||||
/// per-file result, covering revalidation). The arm now resolves the
|
||||
/// file's parent (memoised point read) and recurses into the FOLDER arm
|
||||
/// for the ancestor half — one ltree query per folder, shared by every
|
||||
/// sibling — falling back to the direct-file-grant lookup only when the
|
||||
/// folder half denies. The decomposition is exactly the UNION split in
|
||||
/// two: no decision changes, including the parentless edge (the UNION's
|
||||
/// `folder_id IS NOT NULL` guard ≡ the direct-only fallback).
|
||||
///
|
||||
/// The result is a pure function of the subject's group expansion + the
|
||||
/// resource's grants + folder ancestry; `invalidate_cascade_grant_cache_all`
|
||||
/// (on File/Folder grant writes) and the 30 s TTL (indirect changes) keep
|
||||
/// it fresh. See the `cascade_grant_cache` field doc.
|
||||
/// (on File/Folder grant writes — it holds file AND folder decisions in
|
||||
/// the same map) and the 30 s TTL (indirect changes, incl. moves for the
|
||||
/// parent memo) keep it fresh. See the `cascade_grant_cache` field doc.
|
||||
async fn cascade_grant_cached(
|
||||
&self,
|
||||
subject: Subject,
|
||||
@@ -799,9 +843,10 @@ impl PgAclEngine {
|
||||
counters.cache_hit.fetch_add(1, Ordering::Relaxed);
|
||||
return Ok(allowed);
|
||||
}
|
||||
let (subject_types, subject_ids) = self.subject_match_set(subject, counters).await?;
|
||||
let allowed = match resource {
|
||||
Resource::Folder(id) => {
|
||||
let (subject_types, subject_ids) =
|
||||
self.subject_match_set(subject, counters).await?;
|
||||
self.folder_cascade_grant_exists(
|
||||
&subject_types,
|
||||
&subject_ids,
|
||||
@@ -812,14 +857,34 @@ impl PgAclEngine {
|
||||
.await?
|
||||
}
|
||||
Resource::File(id) => {
|
||||
self.file_cascade_grant_exists(
|
||||
&subject_types,
|
||||
&subject_ids,
|
||||
permission,
|
||||
id,
|
||||
counters,
|
||||
)
|
||||
.await?
|
||||
// Ancestor half first — amortized to one query per FOLDER
|
||||
// via the recursive Folder arm (its own cache entry).
|
||||
let folder_allowed = match self.file_parent_folder_cached(id, counters).await? {
|
||||
Some(parent) => {
|
||||
Box::pin(self.cascade_grant_cached(
|
||||
subject,
|
||||
Resource::Folder(parent),
|
||||
permission,
|
||||
counters,
|
||||
))
|
||||
.await?
|
||||
}
|
||||
None => false,
|
||||
};
|
||||
if folder_allowed {
|
||||
true
|
||||
} else {
|
||||
let (subject_types, subject_ids) =
|
||||
self.subject_match_set(subject, counters).await?;
|
||||
self.file_direct_grant_exists(
|
||||
&subject_types,
|
||||
&subject_ids,
|
||||
permission,
|
||||
id,
|
||||
counters,
|
||||
)
|
||||
.await?
|
||||
}
|
||||
}
|
||||
// Only File/Folder reach this helper (see `check_inner`).
|
||||
_ => return Ok(false),
|
||||
|
||||
@@ -159,6 +159,43 @@ impl BlobStorageBackend for RetryBlobBackend {
|
||||
})
|
||||
}
|
||||
|
||||
// Without this override the trait default would re-route the CDC chunk
|
||||
// write through `put_blob_from_bytes` above — reinstating the remote
|
||||
// backend's exists-probe (HEAD/get_properties) per chunk that the
|
||||
// `_unsynced` fast path exists to skip.
|
||||
fn put_blob_from_bytes_unsynced(
|
||||
&self,
|
||||
hash: &str,
|
||||
data: Bytes,
|
||||
) -> Pin<Box<dyn std::future::Future<Output = Result<u64, DomainError>> + Send + '_>> {
|
||||
let inner = self.inner.clone();
|
||||
let policy = self.policy.clone();
|
||||
let hash = hash.to_string();
|
||||
Box::pin(async move {
|
||||
retry_async(
|
||||
&policy,
|
||||
&format!("put_blob_from_bytes_unsynced({hash})"),
|
||||
|| {
|
||||
let inner = inner.clone();
|
||||
let hash = hash.clone();
|
||||
let data = data.clone();
|
||||
async move { inner.put_blob_from_bytes_unsynced(&hash, data).await }
|
||||
},
|
||||
)
|
||||
.await
|
||||
})
|
||||
}
|
||||
|
||||
// Forwarded WITHOUT retry wrapping: a failed fsync must surface, not be
|
||||
// re-issued — after an fsync error the kernel may have dropped the dirty
|
||||
// pages, so a retried fsync can report success for data that was lost.
|
||||
fn sync_blobs(
|
||||
&self,
|
||||
hashes: &[String],
|
||||
) -> Pin<Box<dyn std::future::Future<Output = Result<(), DomainError>> + Send + '_>> {
|
||||
self.inner.sync_blobs(hashes)
|
||||
}
|
||||
|
||||
fn get_blob_stream(
|
||||
&self,
|
||||
hash: &str,
|
||||
|
||||
@@ -198,7 +198,7 @@ pub async fn list_favorites_resources(
|
||||
// Path is only shown to the owner; non-owners see ""
|
||||
// to avoid leaking another user's folder hierarchy.
|
||||
let path = if row.is_owner {
|
||||
row.path.clone().unwrap_or_default()
|
||||
row.path.unwrap_or_default()
|
||||
} else {
|
||||
String::new()
|
||||
};
|
||||
@@ -208,7 +208,7 @@ pub async fn list_favorites_resources(
|
||||
let dto = FolderDto {
|
||||
etag: resource_id.clone(),
|
||||
id: resource_id,
|
||||
name: row.name.clone(),
|
||||
name: row.name,
|
||||
path,
|
||||
parent_id: row.parent_id.map(|u| u.to_string()),
|
||||
drive_id: row.drive_id,
|
||||
@@ -239,26 +239,30 @@ pub async fn list_favorites_resources(
|
||||
// file. `blob_hash` is `None` only for
|
||||
// folder rows, which take the other branch.
|
||||
let modified_at_u = row.modified_at.timestamp() as u64;
|
||||
let content_hash = row.blob_hash.clone().unwrap_or_default();
|
||||
let content_hash = row.blob_hash.unwrap_or_default();
|
||||
let etag = if content_hash.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
File::compute_etag(&content_hash, modified_at_u)
|
||||
};
|
||||
// Name-derived display classes borrow `row.name`;
|
||||
// compute them before the name moves into the DTO.
|
||||
let icon_class = intern_display(icon_class_for(&row.name, mime));
|
||||
let icon_special_class =
|
||||
intern_display(icon_special_class_for(&row.name, mime));
|
||||
let category = intern_display(category_for(&row.name, mime));
|
||||
let dto = FileDto {
|
||||
id: row.resource_id.to_string(),
|
||||
name: row.name.clone(),
|
||||
name: row.name,
|
||||
path,
|
||||
size: size_bytes,
|
||||
mime_type: intern_mime(mime),
|
||||
folder_id: row.parent_id.map(|u| u.to_string()),
|
||||
created_at: row.resource_created_at.timestamp() as u64,
|
||||
modified_at: modified_at_u,
|
||||
icon_class: intern_display(icon_class_for(&row.name, mime)),
|
||||
icon_special_class: intern_display(icon_special_class_for(
|
||||
&row.name, mime,
|
||||
)),
|
||||
category: intern_display(category_for(&row.name, mime)),
|
||||
icon_class,
|
||||
icon_special_class,
|
||||
category,
|
||||
size_formatted: format_file_size(size_bytes),
|
||||
sort_date: None,
|
||||
content_hash,
|
||||
|
||||
@@ -510,7 +510,7 @@ pub async fn list_folder_resources(
|
||||
// listing's `etag` byte-equals what a
|
||||
// conditional request would compare against.
|
||||
let modified_at_u = row.modified_at.timestamp() as u64;
|
||||
let content_hash = row.blob_hash.clone().unwrap_or_default();
|
||||
let content_hash = row.blob_hash.unwrap_or_default();
|
||||
let etag = if content_hash.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
|
||||
@@ -214,7 +214,7 @@ pub async fn list_recent_resources(
|
||||
// Path is only shown to the owner; non-owners see ""
|
||||
// to avoid leaking another user's folder hierarchy.
|
||||
let path = if row.is_owner {
|
||||
row.path.clone().unwrap_or_default()
|
||||
row.path.unwrap_or_default()
|
||||
} else {
|
||||
String::new()
|
||||
};
|
||||
@@ -224,7 +224,7 @@ pub async fn list_recent_resources(
|
||||
let dto = FolderDto {
|
||||
etag: resource_id.clone(),
|
||||
id: resource_id,
|
||||
name: row.name.clone(),
|
||||
name: row.name,
|
||||
path,
|
||||
parent_id: row.parent_id.map(|u| u.to_string()),
|
||||
drive_id: row.drive_id,
|
||||
@@ -253,26 +253,30 @@ pub async fn list_recent_resources(
|
||||
// listing matches GET/HEAD/PROPFIND byte-for-byte
|
||||
// for the same file.
|
||||
let modified_at_u = row.modified_at.timestamp() as u64;
|
||||
let content_hash = row.blob_hash.clone().unwrap_or_default();
|
||||
let content_hash = row.blob_hash.unwrap_or_default();
|
||||
let etag = if content_hash.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
File::compute_etag(&content_hash, modified_at_u)
|
||||
};
|
||||
// Name-derived display classes borrow `row.name`;
|
||||
// compute them before the name moves into the DTO.
|
||||
let icon_class = intern_display(icon_class_for(&row.name, mime));
|
||||
let icon_special_class =
|
||||
intern_display(icon_special_class_for(&row.name, mime));
|
||||
let category = intern_display(category_for(&row.name, mime));
|
||||
let dto = FileDto {
|
||||
id: row.resource_id.to_string(),
|
||||
name: row.name.clone(),
|
||||
name: row.name,
|
||||
path,
|
||||
size: size_bytes,
|
||||
mime_type: intern_mime(mime),
|
||||
folder_id: row.parent_id.map(|u| u.to_string()),
|
||||
created_at: row.resource_created_at.timestamp() as u64,
|
||||
modified_at: modified_at_u,
|
||||
icon_class: intern_display(icon_class_for(&row.name, mime)),
|
||||
icon_special_class: intern_display(icon_special_class_for(
|
||||
&row.name, mime,
|
||||
)),
|
||||
category: intern_display(category_for(&row.name, mime)),
|
||||
icon_class,
|
||||
icon_special_class,
|
||||
category,
|
||||
size_formatted: format_file_size(size_bytes),
|
||||
sort_date: None,
|
||||
content_hash,
|
||||
|
||||
@@ -28,12 +28,16 @@ use crate::interfaces::middleware::auth::CurrentUser;
|
||||
/// default drive root, so no per-request authorization decision is being
|
||||
/// skipped. The drive-marker branch keeps its `get_folder_with_perms`
|
||||
/// check on every request.
|
||||
static NC_CHROOT_CACHE: LazyLock<moka::sync::Cache<uuid::Uuid, FolderDto>> = LazyLock::new(|| {
|
||||
moka::sync::Cache::builder()
|
||||
.max_capacity(100_000)
|
||||
.time_to_live(Duration::from_secs(30))
|
||||
.build()
|
||||
});
|
||||
// `Arc<FolderDto>` values: a hit hands back a refcount bump instead of a
|
||||
// deep clone of the DTO's ~5 owned Strings (moka's `get` clones `V`), and
|
||||
// the same `Arc` then rides inside `NcSession` for the whole request.
|
||||
static NC_CHROOT_CACHE: LazyLock<moka::sync::Cache<uuid::Uuid, Arc<FolderDto>>> =
|
||||
LazyLock::new(|| {
|
||||
moka::sync::Cache::builder()
|
||||
.max_capacity(100_000)
|
||||
.time_to_live(Duration::from_secs(30))
|
||||
.build()
|
||||
});
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum NextcloudAuthError {
|
||||
@@ -184,13 +188,19 @@ pub async fn basic_auth_middleware(
|
||||
// request would appear in the logs with `user_id=-`,
|
||||
// making it harder to correlate WebDAV / OCS activity to
|
||||
// a specific principal.
|
||||
tracing::Span::current().record("user_id", user_id.to_string());
|
||||
let current_user = CurrentUser {
|
||||
// `field::display` renders lazily into the subscriber's buffer —
|
||||
// no per-request `to_string` (mirrors the JWT path since ROUND5).
|
||||
tracing::Span::current().record("user_id", tracing::field::display(user_id));
|
||||
// One shared identity: the same `Arc` serves the
|
||||
// `Arc<CurrentUser>` extension AND `NcSession.user` (the old
|
||||
// code built the struct, cloned it for the extension, then
|
||||
// moved the original — 2-3 String allocs per request).
|
||||
let current_user = Arc::new(CurrentUser {
|
||||
id: user_id,
|
||||
username: uname,
|
||||
email,
|
||||
role,
|
||||
};
|
||||
});
|
||||
|
||||
// ── Resolve chroot from the Basic Auth drive marker ─────
|
||||
// No marker → caller's default personal drive's root folder
|
||||
@@ -226,9 +236,10 @@ pub async fn basic_auth_middleware(
|
||||
.folder_service
|
||||
.get_folder(&root_id.to_string())
|
||||
.await
|
||||
.ok();
|
||||
.ok()
|
||||
.map(Arc::new);
|
||||
if let Some(f) = &fetched {
|
||||
NC_CHROOT_CACHE.insert(root_id, f.clone());
|
||||
NC_CHROOT_CACHE.insert(root_id, Arc::clone(f));
|
||||
}
|
||||
fetched
|
||||
}
|
||||
@@ -242,7 +253,8 @@ pub async fn basic_auth_middleware(
|
||||
.folder_service
|
||||
.get_folder_with_perms(folder_id, current_user.id)
|
||||
.await
|
||||
.ok(),
|
||||
.ok()
|
||||
.map(Arc::new),
|
||||
};
|
||||
if chroot.is_none() {
|
||||
tracing::warn!(
|
||||
@@ -253,25 +265,20 @@ pub async fn basic_auth_middleware(
|
||||
return Err(NextcloudAuthError::Unauthorized);
|
||||
}
|
||||
|
||||
request
|
||||
.extensions_mut()
|
||||
.insert(Arc::new(current_user.clone()));
|
||||
// Record from the local before it moves into the session —
|
||||
// the old code re-read the just-inserted extension and paid a
|
||||
// `to_string` for the span value.
|
||||
if let Some(c) = &chroot {
|
||||
tracing::Span::current().record("chroot_id", tracing::field::display(&c.id));
|
||||
}
|
||||
request.extensions_mut().insert(Arc::clone(¤t_user));
|
||||
request.extensions_mut().insert(Arc::new(
|
||||
crate::interfaces::nextcloud::session::NcSession {
|
||||
user: current_user,
|
||||
raw_username: raw_username.clone(),
|
||||
raw_username,
|
||||
chroot,
|
||||
},
|
||||
));
|
||||
tracing::Span::current().record(
|
||||
"chroot_id",
|
||||
request
|
||||
.extensions()
|
||||
.get::<Arc<crate::interfaces::nextcloud::session::NcSession>>()
|
||||
.and_then(|s| s.chroot.as_ref())
|
||||
.map(|c| c.id.to_string())
|
||||
.unwrap_or_default(),
|
||||
);
|
||||
Ok(next.run(request).await)
|
||||
}
|
||||
Err(_) => {
|
||||
|
||||
@@ -35,20 +35,51 @@ fn ocs_err(statuscode: u16, message: &str) -> serde_json::Value {
|
||||
}
|
||||
|
||||
pub async fn handle_capabilities_v1(State(state): State<Arc<AppState>>) -> Response {
|
||||
let payload = capabilities_payload(&state, 1);
|
||||
tracing::info!("[NC] capabilities v1 requested, returning payload");
|
||||
Json(payload).into_response()
|
||||
capabilities_response(&state, 1)
|
||||
}
|
||||
|
||||
pub async fn handle_capabilities_v2(State(state): State<Arc<AppState>>) -> Response {
|
||||
let payload = capabilities_payload(&state, 2);
|
||||
tracing::info!("[NC] capabilities v2 requested, returning payload");
|
||||
Json(payload).into_response()
|
||||
capabilities_response(&state, 2)
|
||||
}
|
||||
|
||||
/// Pre-serialized capabilities bodies, `[v1, v2]`. The payload is
|
||||
/// process-invariant (pure config: base URL + emulated NC version), yet
|
||||
/// every desktop/mobile client polls it periodically — the old handler
|
||||
/// re-built the ~40-node `json!` tree, re-read `OXICLOUD_BASE_URL` from
|
||||
/// the environment and re-serialized on every poll. Now that work runs
|
||||
/// once; a poll is a `Bytes` refcount bump.
|
||||
static CAPABILITIES_BODIES: std::sync::OnceLock<[bytes::Bytes; 2]> = std::sync::OnceLock::new();
|
||||
|
||||
fn capabilities_response(state: &AppState, ocs_version: u8) -> Response {
|
||||
let bodies = CAPABILITIES_BODIES.get_or_init(|| {
|
||||
let base_url = state.core.config.base_url();
|
||||
let emulated = state.core.config.nextcloud.emulated_version;
|
||||
let version_string = state.core.config.nextcloud.version_string();
|
||||
[1u8, 2u8].map(|v| {
|
||||
bytes::Bytes::from(
|
||||
serde_json::to_vec(&capabilities_payload(
|
||||
&base_url,
|
||||
emulated,
|
||||
&version_string,
|
||||
v,
|
||||
))
|
||||
.expect("static capabilities JSON serializes"),
|
||||
)
|
||||
})
|
||||
});
|
||||
let body = bodies[usize::from(ocs_version != 1)].clone();
|
||||
(
|
||||
[(axum::http::header::CONTENT_TYPE, "application/json")],
|
||||
body,
|
||||
)
|
||||
.into_response()
|
||||
}
|
||||
|
||||
pub async fn handle_user_info(
|
||||
State(state): State<Arc<AppState>>,
|
||||
session: crate::interfaces::nextcloud::session::NcSession,
|
||||
session: crate::interfaces::nextcloud::session::SharedNcSession,
|
||||
) -> Response {
|
||||
let quota: (i64, i64) = match state.storage_usage_service.as_ref() {
|
||||
Some(service) => match service.get_user_storage_info(session.user.id).await {
|
||||
@@ -530,11 +561,19 @@ fn empty_search_response() -> Json<serde_json::Value> {
|
||||
}))
|
||||
}
|
||||
|
||||
fn capabilities_payload(state: &AppState, ocs_version: u8) -> serde_json::Value {
|
||||
/// Build the capabilities JSON tree from its three config inputs. Public
|
||||
/// only under the `bench` feature caller path via
|
||||
/// [`capabilities_payload_for_bench`]; production reaches it once through
|
||||
/// the [`CAPABILITIES_BODIES`] init.
|
||||
fn capabilities_payload(
|
||||
base_url: &str,
|
||||
emulated_version: (u32, u32, u32),
|
||||
version_string: &str,
|
||||
ocs_version: u8,
|
||||
) -> serde_json::Value {
|
||||
let statuscode = if ocs_version == 1 { 100 } else { 200 };
|
||||
let base_url = state.core.config.base_url();
|
||||
let (nc_major, nc_minor, nc_micro) = state.core.config.nextcloud.emulated_version;
|
||||
let nc_version_str = state.core.config.nextcloud.version_string();
|
||||
let (nc_major, nc_minor, nc_micro) = emulated_version;
|
||||
let nc_version_str = version_string;
|
||||
|
||||
json!({
|
||||
"ocs": {
|
||||
@@ -602,6 +641,19 @@ fn capabilities_payload(state: &AppState, ocs_version: u8) -> serde_json::Value
|
||||
})
|
||||
}
|
||||
|
||||
/// Bench-only public wrapper (feature = "bench") over the private payload
|
||||
/// builder so `examples/bench_capabilities_static.rs` can A/B the
|
||||
/// rebuild-per-poll flow against the memoized bytes.
|
||||
#[cfg(feature = "bench")]
|
||||
pub fn capabilities_payload_for_bench(
|
||||
base_url: &str,
|
||||
emulated_version: (u32, u32, u32),
|
||||
version_string: &str,
|
||||
ocs_version: u8,
|
||||
) -> serde_json::Value {
|
||||
capabilities_payload(base_url, emulated_version, version_string, ocs_version)
|
||||
}
|
||||
|
||||
fn extract_basic_password(headers: &axum::http::HeaderMap) -> Option<String> {
|
||||
let value = headers
|
||||
.get(axum::http::header::AUTHORIZATION)?
|
||||
|
||||
@@ -10,9 +10,7 @@ use quick_xml::{
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::sync::Arc;
|
||||
|
||||
use crate::application::dtos::display_helpers::{
|
||||
category_for, format_file_size, icon_class_for, icon_special_class_for,
|
||||
};
|
||||
use crate::application::dtos::display_helpers::format_file_size;
|
||||
use crate::application::dtos::file_dto::FileDto;
|
||||
use crate::application::dtos::folder_dto::FolderDto;
|
||||
use crate::application::dtos::search_dto::SearchCriteriaDto;
|
||||
@@ -401,15 +399,16 @@ fn file_dto_from_search(fr: &crate::application::dtos::search_dto::SearchFileRes
|
||||
name: fr.name.clone(),
|
||||
path: fr.path.clone(),
|
||||
size: fr.size,
|
||||
mime_type: fr.mime_type.clone().into(),
|
||||
// Interned `Arc<str>` carried through from enrichment — refcount
|
||||
// bumps; the old code re-ran all three display classifiers and
|
||||
// re-allocated each value per converted search row.
|
||||
mime_type: fr.mime_type.clone(),
|
||||
folder_id: fr.folder_id.clone(),
|
||||
created_at: fr.created_at,
|
||||
modified_at: fr.modified_at,
|
||||
icon_class: icon_class_for(&fr.name, &fr.mime_type).to_string().into(),
|
||||
icon_special_class: icon_special_class_for(&fr.name, &fr.mime_type)
|
||||
.to_string()
|
||||
.into(),
|
||||
category: category_for(&fr.name, &fr.mime_type).to_string().into(),
|
||||
icon_class: fr.icon_class.clone(),
|
||||
icon_special_class: fr.icon_special_class.clone(),
|
||||
category: fr.category.clone(),
|
||||
size_formatted: format_file_size(fr.size),
|
||||
sort_date: None,
|
||||
content_hash: fr.blob_hash.clone(),
|
||||
@@ -420,6 +419,16 @@ fn file_dto_from_search(fr: &crate::application::dtos::search_dto::SearchFileRes
|
||||
}
|
||||
}
|
||||
|
||||
/// Bench-only public wrapper (feature = "bench") over the private
|
||||
/// search→FileDto conversion so `examples/bench_search_enrich.rs` can
|
||||
/// measure and equivalence-gate it.
|
||||
#[cfg(feature = "bench")]
|
||||
pub fn file_dto_from_search_for_bench(
|
||||
fr: &crate::application::dtos::search_dto::SearchFileResultDto,
|
||||
) -> FileDto {
|
||||
file_dto_from_search(fr)
|
||||
}
|
||||
|
||||
/// Build a `FolderDto` from a search folder result.
|
||||
fn folder_dto_from_search(
|
||||
sr: &crate::application::dtos::search_dto::SearchFolderResultDto,
|
||||
|
||||
@@ -17,7 +17,7 @@ use crate::interfaces::nextcloud::basic_auth_middleware::basic_auth_middleware;
|
||||
use crate::interfaces::nextcloud::login_v2_handler;
|
||||
use crate::interfaces::nextcloud::ocs_handler;
|
||||
use crate::interfaces::nextcloud::preview_handler;
|
||||
use crate::interfaces::nextcloud::session::NcSession;
|
||||
use crate::interfaces::nextcloud::session::SharedNcSession;
|
||||
use crate::interfaces::nextcloud::status_handler;
|
||||
use crate::interfaces::nextcloud::trashbin_handler;
|
||||
use crate::interfaces::nextcloud::uploads_handler;
|
||||
@@ -216,7 +216,7 @@ pub fn nextcloud_routes_with_state(state: Arc<AppState>) -> Router<Arc<AppState>
|
||||
async fn handle_dav_files(
|
||||
State(state): State<Arc<AppState>>,
|
||||
Path((_url_user, subpath)): Path<(String, String)>,
|
||||
session: NcSession,
|
||||
session: SharedNcSession,
|
||||
req: Request<Body>,
|
||||
) -> Result<Response, Response> {
|
||||
webdav_handler::handle_nc_webdav(state, req, session, subpath)
|
||||
@@ -227,7 +227,7 @@ async fn handle_dav_files(
|
||||
async fn handle_dav_files_root(
|
||||
State(state): State<Arc<AppState>>,
|
||||
Path(_url_user): Path<String>,
|
||||
session: NcSession,
|
||||
session: SharedNcSession,
|
||||
req: Request<Body>,
|
||||
) -> Result<Response, Response> {
|
||||
webdav_handler::handle_nc_webdav(state, req, session, String::new())
|
||||
@@ -238,7 +238,7 @@ async fn handle_dav_files_root(
|
||||
async fn handle_dav_uploads(
|
||||
State(state): State<Arc<AppState>>,
|
||||
Path((_url_user, upload_id, rest)): Path<(String, String, String)>,
|
||||
session: NcSession,
|
||||
session: SharedNcSession,
|
||||
req: Request<Body>,
|
||||
) -> Result<Response, Response> {
|
||||
uploads_handler::handle_nc_uploads(state, req, session, upload_id, rest)
|
||||
@@ -249,7 +249,7 @@ async fn handle_dav_uploads(
|
||||
async fn handle_dav_uploads_root(
|
||||
State(state): State<Arc<AppState>>,
|
||||
Path((_url_user, upload_id)): Path<(String, String)>,
|
||||
session: NcSession,
|
||||
session: SharedNcSession,
|
||||
req: Request<Body>,
|
||||
) -> Result<Response, Response> {
|
||||
uploads_handler::handle_nc_uploads(state, req, session, upload_id, String::new())
|
||||
@@ -279,7 +279,7 @@ async fn handle_legacy_webdav_root(user_ext: AuthUser) -> Response {
|
||||
async fn handle_dav_trashbin(
|
||||
State(state): State<Arc<AppState>>,
|
||||
Path((_url_user, subpath)): Path<(String, String)>,
|
||||
session: NcSession,
|
||||
session: SharedNcSession,
|
||||
req: Request<Body>,
|
||||
) -> Result<Response, Response> {
|
||||
trashbin_handler::handle_nc_trashbin(state, req, session, subpath)
|
||||
@@ -290,7 +290,7 @@ async fn handle_dav_trashbin(
|
||||
async fn handle_dav_trashbin_root(
|
||||
State(state): State<Arc<AppState>>,
|
||||
Path(_url_user): Path<String>,
|
||||
session: NcSession,
|
||||
session: SharedNcSession,
|
||||
req: Request<Body>,
|
||||
) -> Result<Response, Response> {
|
||||
trashbin_handler::handle_nc_trashbin(state, req, session, String::new())
|
||||
|
||||
@@ -3,8 +3,9 @@
|
||||
//! Bundles WHO the caller is, the raw wire username they presented,
|
||||
//! and (for path-scoped endpoints) WHERE they're confined to. Built
|
||||
//! by `basic_auth_middleware` and stashed in request extensions as
|
||||
//! `Arc<NcSession>`; handlers extract it via the [`FromRequestParts`]
|
||||
//! impl below — just declare `session: NcSession` in the signature.
|
||||
//! `Arc<NcSession>`; handlers extract it via [`SharedNcSession`]
|
||||
//! (derefs to `NcSession`) — declare `session: SharedNcSession` in
|
||||
//! the signature.
|
||||
//!
|
||||
//! ## Source of truth
|
||||
//!
|
||||
@@ -46,9 +47,13 @@ use crate::interfaces::middleware::auth::CurrentUser;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct NcSession {
|
||||
pub user: CurrentUser,
|
||||
/// Shared with the `Arc<CurrentUser>` request extension — one identity
|
||||
/// build per request instead of a clone per consumer.
|
||||
pub user: Arc<CurrentUser>,
|
||||
pub raw_username: String,
|
||||
pub chroot: Option<FolderDto>,
|
||||
/// Shared with `NC_CHROOT_CACHE` (markerless branch) — a cache hit is
|
||||
/// an `Arc` bump, not a `FolderDto` deep-clone.
|
||||
pub chroot: Option<Arc<FolderDto>>,
|
||||
}
|
||||
|
||||
impl NcSession {
|
||||
@@ -56,7 +61,7 @@ impl NcSession {
|
||||
/// without one. Documents the invariant that every NC route
|
||||
/// today is path-scoped — if this fires, route wiring is wrong.
|
||||
pub fn require_chroot(&self) -> Result<&FolderDto, AppError> {
|
||||
self.chroot.as_ref().ok_or_else(|| {
|
||||
self.chroot.as_deref().ok_or_else(|| {
|
||||
AppError::internal_error(
|
||||
"NcSession: path-scoped handler reached without a chroot — route wiring bug",
|
||||
)
|
||||
@@ -101,10 +106,13 @@ fn extract_url_user(path: &str) -> Option<String> {
|
||||
urlencoding::decode(user_seg).ok().map(|s| s.into_owned())
|
||||
}
|
||||
|
||||
/// Axum extractor: pulls the `Arc<NcSession>` that
|
||||
/// `basic_auth_middleware` stashed in request extensions and clones
|
||||
/// it (cheap — one `Arc` increment, no field copy) into an owned
|
||||
/// `NcSession` for handler use.
|
||||
/// Axum extractor: the shared handle to the request's [`NcSession`].
|
||||
///
|
||||
/// Derefs to `NcSession`, so handler bodies read `session.user`,
|
||||
/// `session.require_chroot()`, … unchanged. Extraction is one `Arc`
|
||||
/// refcount increment — the previous extractor deep-cloned the whole
|
||||
/// session (`CurrentUser` + `raw_username` + chroot `FolderDto`, ~8-9
|
||||
/// `String` allocs) on every authenticated NC request.
|
||||
///
|
||||
/// On path-scoped DAV routes (`/remote.php/dav/{files,uploads,
|
||||
/// trashbin}/{user}/…`), the URL `{user}` segment is cross-checked
|
||||
@@ -113,14 +121,33 @@ fn extract_url_user(path: &str) -> Option<String> {
|
||||
/// (`get_folder_with_perms`) is what actually prevents cross-user
|
||||
/// access. It just surfaces malformed requests early (403) instead
|
||||
/// of silently letting them through.
|
||||
impl<S: Send + Sync> FromRequestParts<S> for NcSession {
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SharedNcSession(Arc<NcSession>);
|
||||
|
||||
impl SharedNcSession {
|
||||
/// Wrap an already-shared session (used by the bench harness; the
|
||||
/// middleware inserts the `Arc` into request extensions directly).
|
||||
pub fn from_arc(session: Arc<NcSession>) -> Self {
|
||||
Self(session)
|
||||
}
|
||||
}
|
||||
|
||||
impl std::ops::Deref for SharedNcSession {
|
||||
type Target = NcSession;
|
||||
|
||||
fn deref(&self) -> &NcSession {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl<S: Send + Sync> FromRequestParts<S> for SharedNcSession {
|
||||
type Rejection = Response;
|
||||
|
||||
async fn from_request_parts(parts: &mut Parts, _state: &S) -> Result<Self, Self::Rejection> {
|
||||
let session = parts
|
||||
.extensions
|
||||
.get::<Arc<NcSession>>()
|
||||
.map(|arc| (**arc).clone())
|
||||
.cloned()
|
||||
.ok_or_else(|| StatusCode::UNAUTHORIZED.into_response())?;
|
||||
|
||||
if let Some(url_user) = extract_url_user(parts.uri.path())
|
||||
@@ -129,6 +156,6 @@ impl<S: Send + Sync> FromRequestParts<S> for NcSession {
|
||||
return Err(StatusCode::FORBIDDEN.into_response());
|
||||
}
|
||||
|
||||
Ok(session)
|
||||
Ok(Self(session))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,7 +27,7 @@ const HEADER_DAV: HeaderName = HeaderName::from_static("dav");
|
||||
pub async fn handle_nc_trashbin(
|
||||
state: Arc<AppState>,
|
||||
req: Request<Body>,
|
||||
session: crate::interfaces::nextcloud::session::NcSession,
|
||||
session: crate::interfaces::nextcloud::session::SharedNcSession,
|
||||
subpath: String,
|
||||
) -> Result<Response<Body>, AppError> {
|
||||
let method = req.method().clone();
|
||||
|
||||
@@ -110,7 +110,7 @@ async fn session_bytes_so_far(
|
||||
pub async fn handle_nc_uploads(
|
||||
state: Arc<AppState>,
|
||||
req: Request<Body>,
|
||||
session: crate::interfaces::nextcloud::session::NcSession,
|
||||
session: crate::interfaces::nextcloud::session::SharedNcSession,
|
||||
upload_id: String,
|
||||
rest: String, // chunk name or ".file" or empty
|
||||
) -> Result<Response<Body>, AppError> {
|
||||
|
||||
@@ -218,7 +218,7 @@ pub fn nc_href(username: &str, subpath: &str) -> String {
|
||||
pub async fn handle_nc_webdav(
|
||||
state: Arc<AppState>,
|
||||
req: Request<Body>,
|
||||
session: crate::interfaces::nextcloud::session::NcSession,
|
||||
session: crate::interfaces::nextcloud::session::SharedNcSession,
|
||||
subpath: String,
|
||||
) -> Result<Response<Body>, AppError> {
|
||||
// Validate up-front that we have a chroot — every method below is
|
||||
@@ -1566,19 +1566,29 @@ fn build_nc_streaming_propfind(
|
||||
}
|
||||
let batch_len = batch.len();
|
||||
|
||||
// Per-page enrichment: favorites + oc:fileids, two batch queries.
|
||||
let favs = if let Some(fav) = fav_svc {
|
||||
let items: Vec<(&str, &str)> =
|
||||
batch.iter().map(|f| (f.id.as_str(), "file")).collect();
|
||||
fav.batch_check_favorites(user_id, &items).await.unwrap_or_default()
|
||||
} else {
|
||||
HashSet::new()
|
||||
};
|
||||
// Per-page enrichment: favorites + oc:fileids + dead props —
|
||||
// three independent reads over the same id batch, overlapped
|
||||
// with `join!` so a page pays ~max(RTT) instead of 3×RTT
|
||||
// (each query still batched per page: DEAD-PROPS.md). The
|
||||
// round-7 deferred "serial pairs" item, adopted for this
|
||||
// per-page triple after the injected-latency A/B in
|
||||
// benches/ROUND9.md showed no local-PG regression.
|
||||
let fav_items: Vec<(&str, &str)> =
|
||||
batch.iter().map(|f| (f.id.as_str(), "file")).collect();
|
||||
let file_uuids: Vec<&str> = batch.iter().map(|f| f.id.as_str()).collect();
|
||||
let (file_id_map, _) = batch_resolve_ids(file_id_svc, &file_uuids, &[]).await;
|
||||
// One batched dead-props query per page, not one per child
|
||||
// (benches/DEAD-PROPS.md).
|
||||
let file_deads = files_dead_props_map(&state.webdav_dead_props, &batch).await;
|
||||
let (favs, (file_id_map, _), file_deads) = tokio::join!(
|
||||
async {
|
||||
if let Some(fav) = fav_svc {
|
||||
fav.batch_check_favorites(user_id, &fav_items)
|
||||
.await
|
||||
.unwrap_or_default()
|
||||
} else {
|
||||
HashSet::new()
|
||||
}
|
||||
},
|
||||
batch_resolve_ids(file_id_svc, &file_uuids, &[]),
|
||||
files_dead_props_map(&state.webdav_dead_props, &batch),
|
||||
);
|
||||
|
||||
let mut chunk = Vec::with_capacity(batch_len * 1024);
|
||||
{
|
||||
@@ -1624,18 +1634,23 @@ fn build_nc_streaming_propfind(
|
||||
break;
|
||||
}
|
||||
|
||||
let favs = if let Some(fav) = fav_svc {
|
||||
let items: Vec<(&str, &str)> =
|
||||
batch.iter().map(|sf| (sf.id.as_str(), "folder")).collect();
|
||||
fav.batch_check_favorites(user_id, &items).await.unwrap_or_default()
|
||||
} else {
|
||||
HashSet::new()
|
||||
};
|
||||
// Same overlapped enrichment triple as the file pages above.
|
||||
let fav_items: Vec<(&str, &str)> =
|
||||
batch.iter().map(|sf| (sf.id.as_str(), "folder")).collect();
|
||||
let folder_uuids: Vec<&str> = batch.iter().map(|sf| sf.id.as_str()).collect();
|
||||
let (_, sub_id_map) = batch_resolve_ids(file_id_svc, &[], &folder_uuids).await;
|
||||
// Batched — see benches/DEAD-PROPS.md.
|
||||
let sub_deads =
|
||||
folders_dead_props_map(&state.webdav_dead_props, &batch).await;
|
||||
let (favs, (_, sub_id_map), sub_deads) = tokio::join!(
|
||||
async {
|
||||
if let Some(fav) = fav_svc {
|
||||
fav.batch_check_favorites(user_id, &fav_items)
|
||||
.await
|
||||
.unwrap_or_default()
|
||||
} else {
|
||||
HashSet::new()
|
||||
}
|
||||
},
|
||||
batch_resolve_ids(file_id_svc, &[], &folder_uuids),
|
||||
folders_dead_props_map(&state.webdav_dead_props, &batch),
|
||||
);
|
||||
|
||||
let mut chunk = Vec::with_capacity(batch.len() * 1024);
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user