perf: round 6 backend — CardDAV cursor streaming, borrowed NC id chain, binary UUID decode, one-alloc hex
Benchmark-gated (equivalence + BEFORE/AFTER in examples/bench_*, results and reproduce commands in benches/ROUND6.md): - CardDAV whole-book REPORT + depth-1 PROPFIND stream through a PG cursor (stream_contacts_by_book, 500-contact pages) instead of materialising every vCard twice: 8 000 contacts TTFB 37.4 → 7.6 ms (4.9x), peak heap 19.0 → 7.0 MiB (2.7x), wall -23%; REPORT and PROPFIND byte-identical to the buffered writers. - NC numeric-id chain fully borrowed: get_or_create_file_ids/folder_ids take &[&str] and return HashMap<Uuid, i64>; batch_resolve_ids callers (PROPFIND pages, REPORT, trashbin, OCS search) pass id slices and look up via nc_id_of. 2.006 → 0.006 allocs/child (334x), 1.53x wall per 500-child page. batch_check_favorites binds &[&str] as text[]. - file_blob_read_repository listing SELECTs drop id::text/folder_id::text server casts: rows decode binary Uuid (16 vs 36 bytes on the wire) and render once in row_to_file. A/B on 500-row pages: 1.225 → 1.044 ms mean (1.17x), p95 1.686 → 1.345 (bench_uuid_text_cast; single-row, param and min() sites left as-is deliberately). - IncrementalHasher::finalize_hex renders through common::fmt::hex_lower instead of one format! per digest byte: 18 → 1 (md5) / 35 → 1 (sha256) allocs per chunk finalize, 14-15x wall. - Share landing overlaps the access-count UPDATE with the unlock fetch via tokio::join! (one round-trip off every public link hit). - REJECTED by benchmark and reverted: try_join_all fan-out of the batch-favorites authz pre-check — 42.6 → 56.4 ms cold, 0.15 → 0.23 ms warm against local-socket PG (bench_favorites_authz kept as evidence). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
This commit is contained in:
Claude
+29
@@ -350,6 +350,35 @@ name = "bench_micro_allocs"
|
||||
path = "examples/bench_micro_allocs.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# Round-6 battery ─────────────────────────────────────────────────────────────
|
||||
|
||||
# CardDAV whole-book REPORT/PROPFIND — buffered double-residency vs cursor
|
||||
# streaming; TTFB + peak live heap (needs the dev Postgres up).
|
||||
[[example]]
|
||||
name = "bench_carddav_stream"
|
||||
path = "examples/bench_carddav_stream.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# Batch-favorites authz pre-check — serial require loop vs try_join_all
|
||||
# against the real PgAclEngine (needs the dev Postgres up).
|
||||
[[example]]
|
||||
name = "bench_favorites_authz"
|
||||
path = "examples/bench_favorites_authz.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# Digest-hex rendering + NC id-batch marshalling micro-allocs (pure CPU).
|
||||
[[example]]
|
||||
name = "bench_hex_ids"
|
||||
path = "examples/bench_hex_ids.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# `id::text` server cast vs binary UUID decode + app-side formatting A/B
|
||||
# (needs the dev Postgres up).
|
||||
[[example]]
|
||||
name = "bench_uuid_text_cast"
|
||||
path = "examples/bench_uuid_text_cast.rs"
|
||||
required-features = ["bench"]
|
||||
|
||||
# Round-3 battery ─────────────────────────────────────────────────────────────
|
||||
|
||||
# Web-UI folder listing — whole-folder rescan + top-N sort per page vs keyset
|
||||
|
||||
@@ -0,0 +1,292 @@
|
||||
# Round 6 — CardDAV streaming, SPA quadratic re-render, borrowed NC id chain, authz fan-out
|
||||
|
||||
Benchmark-gated changes, same rule as ROUND2-5: every change ships with a
|
||||
BEFORE/AFTER benchmark; an AFTER that doesn't beat its BEFORE gets rolled
|
||||
back. Equivalence gates (byte-identical responses / identical outputs)
|
||||
guard every behavior-preserving rewrite. New this round: the frontend
|
||||
changes carry the same discipline as vitest benchmark gates (verbatim
|
||||
BEFORE replicas + perf assertions) committed beside the code, so CI
|
||||
re-verifies the wins on every run.
|
||||
|
||||
Measured on 4 cores / 15 GiB, local PostgreSQL 16 (fsync off), release
|
||||
profile; frontend on Node 22 / vitest 4 (jsdom). Reproduce any row with
|
||||
the command in its section.
|
||||
|
||||
## Summary
|
||||
|
||||
| # | change | key metric | before → after |
|
||||
|--:|---|---|---|
|
||||
| 1 | CardDAV whole-book streaming | TTFB / peak heap (8k contacts) | 37.4 → 7.6 ms (**4.9x**) / 19.0 → 7.0 MiB (**2.7x**), wall also -23% |
|
||||
| 2 | SPA progressive listing coalescing | 25-page load: emissions / sorted elements / wall | 25 → 2 / 65 000 → 5 200 (**12.5x**) / 30.9 → 4.0 ms (**7.8x**) |
|
||||
| 3 | SPA in-place `SvelteSet` selection/badges | 1 000 toggles @ N=5 000 / fan-out of 1 toggle over 40 rows | 771.9 → 1.9 ms (**399x**) / 40 → 3 re-runs (dense) |
|
||||
| 4 | SPA batch delete/move fan-out + id index | 100-item delete @ 5 ms RTT / id probes | 525 → 89 ms (**5.9x**) / 38 825 → 500 |
|
||||
| 5 | `t()` resolved-value cache + `{{` guard | 20k mixed translations | 22.7 → 8.6 ms (**2.63x**) |
|
||||
| 6 | Borrowed NC id chain (`&[&str]` / `Uuid` keys) | allocs/child (500-child page) | 2.006 → 0.006 (**334x**), wall **1.53x** |
|
||||
| 7 | `finalize_hex` one-alloc rendering | allocs/finalize (md5 / sha256) | 18 → 1 / 35 → 1 (**14-15x** wall) |
|
||||
| 8 | Batch-favorites authz `try_join_all` | 200-item pre-check, cold engine | **REJECTED**: 42.6 → 56.4 ms cold, 0.15 → 0.23 ms warm |
|
||||
| 9 | Share-landing `join!` | access-count + unlock serial → concurrent | (round-trip overlap; see §9) |
|
||||
| 10 | `::text` casts A/B (decide-by-bench) | 500-row page fetch | **ADOPTED** binary decode: 1.225 → 1.044 ms mean (**1.17x**), p95 1.686 → 1.345 |
|
||||
|
||||
## [1] CardDAV whole-book responses — buffered double-residency → cursor streaming
|
||||
|
||||
The round-5 CalDAV streaming pattern, applied to CardDAV: the
|
||||
addressbook REPORT path (`addressbook-query` without a uid filter,
|
||||
`sync-collection`) and the depth-1 collection PROPFIND materialised
|
||||
every contact DTO — each row carrying its full `vcard` body — into one
|
||||
Vec, then rendered the complete multistatus into a second in-RAM
|
||||
buffer: the book resident twice, TTFB = full generation time.
|
||||
|
||||
Now `ContactRepository::stream_contacts_by_book` serves one
|
||||
`ORDER BY full_name, first_name, last_name` scan through a PG cursor
|
||||
(same order as the buffered listing), and
|
||||
`build_streaming_contacts_report` / `build_streaming_book_propfind`
|
||||
cut pages of 500 contacts (no adjacency constraint — vCards are
|
||||
independent, unlike CalDAV's recurring-event UID bundles), streaming
|
||||
header → page chunks → footer through the split adapter writers
|
||||
(`write_report_multistatus_start` / `write_contacts_report_page` /
|
||||
`write_collection_head` / `write_collection_contact_page`, each with a
|
||||
reused href buffer). Multiget and depth-0 keep the buffered path. The
|
||||
address-book Read/public gate runs once before the cursor opens.
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_carddav_stream
|
||||
# 8000 contacts, page=500, 9 passes
|
||||
# [1] REPORT addressbook-query (getetag) TTFB ms wall ms peak heap MiB
|
||||
# BEFORE (buffered) 37.4 37.4 19.0
|
||||
# AFTER (cursor stream) 7.6 28.9 7.0
|
||||
# TTFB 4.9x, peak heap 2.7x lower, wall -23% (unlike CalDAV, no
|
||||
# wall trade: the vCard listing needs no window aggregate)
|
||||
# [gate] REPORT byte-identical: OK · collection PROPFIND byte-identical: OK
|
||||
```
|
||||
|
||||
## [2] SPA progressive listing — emit-per-page O(N²) re-derive → coalesced emissions
|
||||
|
||||
`fetchFolderListing` pages `/api/folders/{id}/resources` 200 rows at a
|
||||
time and invoked `onPage` after EVERY page with a fresh copy of the
|
||||
whole accumulated listing; the files view re-derives its filtered +
|
||||
sorted view (two `localeCompare` sorts + entries/orderedIds rebuild)
|
||||
from each emission. A 5 000-item folder = 25 pages = Σ 65 000 elements
|
||||
re-sorted on the main thread during one load — hundreds of ms of jank
|
||||
on exactly the large folders progressive rendering was meant to help.
|
||||
Now page one (first paint) and the final page always emit, and
|
||||
intermediate pages emit at most once per 150 ms
|
||||
(`PAGE_EMIT_MIN_INTERVAL_MS`).
|
||||
|
||||
Gates: final listing identical to the emit-every-page reference; first
|
||||
emission still page one; exactly one `done` emission carrying the
|
||||
complete listing; on a fast connection the consumer derive work must
|
||||
collapse ≥5x and wall ≥3x.
|
||||
|
||||
```
|
||||
cd frontend && npx vitest run src/lib/api/endpoints/folders.bench.test.ts --disable-console-intercept
|
||||
# progressive load 25×200: before 25 emissions / 65000 sorted elements / 30.9 ms
|
||||
# after 2 emissions / 5200 sorted elements / 4.0 ms
|
||||
# (7.8x wall, 12.5x fewer sorted elements)
|
||||
```
|
||||
|
||||
## [3] SPA selection/badge sets — copy-reassign → in-place `SvelteSet`
|
||||
|
||||
The files view's `selected` / `favoriteIds` / `sharedIds` (and the
|
||||
recent view's `favoriteIds`) were plain `$state<Set>`s rebuilt from a
|
||||
full copy on every single-item toggle (`new SvelteSet(selected)` +
|
||||
reassign): an O(N) copy per toggle — N unbounded under "select all →
|
||||
refine" — plus a state-reference swap that invalidates every mounted
|
||||
row's `.has()` read. Now each is one `SvelteSet` mutated in place (the
|
||||
pattern `useSelection` already shipped; the views now match it), with
|
||||
`replaceSet` (`lib/utils/sets.ts`) for wholesale refills.
|
||||
|
||||
Measured `SvelteSet` granularity (svelte 5.56 `reactivity/set.js`):
|
||||
present keys are per-key sources; `.has()` on an absent key tracks the
|
||||
set-version signal, so miss-readers re-run on any mutation in both
|
||||
patterns. The in-place win = no O(N) copy + every other present-key
|
||||
reader spared. Fan-out for one toggle across 40 mounted row effects:
|
||||
sparse selection (10/40) 40 → 31 re-runs; dense "select all → refine"
|
||||
(38/40) 40 → **3**.
|
||||
|
||||
```
|
||||
cd frontend && npx vitest run src/lib/composables/selectionPatterns.bench.test.ts --disable-console-intercept
|
||||
# 1000 toggles @ N=5000: copy-reassign 771.9 ms vs in-place 1.9 ms (398.8x)
|
||||
# fan-out of 1 toggle across 40 row effects:
|
||||
# 10/40 selected: copy 40 vs in-place 31 · 38/40 selected: copy 40 vs in-place 3
|
||||
```
|
||||
|
||||
## [4] SPA batch operations — serial await + O(N·M) probes → id index + `mapLimit(6)`
|
||||
|
||||
`batchDelete` / `moveInto` awaited one request per item in a serial
|
||||
loop, and `batchDelete` / `batchDownload` / `selectionTargets` probed
|
||||
`listing.folders.find(...)` / `.some(...)` per selected id (O(N·M)
|
||||
scans). Now a `Set`/`Map` id index is built once per operation (O(M))
|
||||
and the per-item requests fan out through the view's existing
|
||||
`mapLimit` with 6 in flight. Failure semantics preserved: deletes toast
|
||||
individually and continue (as the serial loop did); `moveInto` attempts
|
||||
every item, surfaces the first error and keeps the selection for retry.
|
||||
|
||||
```
|
||||
cd frontend && npx vitest run src/routes/files/batchOps.bench.test.ts --disable-console-intercept
|
||||
# batch delete 100 items @ 5 ms RTT:
|
||||
# serial 525 ms (38825 id probes) vs mapLimit(6) 89 ms (500 probes) — 5.9x
|
||||
```
|
||||
|
||||
## [5] i18n `t()` — split+walk+regex per call → resolved-value cache + `{{` guard
|
||||
|
||||
The locale dicts are nested, so every `t('a.b.c')` re-split its key and
|
||||
walked the tree; `interpolate` ran its global-regex `.replace` on every
|
||||
string although only ~7% of en.json values contain `{{`. A rendered
|
||||
list row calls `t()` ~10×. Now the resolved value is cached per
|
||||
(dict, key) in a `WeakMap<Dict, Map>` — dicts are load-once-immutable —
|
||||
and `interpolate` short-circuits on `!text.includes('{{')`.
|
||||
|
||||
Gates: byte-identical to the pre-fix reference across every real
|
||||
en.json key (nested, flat, underscore-fallback, missing), cold and
|
||||
warm; ≥1.5x on a 20k-call mixed workload. (A first attempt cached only
|
||||
the key split: 1.12x — below the gate; the value cache landed 2.63x.)
|
||||
|
||||
```
|
||||
cd frontend && npx vitest run src/lib/i18n/i18n.bench.test.ts --disable-console-intercept
|
||||
# t() hot path x 20000: cached+guarded 8.6 ms vs split+regex-per-call 22.7 ms (2.63x)
|
||||
```
|
||||
|
||||
## [6] NC numeric-id chain — `Vec<String>` clones + `String`-keyed maps → borrowed `&[&str]` / `Uuid` keys
|
||||
|
||||
`batch_resolve_ids` (NC PROPFIND/REPORT/trashbin/OCS-search) cloned
|
||||
every child id into a `Vec<String>`, and `NextcloudFileIdService`
|
||||
re-keyed its result map with another `String` per id — ~3 heap allocs
|
||||
per child per 500-child page, every page. The whole chain is now
|
||||
borrowed: `get_or_create_file_ids(&[&str]) -> HashMap<Uuid, i64>`
|
||||
(cache-miss dedup via sort+dedup on `Vec<Uuid>` instead of a
|
||||
`HashMap<Uuid, String>`), callers pass `&[&str]` slices, and lookups go
|
||||
through `nc_id_of` (`Uuid::parse_str` + `HashMap<Uuid, i64>` get — a
|
||||
16-byte hash instead of a 36-byte string hash). `batch_check_favorites`
|
||||
drops its id `to_string` loop the same way (sqlx binds `&[&str]` as
|
||||
`text[]`).
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_hex_ids
|
||||
# batch_resolve_ids marshalling: String-keyed vs borrowed+Uuid
|
||||
# (1000 pages x 500 children/arm)
|
||||
# arm | allocs | wall ms | allocs/child
|
||||
# BEFORE | 1 003 000 | 85.97 | 2.006
|
||||
# AFTER | 3 000 | 56.27 | 0.006 (334x fewer allocs, 1.53x wall)
|
||||
```
|
||||
|
||||
## [7] `finalize_hex` — one `format!` per digest byte → single-buffer hex
|
||||
|
||||
`IncrementalHasher::finalize_hex` rendered MD5 / SHA-256 digests with
|
||||
`.map(|b| format!("{b:02x}")).collect()` — a heap `String` per digest
|
||||
byte (16 / 32 allocs) on every chunk finalize of every chunked upload.
|
||||
Now `common::fmt::hex_lower` (new, unit-tested against the `format!`
|
||||
reference) writes both nibbles per byte into one preallocated String.
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_hex_ids
|
||||
# finalize_hex: per-byte format! vs hex_lower (10 000 finalizes/arm)
|
||||
# digest | arm | allocs | wall ms | allocs/call
|
||||
# md5 | BEFORE | 180 000 | 6.44 | 18.00
|
||||
# md5 | AFTER | 10 000 | 0.45 | 1.00 (14.3x wall)
|
||||
# sha256 | BEFORE | 350 000 | 12.34 | 35.00
|
||||
# sha256 | AFTER | 10 000 | 0.80 | 1.00 (15.4x wall)
|
||||
```
|
||||
|
||||
## [8] Batch-favorites authz pre-check — serial `require` loop → `try_join_all`
|
||||
|
||||
`batch_add_to_favorites` awaited `Permission::Read` per item
|
||||
one-by-one; for a "select all → add to favorites" over N items whose
|
||||
drive lookups aren't cached, that is N sequential point-SELECT
|
||||
round-trips before the batched insert starts. The checks are
|
||||
independent, so they now fan out with `futures::future::try_join_all` —
|
||||
fail-fast on any denial preserved (the anti-oracle all-or-nothing
|
||||
response shape is unchanged; unparseable ids now fail before any check
|
||||
runs instead of mid-loop).
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_favorites_authz
|
||||
# files=200 pool=20 (shared-drive member, editor grant)
|
||||
# arm | wall ms | us/item
|
||||
# serial COLD | 42.62 | 213.12
|
||||
# join COLD | 56.44 | 282.20 <-- WORSE
|
||||
# serial WARM | 0.15 | 0.73
|
||||
# join WARM | 0.23 | 1.16 <-- WORSE
|
||||
```
|
||||
|
||||
## [9] Share landing — serial access-count + unlock → `tokio::join!`
|
||||
|
||||
`access_shared_item` awaited `register_shared_link_access` (an UPDATE)
|
||||
and then `get_shared_link_with_unlock` — two dependent-free round trips
|
||||
in series on every public share-link hit. They now run under one
|
||||
`tokio::join!`, overlapping the UPDATE with the SELECT+unlock chain;
|
||||
response semantics unchanged (the handler only branches on the second
|
||||
result, and the access-count write was already fire-and-forget with
|
||||
respect to the response). Covered by the round-trip arithmetic rather
|
||||
than a dedicated harness: the landing's latency is now
|
||||
`max(update, select)` instead of `update + select`.
|
||||
|
||||
## [10] `id::text` casts A/B — decided by bench
|
||||
|
||||
~18 SELECT sites in `file_blob_read_repository.rs` cast UUID columns to
|
||||
text server-side (`id::text`) and decode `String`. The alternative
|
||||
(binary `Uuid` decode + app-side `to_string`) was benched on identical
|
||||
500-row pages, interleaved A/B, equivalence-gated on identical string
|
||||
triples:
|
||||
|
||||
```
|
||||
cargo run --release --features bench --example bench_uuid_text_cast
|
||||
# rows/page=500 passes=200 (interleaved)
|
||||
# arm | mean ms | p50 ms | p95 ms
|
||||
# A ::text (current) | 1.225 | 1.176 | 1.686
|
||||
# B binary + to_string | 1.044 | 1.026 | 1.345
|
||||
# B/A mean ratio: 0.853 -> binary decode wins (1.17x)
|
||||
```
|
||||
|
||||
**Adopted**: `file_blob_read_repository.rs`'s page-shaped SELECTs (the 14
|
||||
`fi.id/fi.folder_id` listing queries + the Photos `top.*` feed — every
|
||||
`FileRow`/`MediaFileRow`/inline tuple) now decode binary `Uuid` and render
|
||||
once in `row_to_file`, the single choke point. Wire size for the two id
|
||||
columns drops 36+36 → 16+16 bytes/row and the server skips the cast.
|
||||
Left as `::text` deliberately: the one-row `fetch_optional` folder lookup
|
||||
(cast cost is sub-µs per call, no page effect), the `$3::text IS NULL`
|
||||
param cast, and `min(fm.file_id::text)` (text-min ≠ uuid-min ordering —
|
||||
changing it would alter which sample id is returned). Other repos with
|
||||
the same shape are queued for round 7 with this bench as the evidence.
|
||||
|
||||
## Rejected / deferred this round
|
||||
|
||||
- **JWT claims `Arc<str>`** (round-5 follow-up): `CurrentUser.username`
|
||||
/ `.email` are `String`s cloned per request from the cached
|
||||
`Arc<TokenClaims>`. Converting both structs to `Arc<str>` needs
|
||||
serde's `rc` feature for the JWT `Deserialize` and touches every
|
||||
`current_user.username` read site (~dozens across REST/DAV/NC
|
||||
handlers) for two small allocs per request — deferred to round 7 as a
|
||||
contained refactor with its own bench.
|
||||
- **Thumbnail ACL-before-304** (hunt finding): the ETag-304 and
|
||||
moka/disk short-circuits in `get_thumbnail_impl` run after
|
||||
`require_permission(Read)`, so shared-album recipients pay a grant
|
||||
cascade query per thumbnail revalidation. The fix (back the non-owner
|
||||
path with `drive_role_cache`, or reorder the 304 check) is
|
||||
authz-sensitive and needs its own carefully-gated round-7 slot.
|
||||
- **Thumbnail cache `String` key per request** and **`batch_operations`
|
||||
per-item `target_folder.to_string()`**: micro-allocs; the first needs
|
||||
a `Borrow`-friendly moka key design, the second an `Option<&str>`
|
||||
widening of `_with_perms` signatures. Both queued for a micro-alloc
|
||||
sweep with `bench_hex_ids`-style gates.
|
||||
|
||||
## Notes
|
||||
|
||||
- `deltaUpload.hash.test.ts`'s pre-existing "3-lane pool beats
|
||||
sequential" gate does not hold in this 4-core CI-class container
|
||||
(0.9-1.0x isolated, repeatedly) — environmental, unrelated to this
|
||||
round's changes, left untouched.
|
||||
- The frontend engine floor (`node >= 24`) makes `npm ci` require npm
|
||||
≥ 11 lockfile resolution; on a Node 22 box use `npx npm@12 ci`.
|
||||
|
||||
## Follow-ups seeded for round 7
|
||||
|
||||
- JWT claims `Arc<str>` end-to-end (see above).
|
||||
- Thumbnail 304/cache path vs ACL ordering (see above).
|
||||
- `fetchFolderListing` returns empty `favoriteIds`/`sharedIds` since the
|
||||
combined `/listing` route was removed — the files-view badge sets are
|
||||
seeded empty on navigation (functional regression flag, not perf).
|
||||
- Search page lacks a stale-response `seq` guard (files view has
|
||||
`loadSeq`); a slow stale filter response can clobber a newer one.
|
||||
- `list_folder_resources` clones `row.name` only because `icon_class_for`
|
||||
borrows it later — reorder to let the name move.
|
||||
- Swimlane/photos virtualization (carried from round 5).
|
||||
@@ -0,0 +1,409 @@
|
||||
//! CardDAV whole-book response benchmark — buffered vs cursor streaming
|
||||
//! (ROUND6).
|
||||
//!
|
||||
//! The REPORT path (addressbook-query, sync-collection) and the depth-1
|
||||
//! collection PROPFIND materialised EVERY contact DTO of the book in
|
||||
//! one Vec, then rendered the complete multistatus into a second in-RAM
|
||||
//! buffer — the book resident twice, TTFB = full generation. AFTER
|
||||
//! streams ONE ordered scan (`full_name, first_name, last_name`, the
|
||||
//! buffered listing's order) through a PG cursor and emits fixed-size
|
||||
//! pages (contacts carry no bundling constraint).
|
||||
//!
|
||||
//! Drives the REAL repository + adapter writers both ways at the repo
|
||||
//! layer (authz identical both sides, excluded). Gates: streamed
|
||||
//! concatenation byte-identical to the buffered output for the REPORT
|
||||
//! (getetag poll shape) AND the collection PROPFIND (allprop), seeded
|
||||
//! with strictly distinct names so ordering is deterministic.
|
||||
//!
|
||||
//! Run (needs Postgres up; reads DATABASE_URL from .env):
|
||||
//! cargo run --release --features bench --example bench_carddav_stream
|
||||
//! Tunables (env): BENCH_CONTACTS (8000), BENCH_PAGE (500), BENCH_PASSES (9).
|
||||
|
||||
use std::alloc::{GlobalAlloc, Layout, System};
|
||||
use std::env;
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use oxicloud::application::adapters::carddav_adapter::{CardDavAdapter, CardDavReportType};
|
||||
use oxicloud::application::adapters::webdav_adapter::{
|
||||
PropFindRequest, PropFindType, QualifiedName,
|
||||
};
|
||||
use oxicloud::application::dtos::address_book_dto::AddressBookDto;
|
||||
use oxicloud::application::dtos::contact_dto::ContactDto;
|
||||
use oxicloud::domain::repositories::contact_repository::ContactRepository;
|
||||
use oxicloud::infrastructure::repositories::pg::ContactPgRepository;
|
||||
use sqlx::PgPool;
|
||||
use sqlx::postgres::PgPoolOptions;
|
||||
use uuid::Uuid;
|
||||
|
||||
// ─── Peak-live-heap tracking allocator ──────────────────────────────────────
|
||||
|
||||
static LIVE: AtomicU64 = AtomicU64::new(0);
|
||||
static PEAK: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
struct PeakAlloc;
|
||||
|
||||
fn bump(sz: u64) {
|
||||
let live = LIVE.fetch_add(sz, Ordering::Relaxed) + sz;
|
||||
PEAK.fetch_max(live, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
unsafe impl GlobalAlloc for PeakAlloc {
|
||||
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
|
||||
bump(layout.size() as u64);
|
||||
unsafe { System.alloc(layout) }
|
||||
}
|
||||
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
|
||||
LIVE.fetch_sub(layout.size() as u64, Ordering::Relaxed);
|
||||
unsafe { System.dealloc(ptr, layout) }
|
||||
}
|
||||
unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
|
||||
if new_size > layout.size() {
|
||||
bump((new_size - layout.size()) as u64);
|
||||
} else {
|
||||
LIVE.fetch_sub((layout.size() - new_size) as u64, Ordering::Relaxed);
|
||||
}
|
||||
unsafe { System.realloc(ptr, layout, new_size) }
|
||||
}
|
||||
unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
|
||||
bump(layout.size() as u64);
|
||||
unsafe { System.alloc_zeroed(layout) }
|
||||
}
|
||||
}
|
||||
|
||||
#[global_allocator]
|
||||
static GLOBAL: PeakAlloc = PeakAlloc;
|
||||
|
||||
struct Seeded {
|
||||
book_id: Uuid,
|
||||
owner_id: Uuid,
|
||||
}
|
||||
|
||||
async fn seed(pool: &PgPool, n: usize) -> Seeded {
|
||||
let owner_id: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO auth.users (username, email, role)
|
||||
VALUES ('bench_cardstream', 'bench_cardstream@bench.invalid', 'user') RETURNING id",
|
||||
)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.expect("seed user");
|
||||
let book_id: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO carddav.address_books (id, name, owner_id)
|
||||
VALUES (gen_random_uuid(), 'Libreta grande', $1) RETURNING id",
|
||||
)
|
||||
.bind(owner_id)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.expect("seed book");
|
||||
|
||||
let mut tx = pool.begin().await.expect("begin");
|
||||
for i in 0..n {
|
||||
// Strictly distinct full_names keep the listing order (and thus
|
||||
// the byte gate) deterministic. Every production row carries its
|
||||
// full serialized vCard — the payload whose double-residency the
|
||||
// streaming path removes — so the seed does too (~250 B each).
|
||||
let uid = format!("contact-{i:06}");
|
||||
let vcard = format!(
|
||||
"BEGIN:VCARD\r\nVERSION:3.0\r\nUID:{uid}\r\nFN:Persona {i:06}\r\nN:Apellido{i};Nombre{i};;;\r\nEMAIL;TYPE=INTERNET:persona{i}@bench.invalid\r\nTEL;TYPE=CELL:+34 600 {i:06}\r\nORG:OxiCloud Bench\r\nNOTE:Fila sintetica del banco de pruebas CardDAV.\r\nEND:VCARD\r\n"
|
||||
);
|
||||
sqlx::query(
|
||||
"INSERT INTO carddav.contacts
|
||||
(id, address_book_id, uid, full_name, first_name, last_name, vcard, etag)
|
||||
VALUES (gen_random_uuid(), $1, $2, $3, $4, $5, $6, $7)",
|
||||
)
|
||||
.bind(book_id)
|
||||
.bind(&uid)
|
||||
.bind(format!("Persona {i:06}"))
|
||||
.bind(format!("Nombre{i}"))
|
||||
.bind(format!("Apellido{i}"))
|
||||
.bind(&vcard)
|
||||
.bind(format!("{:016x}", (i as u64).wrapping_mul(2_654_435_761)))
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.expect("seed contact");
|
||||
}
|
||||
tx.commit().await.expect("commit");
|
||||
Seeded { book_id, owner_id }
|
||||
}
|
||||
|
||||
async fn cleanup(pool: &PgPool, s: &Seeded) {
|
||||
let _ = sqlx::query("DELETE FROM carddav.contacts WHERE address_book_id = $1")
|
||||
.bind(s.book_id)
|
||||
.execute(pool)
|
||||
.await;
|
||||
let _ = sqlx::query("DELETE FROM carddav.address_books WHERE id = $1")
|
||||
.bind(s.book_id)
|
||||
.execute(pool)
|
||||
.await;
|
||||
let _ = sqlx::query("DELETE FROM auth.users WHERE id = $1")
|
||||
.bind(s.owner_id)
|
||||
.execute(pool)
|
||||
.await;
|
||||
}
|
||||
|
||||
fn report_shape() -> CardDavReportType {
|
||||
CardDavReportType::AddressbookQuery {
|
||||
props: vec![
|
||||
QualifiedName::new("DAV:", "getetag"),
|
||||
QualifiedName::new("DAV:", "getcontenttype"),
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
async fn fetch_all_dtos(repo: &ContactPgRepository, book_id: &Uuid) -> Vec<ContactDto> {
|
||||
repo.get_contacts_by_address_book(book_id)
|
||||
.await
|
||||
.expect("list contacts")
|
||||
.into_iter()
|
||||
.map(ContactDto::from)
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// BEFORE: full fetch + whole-response buffer. First byte exists only
|
||||
/// when everything does.
|
||||
async fn buffered_report(
|
||||
repo: &ContactPgRepository,
|
||||
book_id: &Uuid,
|
||||
base_href: &str,
|
||||
) -> (f64, Vec<u8>) {
|
||||
let t0 = Instant::now();
|
||||
let contacts = fetch_all_dtos(repo, book_id).await;
|
||||
let mut out = Vec::with_capacity(contacts.len() * 256);
|
||||
CardDavAdapter::generate_contacts_response(&mut out, &contacts, &report_shape(), base_href)
|
||||
.expect("generate");
|
||||
(t0.elapsed().as_secs_f64() * 1e3, out)
|
||||
}
|
||||
|
||||
/// AFTER: cursor + page writers (the handler loop over public pieces).
|
||||
/// Returns (ttfb_ms — first data page rendered, wall_ms, bytes).
|
||||
async fn streamed_report(
|
||||
repo: &ContactPgRepository,
|
||||
book_id: &Uuid,
|
||||
base_href: &str,
|
||||
page_rows: usize,
|
||||
accumulate: bool,
|
||||
) -> (f64, f64, Vec<u8>) {
|
||||
use futures::TryStreamExt;
|
||||
let t0 = Instant::now();
|
||||
let mut ttfb = None;
|
||||
let mut all = Vec::new();
|
||||
let report = report_shape();
|
||||
|
||||
let mut chunk = Vec::with_capacity(160);
|
||||
{
|
||||
let mut w = quick_xml::Writer::new(&mut chunk);
|
||||
CardDavAdapter::write_report_multistatus_start(&mut w).expect("start");
|
||||
}
|
||||
if accumulate {
|
||||
all.extend_from_slice(&chunk);
|
||||
}
|
||||
|
||||
let mut rows = repo.stream_contacts_by_book(*book_id);
|
||||
let mut page: Vec<ContactDto> = Vec::with_capacity(page_rows);
|
||||
loop {
|
||||
let next = rows
|
||||
.try_next()
|
||||
.await
|
||||
.expect("stream row")
|
||||
.map(ContactDto::from);
|
||||
let flush = match &next {
|
||||
Some(_) => page.len() >= page_rows,
|
||||
None => !page.is_empty(),
|
||||
};
|
||||
if flush {
|
||||
let mut chunk = Vec::with_capacity(page.len() * 256 + 64);
|
||||
{
|
||||
let mut w = quick_xml::Writer::new(&mut chunk);
|
||||
CardDavAdapter::write_contacts_report_page(&mut w, &page, &report, base_href)
|
||||
.expect("page");
|
||||
}
|
||||
ttfb.get_or_insert_with(|| t0.elapsed().as_secs_f64() * 1e3);
|
||||
page.clear();
|
||||
if accumulate {
|
||||
all.extend_from_slice(&chunk);
|
||||
}
|
||||
std::hint::black_box(&chunk);
|
||||
}
|
||||
match next {
|
||||
Some(c) => page.push(c),
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
|
||||
let mut chunk = Vec::with_capacity(32);
|
||||
{
|
||||
let mut w = quick_xml::Writer::new(&mut chunk);
|
||||
CardDavAdapter::write_carddav_multistatus_end(&mut w).expect("end");
|
||||
}
|
||||
if accumulate {
|
||||
all.extend_from_slice(&chunk);
|
||||
}
|
||||
(
|
||||
ttfb.unwrap_or(f64::NAN),
|
||||
t0.elapsed().as_secs_f64() * 1e3,
|
||||
all,
|
||||
)
|
||||
}
|
||||
|
||||
fn p50(mut xs: Vec<f64>) -> f64 {
|
||||
xs.sort_by(|a, b| a.partial_cmp(b).unwrap());
|
||||
xs[xs.len() / 2]
|
||||
}
|
||||
|
||||
fn reset_peak() {
|
||||
PEAK.store(LIVE.load(Ordering::Relaxed), Ordering::Relaxed);
|
||||
}
|
||||
|
||||
fn peak_mib() -> f64 {
|
||||
PEAK.load(Ordering::Relaxed) as f64 / (1024.0 * 1024.0)
|
||||
}
|
||||
|
||||
fn book_dto(seeded: &Seeded) -> AddressBookDto {
|
||||
AddressBookDto {
|
||||
id: seeded.book_id.to_string(),
|
||||
name: "Libreta grande".to_string(),
|
||||
owner_id: seeded.owner_id.to_string(),
|
||||
..AddressBookDto::default()
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::main(flavor = "multi_thread")]
|
||||
async fn main() {
|
||||
dotenvy::dotenv().ok();
|
||||
let url = env::var("DATABASE_URL")
|
||||
.or_else(|_| env::var("OXICLOUD_DB_CONNECTION_STRING"))
|
||||
.expect("set DATABASE_URL — the dev Postgres URL");
|
||||
let n: usize = env::var("BENCH_CONTACTS")
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(8000);
|
||||
let page_rows: usize = env::var("BENCH_PAGE")
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(500);
|
||||
let passes: usize = env::var("BENCH_PASSES")
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(9);
|
||||
|
||||
let pool = Arc::new(
|
||||
PgPoolOptions::new()
|
||||
.max_connections(10)
|
||||
.min_connections(10)
|
||||
.acquire_timeout(Duration::from_secs(10))
|
||||
.connect(&url)
|
||||
.await
|
||||
.expect("connect Postgres"),
|
||||
);
|
||||
|
||||
let seeded = seed(&pool, n).await;
|
||||
let repo = ContactPgRepository::new(pool.clone());
|
||||
let base_href = format!("/carddav/{}/", seeded.book_id);
|
||||
|
||||
println!("bench_carddav_stream — {n} contacts, page={page_rows}, {passes} passes\n");
|
||||
|
||||
// ── Equivalence gates ───────────────────────────────────────────────────
|
||||
let (_, before_bytes) = buffered_report(&repo, &seeded.book_id, &base_href).await;
|
||||
let (_, _, after_bytes) =
|
||||
streamed_report(&repo, &seeded.book_id, &base_href, page_rows, true).await;
|
||||
let gate_report = before_bytes == after_bytes;
|
||||
|
||||
// Collection PROPFIND (allprop): buffered generator vs head+pages.
|
||||
let request = PropFindRequest {
|
||||
prop_find_type: PropFindType::AllProp,
|
||||
};
|
||||
let book = book_dto(&seeded);
|
||||
let contacts_all = fetch_all_dtos(&repo, &seeded.book_id).await;
|
||||
let mut coll_before = Vec::new();
|
||||
CardDavAdapter::generate_addressbook_collection_propfind(
|
||||
&mut coll_before,
|
||||
&book,
|
||||
&contacts_all,
|
||||
&request,
|
||||
&base_href,
|
||||
"1",
|
||||
)
|
||||
.expect("collection");
|
||||
drop(contacts_all);
|
||||
let coll_after = {
|
||||
use futures::TryStreamExt;
|
||||
let mut out = Vec::new();
|
||||
{
|
||||
let mut w = quick_xml::Writer::new(&mut out);
|
||||
CardDavAdapter::write_collection_head(&mut w, &book, &request, &base_href)
|
||||
.expect("head");
|
||||
}
|
||||
let mut rows = repo.stream_contacts_by_book(seeded.book_id);
|
||||
let mut page: Vec<ContactDto> = Vec::with_capacity(page_rows);
|
||||
loop {
|
||||
let next = rows
|
||||
.try_next()
|
||||
.await
|
||||
.expect("stream row")
|
||||
.map(ContactDto::from);
|
||||
let flush = match &next {
|
||||
Some(_) => page.len() >= page_rows,
|
||||
None => !page.is_empty(),
|
||||
};
|
||||
if flush {
|
||||
let mut w = quick_xml::Writer::new(&mut out);
|
||||
CardDavAdapter::write_collection_contact_page(&mut w, &page, &base_href)
|
||||
.expect("page");
|
||||
page.clear();
|
||||
}
|
||||
match next {
|
||||
Some(c) => page.push(c),
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
let mut w = quick_xml::Writer::new(&mut out);
|
||||
CardDavAdapter::write_carddav_multistatus_end(&mut w).expect("end");
|
||||
out
|
||||
};
|
||||
let gate_coll = coll_before == coll_after;
|
||||
drop(coll_before);
|
||||
drop(coll_after);
|
||||
|
||||
// ── [1] REPORT timing + peak ────────────────────────────────────────────
|
||||
let mut b_wall = Vec::new();
|
||||
let mut a_wall = Vec::new();
|
||||
let mut a_ttfb = Vec::new();
|
||||
for _ in 0..passes {
|
||||
let (w, out) = buffered_report(&repo, &seeded.book_id, &base_href).await;
|
||||
std::hint::black_box(out);
|
||||
b_wall.push(w);
|
||||
let (t, w, _) = streamed_report(&repo, &seeded.book_id, &base_href, page_rows, false).await;
|
||||
a_ttfb.push(t);
|
||||
a_wall.push(w);
|
||||
}
|
||||
reset_peak();
|
||||
let (_, out) = buffered_report(&repo, &seeded.book_id, &base_href).await;
|
||||
drop(out);
|
||||
let peak_before = peak_mib();
|
||||
reset_peak();
|
||||
let _ = streamed_report(&repo, &seeded.book_id, &base_href, page_rows, false).await;
|
||||
let peak_after = peak_mib();
|
||||
|
||||
let bw = p50(b_wall);
|
||||
let aw = p50(a_wall);
|
||||
let at = p50(a_ttfb);
|
||||
println!("[1] REPORT addressbook-query (getetag) TTFB ms wall ms peak heap MiB");
|
||||
println!(" BEFORE (buffered) {bw:8.1} {bw:8.1} {peak_before:10.1}");
|
||||
println!(
|
||||
" AFTER (cursor stream) {at:8.1} {aw:8.1} {peak_after:10.1} TTFB {:.1}x, heap {:.1}x lower",
|
||||
bw / at,
|
||||
peak_before / peak_after
|
||||
);
|
||||
|
||||
cleanup(&pool, &seeded).await;
|
||||
|
||||
println!(
|
||||
"\n[gate] REPORT byte-identical: {} · collection PROPFIND byte-identical: {}",
|
||||
if gate_report { "OK" } else { "FAILED" },
|
||||
if gate_coll { "OK" } else { "FAILED" }
|
||||
);
|
||||
if !gate_report || !gate_coll {
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,305 @@
|
||||
//! Batch-favorites AuthZ fan-out benchmark — serial `require` loop vs
|
||||
//! `try_join_all`.
|
||||
//!
|
||||
//! VERDICT (round 6): the fan-out measured WORSE on both the cold and the
|
||||
//! warm path against local-socket Postgres (see benches/ROUND6.md), so the
|
||||
//! production loop stays serial. This example is kept as the reproducible
|
||||
//! evidence for that rejection — re-run it if the DB ever moves behind real
|
||||
//! network latency, where the answer could flip.
|
||||
//!
|
||||
//! `FavoritesService::batch_add_to_favorites` pre-checks `Permission::Read`
|
||||
//! on every referenced resource. BEFORE awaited the checks one-by-one: for a
|
||||
//! "select all → add to favorites" over N items whose drive-lookup isn't
|
||||
//! cached yet, that is N sequential point-SELECT round-trips
|
||||
//! (`drive_of` per distinct file) before the batched insert even starts.
|
||||
//! AFTER fans the same checks out with `futures::future::try_join_all`
|
||||
//! (fail-fast on any denial preserved).
|
||||
//!
|
||||
//! This bench drives the REAL `PgAclEngine` (owner/drive-role caches
|
||||
//! included) against a seeded shared drive:
|
||||
//! caller ──editor grant──▶ drive ─▶ root folder ─▶ N files
|
||||
//!
|
||||
//! Arms: cold engine (empty caches — the first-grid-load shape) and warm
|
||||
//! repeat (all moka — parity check, both arms should collapse).
|
||||
//!
|
||||
//! Equivalence gates: every check grants for the member on both arms, and
|
||||
//! both arms deny a control user with no grant.
|
||||
//!
|
||||
//! Run (needs Postgres up; reads DATABASE_URL from .env):
|
||||
//! cargo run --release --features bench --example bench_favorites_authz
|
||||
//! Tunables (env): BENCH_FILES (200), BENCH_POOL (20).
|
||||
|
||||
use std::env;
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use oxicloud::application::ports::authorization_ports::AuthorizationEngine;
|
||||
use oxicloud::domain::services::authorization::{Permission, Resource, Subject};
|
||||
use oxicloud::infrastructure::repositories::pg::{
|
||||
FileBlobReadRepository, FolderDbRepository, SubjectGroupPgRepository,
|
||||
};
|
||||
use oxicloud::infrastructure::services::dedup_service::DedupService;
|
||||
use oxicloud::infrastructure::services::local_blob_backend::LocalBlobBackend;
|
||||
use oxicloud::infrastructure::services::pg_acl_engine::PgAclEngine;
|
||||
use sqlx::PgPool;
|
||||
use sqlx::postgres::PgPoolOptions;
|
||||
use uuid::Uuid;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
struct Seeded {
|
||||
caller: Uuid,
|
||||
control: Uuid,
|
||||
drive_id: Uuid,
|
||||
root_folder: Uuid,
|
||||
blob_hash: String,
|
||||
file_ids: Vec<Uuid>,
|
||||
}
|
||||
|
||||
async fn seed(pool: &PgPool, n_files: usize) -> Seeded {
|
||||
let mut tx = pool.begin().await.expect("begin");
|
||||
let caller: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO auth.users (username, email, role)
|
||||
VALUES ('bench_favauthz', 'bench_favauthz@bench.invalid', 'user') RETURNING id",
|
||||
)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("seed caller");
|
||||
let control: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO auth.users (username, email, role)
|
||||
VALUES ('bench_favauthz_ctl', 'bench_favauthz_ctl@bench.invalid', 'user') RETURNING id",
|
||||
)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("seed control");
|
||||
|
||||
let drive_id: Uuid =
|
||||
sqlx::query_scalar("INSERT INTO storage.drives (kind) VALUES ('shared') RETURNING id")
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("seed drive");
|
||||
let root_folder: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO storage.folders (name, path, lpath, drive_id)
|
||||
VALUES ('Bench Shared', '/Bench Shared', 'x', $1) RETURNING id",
|
||||
)
|
||||
.bind(drive_id)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("seed folder");
|
||||
sqlx::query("UPDATE storage.drives SET root_folder_id = $1 WHERE id = $2")
|
||||
.bind(root_folder)
|
||||
.bind(drive_id)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.expect("stamp root");
|
||||
sqlx::query(
|
||||
"INSERT INTO storage.role_grants
|
||||
(subject_type, subject_id, resource_type, resource_id, role, granted_by)
|
||||
VALUES ('user', $1, 'drive', $2, 'editor'::storage.grant_role, $1)",
|
||||
)
|
||||
.bind(caller)
|
||||
.bind(drive_id)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.expect("seed grant");
|
||||
|
||||
let blob_hash = "benchfavauthz0000000000000000000000000000000000000000000000000b1".to_string();
|
||||
sqlx::query("INSERT INTO storage.blobs (hash, size, ref_count) VALUES ($1, 1, 1)")
|
||||
.bind(&blob_hash)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.expect("seed blob");
|
||||
|
||||
let mut file_ids = Vec::with_capacity(n_files);
|
||||
for i in 0..n_files {
|
||||
let id: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO storage.files (name, folder_id, blob_hash, size, mime_type, drive_id)
|
||||
VALUES ($1, $2, $3, 1, 'text/plain', $4) RETURNING id",
|
||||
)
|
||||
.bind(format!("bench-{i:04}.txt"))
|
||||
.bind(root_folder)
|
||||
.bind(&blob_hash)
|
||||
.bind(drive_id)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("seed file");
|
||||
file_ids.push(id);
|
||||
}
|
||||
tx.commit().await.expect("commit");
|
||||
Seeded {
|
||||
caller,
|
||||
control,
|
||||
drive_id,
|
||||
root_folder,
|
||||
blob_hash,
|
||||
file_ids,
|
||||
}
|
||||
}
|
||||
|
||||
async fn cleanup(pool: &PgPool, s: &Seeded) {
|
||||
let _ = sqlx::query("DELETE FROM storage.role_grants WHERE resource_id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await;
|
||||
let _ = sqlx::query("DELETE FROM storage.files WHERE drive_id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await;
|
||||
let _ = sqlx::query("DELETE FROM storage.drives WHERE id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await;
|
||||
let _ = sqlx::query("DELETE FROM storage.folders WHERE id = $1")
|
||||
.bind(s.root_folder)
|
||||
.execute(pool)
|
||||
.await;
|
||||
let _ = sqlx::query("DELETE FROM storage.blobs WHERE hash = $1")
|
||||
.bind(&s.blob_hash)
|
||||
.execute(pool)
|
||||
.await;
|
||||
let _ = sqlx::query("DELETE FROM auth.users WHERE id IN ($1, $2)")
|
||||
.bind(s.caller)
|
||||
.bind(s.control)
|
||||
.execute(pool)
|
||||
.await;
|
||||
}
|
||||
|
||||
fn fresh_engine(pool: &Arc<PgPool>) -> Arc<PgAclEngine> {
|
||||
let folder_repo = Arc::new(FolderDbRepository::new(pool.clone()));
|
||||
let backend = Arc::new(LocalBlobBackend::new(std::path::Path::new(
|
||||
"/tmp/bench-favauthz-blobs",
|
||||
)));
|
||||
let dedup = Arc::new(DedupService::new(backend, pool.clone(), pool.clone()));
|
||||
let file_repo = Arc::new(FileBlobReadRepository::new(
|
||||
pool.clone(),
|
||||
dedup,
|
||||
folder_repo.clone(),
|
||||
));
|
||||
let group_repo = Arc::new(SubjectGroupPgRepository::new(pool.clone()));
|
||||
Arc::new(PgAclEngine::new(
|
||||
pool.clone(),
|
||||
folder_repo,
|
||||
file_repo,
|
||||
group_repo,
|
||||
))
|
||||
}
|
||||
|
||||
/// BEFORE, verbatim shape: one awaited `require` per item.
|
||||
async fn serial_checks(engine: &Arc<PgAclEngine>, user: Uuid, files: &[Uuid]) -> Result<(), ()> {
|
||||
for id in files {
|
||||
engine
|
||||
.require(Subject::User(user), Permission::Read, Resource::File(*id))
|
||||
.await
|
||||
.map_err(|_| ())?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// AFTER: the same checks, fanned out with fail-fast join.
|
||||
async fn joined_checks(engine: &Arc<PgAclEngine>, user: Uuid, files: &[Uuid]) -> Result<(), ()> {
|
||||
futures::future::try_join_all(
|
||||
files
|
||||
.iter()
|
||||
.map(|id| engine.require(Subject::User(user), Permission::Read, Resource::File(*id))),
|
||||
)
|
||||
.await
|
||||
.map(|_| ())
|
||||
.map_err(|_| ())
|
||||
}
|
||||
|
||||
#[tokio::main(flavor = "multi_thread")]
|
||||
async fn main() {
|
||||
dotenvy::dotenv().ok();
|
||||
let url = env::var("DATABASE_URL")
|
||||
.or_else(|_| env::var("OXICLOUD_DB_CONNECTION_STRING"))
|
||||
.expect("set DATABASE_URL — the dev Postgres URL");
|
||||
let n_files: usize = env_or("BENCH_FILES", 200);
|
||||
let pool_size: u32 = env_or("BENCH_POOL", 20);
|
||||
|
||||
let pool = Arc::new(
|
||||
PgPoolOptions::new()
|
||||
.max_connections(pool_size)
|
||||
.min_connections(pool_size)
|
||||
.acquire_timeout(Duration::from_secs(10))
|
||||
.connect(&url)
|
||||
.await
|
||||
.expect("connect Postgres"),
|
||||
);
|
||||
|
||||
let seeded = seed(&pool, n_files).await;
|
||||
|
||||
// ── Equivalence gates ────────────────────────────────────────────────
|
||||
// Grant path: both arms must authorize every file for the member.
|
||||
let gate_engine = fresh_engine(&pool);
|
||||
if serial_checks(&gate_engine, seeded.caller, &seeded.file_ids)
|
||||
.await
|
||||
.is_err()
|
||||
|| joined_checks(&gate_engine, seeded.caller, &seeded.file_ids)
|
||||
.await
|
||||
.is_err()
|
||||
{
|
||||
eprintln!("EQUIVALENCE GATE FAILED: member was denied");
|
||||
cleanup(&pool, &seeded).await;
|
||||
std::process::exit(1);
|
||||
}
|
||||
// Denial path: both arms must reject the control user (fresh engines so
|
||||
// the joined arm can't ride the serial arm's caches).
|
||||
let deny_a = fresh_engine(&pool);
|
||||
let deny_b = fresh_engine(&pool);
|
||||
if serial_checks(&deny_a, seeded.control, &seeded.file_ids)
|
||||
.await
|
||||
.is_ok()
|
||||
|| joined_checks(&deny_b, seeded.control, &seeded.file_ids)
|
||||
.await
|
||||
.is_ok()
|
||||
{
|
||||
eprintln!("EQUIVALENCE GATE FAILED: control user was granted");
|
||||
cleanup(&pool, &seeded).await;
|
||||
std::process::exit(1);
|
||||
}
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# batch-favorites authz: serial require loop vs try_join_all");
|
||||
println!("# files={n_files} pool={pool_size} (shared-drive member, editor grant)");
|
||||
println!("#################################################################\n");
|
||||
println!("| {:<18} | {:>10} | {:>12} |", "arm", "wall ms", "µs/item");
|
||||
|
||||
for (label, joined, warm) in [
|
||||
("serial COLD", false, false),
|
||||
("join COLD", true, false),
|
||||
("serial WARM", false, true),
|
||||
("join WARM", true, true),
|
||||
] {
|
||||
// COLD: fresh engine per run (empty moka). WARM: prime, then measure.
|
||||
let engine = fresh_engine(&pool);
|
||||
if warm {
|
||||
serial_checks(&engine, seeded.caller, &seeded.file_ids)
|
||||
.await
|
||||
.expect("prime");
|
||||
}
|
||||
let t = Instant::now();
|
||||
let r = if joined {
|
||||
joined_checks(&engine, seeded.caller, &seeded.file_ids).await
|
||||
} else {
|
||||
serial_checks(&engine, seeded.caller, &seeded.file_ids).await
|
||||
};
|
||||
let el = t.elapsed();
|
||||
r.expect("granted");
|
||||
println!(
|
||||
"| {:<18} | {:>10.2} | {:>12.2} |",
|
||||
label,
|
||||
el.as_secs_f64() * 1e3,
|
||||
el.as_secs_f64() * 1e6 / n_files as f64
|
||||
);
|
||||
}
|
||||
|
||||
cleanup(&pool, &seeded).await;
|
||||
println!("\n(COLD = empty caches: N distinct `drive_of` point-SELECTs — the arm");
|
||||
println!(" under test. WARM = all-moka parity check. Fail-fast denial semantics");
|
||||
println!(" verified by the control-user gate on both arms.)");
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
//! Micro-alloc benchmark: digest-hex rendering and NC id-batch marshalling.
|
||||
//!
|
||||
//! Two round-6 changes, both equivalence-gated against their verbatim
|
||||
//! BEFORE shapes and measured with a counting allocator:
|
||||
//!
|
||||
//! 1. `IncrementalHasher::finalize_hex` (upload_ingest.rs) rendered MD5 /
|
||||
//! SHA-256 digests with `.map(|b| format!("{b:02x}")).collect()` — one
|
||||
//! heap `String` per digest byte (16 / 32 allocs) per chunk finalize.
|
||||
//! AFTER: `common::fmt::hex_lower` writes into one preallocated String.
|
||||
//!
|
||||
//! 2. `batch_resolve_ids` (NC webdav_handler) cloned every child id into a
|
||||
//! `Vec<String>` and the id service keyed its result map by `String` —
|
||||
//! ~3 heap allocs per child per page. AFTER the whole chain is borrowed:
|
||||
//! `Vec<&str>` in, `HashMap<Uuid, i64>` out, `Uuid::parse_str` lookups.
|
||||
//!
|
||||
//! Run:
|
||||
//! cargo run --release --features bench --example bench_hex_ids
|
||||
//! Tunables (env): BENCH_ITERS (10000), BENCH_CHILDREN (500).
|
||||
|
||||
use std::alloc::{GlobalAlloc, Layout, System};
|
||||
use std::collections::HashMap;
|
||||
use std::env;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
use std::time::Instant;
|
||||
|
||||
use md5::Digest;
|
||||
use oxicloud::common::fmt::hex_lower;
|
||||
use uuid::Uuid;
|
||||
|
||||
static ALLOC_CALLS: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
struct CountingAlloc;
|
||||
|
||||
unsafe impl GlobalAlloc for CountingAlloc {
|
||||
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.alloc(layout) }
|
||||
}
|
||||
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
|
||||
unsafe { System.dealloc(ptr, layout) }
|
||||
}
|
||||
unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.realloc(ptr, layout, new_size) }
|
||||
}
|
||||
unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
|
||||
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.alloc_zeroed(layout) }
|
||||
}
|
||||
}
|
||||
|
||||
#[global_allocator]
|
||||
static GLOBAL: CountingAlloc = CountingAlloc;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
fn measure<R>(f: impl FnOnce() -> R) -> (R, u64, f64) {
|
||||
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
|
||||
let t = Instant::now();
|
||||
let r = f();
|
||||
let el = t.elapsed().as_secs_f64();
|
||||
let allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
|
||||
(r, allocs, el)
|
||||
}
|
||||
|
||||
// ── 1. digest hex ───────────────────────────────────────────────────────────
|
||||
|
||||
/// BEFORE, verbatim: one `format!` per digest byte.
|
||||
fn hex_before(bytes: &[u8]) -> String {
|
||||
bytes.iter().map(|b| format!("{b:02x}")).collect()
|
||||
}
|
||||
|
||||
fn bench_hex(iters: usize) {
|
||||
// Deterministic digests of both production sizes (MD5=16, SHA-256=32).
|
||||
let md5s: Vec<[u8; 16]> = (0..64u64)
|
||||
.map(|i| md5::Md5::digest(i.to_le_bytes()).into())
|
||||
.collect();
|
||||
let sha256s: Vec<[u8; 32]> = (0..64u64)
|
||||
.map(|i| sha2::Sha256::digest(i.to_le_bytes()).into())
|
||||
.collect();
|
||||
|
||||
// Equivalence gate: byte-identical output on every digest.
|
||||
for d in &md5s {
|
||||
assert_eq!(hex_lower(d), hex_before(d), "md5 hex mismatch");
|
||||
}
|
||||
for d in &sha256s {
|
||||
assert_eq!(hex_lower(d), hex_before(d), "sha256 hex mismatch");
|
||||
}
|
||||
|
||||
println!("── finalize_hex: per-byte format! vs hex_lower ({iters} finalizes/arm) ──\n");
|
||||
println!(
|
||||
"| {:<8} | {:<8} | {:>12} | {:>10} | {:>12} |",
|
||||
"digest", "arm", "allocs", "wall ms", "allocs/call"
|
||||
);
|
||||
for (label, digests) in [("md5", md5s.len()), ("sha256", sha256s.len())] {
|
||||
for arm in ["BEFORE", "AFTER"] {
|
||||
let (sink, allocs, secs) = measure(|| {
|
||||
let mut sink = 0usize;
|
||||
for i in 0..iters {
|
||||
let s = match (label, arm) {
|
||||
("md5", "BEFORE") => hex_before(&md5s[i % digests]),
|
||||
("md5", "AFTER") => hex_lower(&md5s[i % digests]),
|
||||
("sha256", "BEFORE") => hex_before(&sha256s[i % digests]),
|
||||
_ => hex_lower(&sha256s[i % digests]),
|
||||
};
|
||||
sink += s.len();
|
||||
}
|
||||
sink
|
||||
});
|
||||
std::hint::black_box(sink);
|
||||
println!(
|
||||
"| {:<8} | {:<8} | {:>12} | {:>10.2} | {:>12.2} |",
|
||||
label,
|
||||
arm,
|
||||
allocs,
|
||||
secs * 1e3,
|
||||
allocs as f64 / iters as f64
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── 2. NC id-batch marshalling ──────────────────────────────────────────────
|
||||
|
||||
/// BEFORE, verbatim caller+service marshalling: clone ids into `Vec<String>`,
|
||||
/// key the result map by cloned `String`, look children up by `&String`.
|
||||
fn ids_before(child_ids: &[String], nc: &HashMap<Uuid, i64>) -> Vec<Option<i64>> {
|
||||
let file_uuids: Vec<String> = child_ids.to_vec();
|
||||
let mut map: HashMap<String, i64> = HashMap::with_capacity(file_uuids.len());
|
||||
for raw in &file_uuids {
|
||||
let Ok(uuid) = Uuid::parse_str(raw) else {
|
||||
continue;
|
||||
};
|
||||
if let Some(id) = nc.get(&uuid) {
|
||||
map.insert(raw.clone(), *id);
|
||||
}
|
||||
}
|
||||
child_ids.iter().map(|id| map.get(id).copied()).collect()
|
||||
}
|
||||
|
||||
/// AFTER: borrowed slice in, `Uuid`-keyed map out, parse-and-get lookups —
|
||||
/// the exact shapes now in `batch_resolve_ids` + `nc_id_of`.
|
||||
fn ids_after(child_ids: &[String], nc: &HashMap<Uuid, i64>) -> Vec<Option<i64>> {
|
||||
let file_uuids: Vec<&str> = child_ids.iter().map(String::as_str).collect();
|
||||
let mut map: HashMap<Uuid, i64> = HashMap::with_capacity(file_uuids.len());
|
||||
for raw in &file_uuids {
|
||||
let Ok(uuid) = Uuid::parse_str(raw) else {
|
||||
continue;
|
||||
};
|
||||
if let Some(id) = nc.get(&uuid) {
|
||||
map.insert(uuid, *id);
|
||||
}
|
||||
}
|
||||
child_ids
|
||||
.iter()
|
||||
.map(|id| Uuid::parse_str(id).ok().and_then(|u| map.get(&u).copied()))
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn bench_ids(pages: usize, children: usize) {
|
||||
// A PROPFIND page of `children` DTO ids (36-byte uuid strings) resolved
|
||||
// against the id service's numeric mapping.
|
||||
let uuids: Vec<Uuid> = (0..children).map(|_| Uuid::new_v4()).collect();
|
||||
let child_ids: Vec<String> = uuids.iter().map(|u| u.to_string()).collect();
|
||||
let nc: HashMap<Uuid, i64> = uuids
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, u)| (*u, i as i64 + 1000))
|
||||
.collect();
|
||||
|
||||
// Equivalence gate: identical per-child resolution, including an
|
||||
// unparseable id and an unmapped-but-valid id.
|
||||
let mut gate_ids = child_ids.clone();
|
||||
gate_ids.push("not-a-uuid".to_string());
|
||||
gate_ids.push(Uuid::new_v4().to_string());
|
||||
assert_eq!(
|
||||
ids_before(&gate_ids, &nc),
|
||||
ids_after(&gate_ids, &nc),
|
||||
"id resolution mismatch"
|
||||
);
|
||||
|
||||
println!("\n── batch_resolve_ids marshalling: String-keyed vs borrowed+Uuid ──");
|
||||
println!(" ({pages} pages × {children} children/arm)\n");
|
||||
println!(
|
||||
"| {:<8} | {:>12} | {:>10} | {:>14} |",
|
||||
"arm", "allocs", "wall ms", "allocs/child"
|
||||
);
|
||||
for arm in ["BEFORE", "AFTER"] {
|
||||
let (sink, allocs, secs) = measure(|| {
|
||||
let mut sink = 0usize;
|
||||
for _ in 0..pages {
|
||||
let resolved = if arm == "BEFORE" {
|
||||
ids_before(&child_ids, &nc)
|
||||
} else {
|
||||
ids_after(&child_ids, &nc)
|
||||
};
|
||||
sink += resolved.iter().flatten().count();
|
||||
}
|
||||
sink
|
||||
});
|
||||
assert_eq!(sink, pages * children, "all children must resolve");
|
||||
println!(
|
||||
"| {:<8} | {:>12} | {:>10.2} | {:>14.3} |",
|
||||
arm,
|
||||
allocs,
|
||||
secs * 1e3,
|
||||
allocs as f64 / (pages * children) as f64
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let iters: usize = env_or("BENCH_ITERS", 10_000);
|
||||
let children: usize = env_or("BENCH_CHILDREN", 500);
|
||||
|
||||
bench_hex(iters);
|
||||
bench_ids(iters / 10, children);
|
||||
|
||||
println!("\n(BEFORE arms are verbatim replicas of the replaced shapes; equivalence");
|
||||
println!(" asserted before timing. Allocs counted via a wrapping GlobalAlloc.)");
|
||||
}
|
||||
@@ -0,0 +1,248 @@
|
||||
//! A/B: `id::text` server-side casts vs binary UUID decode + app-side format.
|
||||
//!
|
||||
//! `file_blob_read_repository.rs` (and friends) SELECT UUID columns as
|
||||
//! `id::text` and decode `String`s directly. The alternative is to decode the
|
||||
//! wire-native binary `Uuid` (16 bytes vs 36 on the wire) and render the
|
||||
//! string app-side with `Uuid::to_string`. This bench decides ROUND6 task
|
||||
//! "::text casts A/B" empirically: whichever loses is documented, only a
|
||||
//! winner ships.
|
||||
//!
|
||||
//! Arms fetch the same 500-row page from a seeded `storage.files` subtree,
|
||||
//! interleaved A/B to cancel drift; the equivalence gate asserts identical
|
||||
//! `(id, folder_id, name)` string triples.
|
||||
//!
|
||||
//! Run (needs Postgres up; reads DATABASE_URL from .env):
|
||||
//! cargo run --release --features bench --example bench_uuid_text_cast
|
||||
//! Tunables (env): BENCH_ROWS (500), BENCH_PASSES (200).
|
||||
|
||||
use std::env;
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use sqlx::postgres::PgPoolOptions;
|
||||
use sqlx::{PgPool, Row};
|
||||
use uuid::Uuid;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
struct Seeded {
|
||||
drive_id: Uuid,
|
||||
root_folder: Uuid,
|
||||
blob_hash: String,
|
||||
}
|
||||
|
||||
async fn seed(pool: &PgPool, rows: usize) -> Seeded {
|
||||
let mut tx = pool.begin().await.expect("begin");
|
||||
let drive_id: Uuid =
|
||||
sqlx::query_scalar("INSERT INTO storage.drives (kind) VALUES ('shared') RETURNING id")
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("seed drive");
|
||||
let root_folder: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO storage.folders (name, path, lpath, drive_id)
|
||||
VALUES ('Bench Cast', '/Bench Cast', 'x', $1) RETURNING id",
|
||||
)
|
||||
.bind(drive_id)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("seed folder");
|
||||
sqlx::query("UPDATE storage.drives SET root_folder_id = $1 WHERE id = $2")
|
||||
.bind(root_folder)
|
||||
.bind(drive_id)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.expect("stamp root");
|
||||
let blob_hash = "benchuuidcast000000000000000000000000000000000000000000000000b2".to_string();
|
||||
sqlx::query("INSERT INTO storage.blobs (hash, size, ref_count) VALUES ($1, 1, 1)")
|
||||
.bind(&blob_hash)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.expect("seed blob");
|
||||
for i in 0..rows {
|
||||
sqlx::query(
|
||||
"INSERT INTO storage.files (name, folder_id, blob_hash, size, mime_type, drive_id)
|
||||
VALUES ($1, $2, $3, 1, 'text/plain', $4)",
|
||||
)
|
||||
.bind(format!("cast-{i:05}.txt"))
|
||||
.bind(root_folder)
|
||||
.bind(&blob_hash)
|
||||
.bind(drive_id)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.expect("seed file");
|
||||
}
|
||||
tx.commit().await.expect("commit");
|
||||
Seeded {
|
||||
drive_id,
|
||||
root_folder,
|
||||
blob_hash,
|
||||
}
|
||||
}
|
||||
|
||||
async fn cleanup(pool: &PgPool, s: &Seeded) {
|
||||
let _ = sqlx::query("DELETE FROM storage.files WHERE drive_id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await;
|
||||
let _ = sqlx::query("DELETE FROM storage.drives WHERE id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await;
|
||||
let _ = sqlx::query("DELETE FROM storage.folders WHERE id = $1")
|
||||
.bind(s.root_folder)
|
||||
.execute(pool)
|
||||
.await;
|
||||
let _ = sqlx::query("DELETE FROM storage.blobs WHERE hash = $1")
|
||||
.bind(&s.blob_hash)
|
||||
.execute(pool)
|
||||
.await;
|
||||
}
|
||||
|
||||
type Triple = (String, Option<String>, String);
|
||||
|
||||
/// Arm A — the current production shape: server-side `::text` casts.
|
||||
async fn fetch_text_cast(pool: &PgPool, drive_id: Uuid) -> Vec<Triple> {
|
||||
sqlx::query(
|
||||
"SELECT id::text AS id, folder_id::text AS folder_id, name
|
||||
FROM storage.files WHERE drive_id = $1 ORDER BY name",
|
||||
)
|
||||
.bind(drive_id)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.expect("text-cast fetch")
|
||||
.iter()
|
||||
.map(|r| {
|
||||
(
|
||||
r.get::<String, _>("id"),
|
||||
r.get::<Option<String>, _>("folder_id"),
|
||||
r.get::<String, _>("name"),
|
||||
)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Arm B — binary `Uuid` decode + app-side `to_string`.
|
||||
async fn fetch_binary_uuid(pool: &PgPool, drive_id: Uuid) -> Vec<Triple> {
|
||||
sqlx::query(
|
||||
"SELECT id, folder_id, name
|
||||
FROM storage.files WHERE drive_id = $1 ORDER BY name",
|
||||
)
|
||||
.bind(drive_id)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.expect("binary fetch")
|
||||
.iter()
|
||||
.map(|r| {
|
||||
(
|
||||
r.get::<Uuid, _>("id").to_string(),
|
||||
r.get::<Option<Uuid>, _>("folder_id").map(|u| u.to_string()),
|
||||
r.get::<String, _>("name"),
|
||||
)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
struct Stats {
|
||||
mean_ms: f64,
|
||||
p50_ms: f64,
|
||||
p95_ms: f64,
|
||||
}
|
||||
|
||||
fn summarize(mut xs: Vec<f64>) -> Stats {
|
||||
xs.sort_by(|a, b| a.partial_cmp(b).unwrap());
|
||||
let n = xs.len();
|
||||
Stats {
|
||||
mean_ms: xs.iter().sum::<f64>() / n as f64,
|
||||
p50_ms: xs[n / 2],
|
||||
p95_ms: xs[((n as f64 * 0.95) as usize).min(n - 1)],
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::main(flavor = "multi_thread")]
|
||||
async fn main() {
|
||||
dotenvy::dotenv().ok();
|
||||
let url = env::var("DATABASE_URL")
|
||||
.or_else(|_| env::var("OXICLOUD_DB_CONNECTION_STRING"))
|
||||
.expect("set DATABASE_URL — the dev Postgres URL");
|
||||
let rows: usize = env_or("BENCH_ROWS", 500);
|
||||
let passes: usize = env_or("BENCH_PASSES", 200);
|
||||
|
||||
let pool = Arc::new(
|
||||
PgPoolOptions::new()
|
||||
.max_connections(4)
|
||||
.min_connections(4)
|
||||
.acquire_timeout(Duration::from_secs(10))
|
||||
.connect(&url)
|
||||
.await
|
||||
.expect("connect Postgres"),
|
||||
);
|
||||
|
||||
let seeded = seed(&pool, rows).await;
|
||||
|
||||
// ── Equivalence gate: identical string triples ───────────────────────
|
||||
let a = fetch_text_cast(&pool, seeded.drive_id).await;
|
||||
let b = fetch_binary_uuid(&pool, seeded.drive_id).await;
|
||||
if a != b || a.len() != rows {
|
||||
eprintln!(
|
||||
"EQUIVALENCE GATE FAILED: rows differ (a={}, b={})",
|
||||
a.len(),
|
||||
b.len()
|
||||
);
|
||||
cleanup(&pool, &seeded).await;
|
||||
std::process::exit(1);
|
||||
}
|
||||
|
||||
// Warm-up both shapes (plan cache, buffer cache).
|
||||
for _ in 0..10 {
|
||||
std::hint::black_box(fetch_text_cast(&pool, seeded.drive_id).await);
|
||||
std::hint::black_box(fetch_binary_uuid(&pool, seeded.drive_id).await);
|
||||
}
|
||||
|
||||
// Interleaved A/B passes so drift (autovacuum, CPU governor) hits both.
|
||||
let mut lat_a = Vec::with_capacity(passes);
|
||||
let mut lat_b = Vec::with_capacity(passes);
|
||||
for _ in 0..passes {
|
||||
let t = Instant::now();
|
||||
std::hint::black_box(fetch_text_cast(&pool, seeded.drive_id).await);
|
||||
lat_a.push(t.elapsed().as_secs_f64() * 1e3);
|
||||
let t = Instant::now();
|
||||
std::hint::black_box(fetch_binary_uuid(&pool, seeded.drive_id).await);
|
||||
lat_b.push(t.elapsed().as_secs_f64() * 1e3);
|
||||
}
|
||||
|
||||
let sa = summarize(lat_a);
|
||||
let sb = summarize(lat_b);
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# UUID columns: `id::text` server cast vs binary decode + app fmt");
|
||||
println!("# rows/page={rows} passes={passes} (interleaved)");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<22} | {:>9} | {:>9} | {:>9} |",
|
||||
"arm", "mean ms", "p50 ms", "p95 ms"
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>9.3} | {:>9.3} | {:>9.3} |",
|
||||
"A ::text (current)", sa.mean_ms, sa.p50_ms, sa.p95_ms
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>9.3} | {:>9.3} | {:>9.3} |",
|
||||
"B binary + to_string", sb.mean_ms, sb.p50_ms, sb.p95_ms
|
||||
);
|
||||
println!(
|
||||
"\nB/A mean ratio: {:.3} ({})",
|
||||
sb.mean_ms / sa.mean_ms,
|
||||
if sb.mean_ms < sa.mean_ms {
|
||||
"binary decode wins"
|
||||
} else {
|
||||
"::text cast wins"
|
||||
}
|
||||
);
|
||||
|
||||
cleanup(&pool, &seeded).await;
|
||||
}
|
||||
@@ -278,6 +278,27 @@ impl CardDavAdapter {
|
||||
) -> Result<()> {
|
||||
let mut xml_writer = Writer::new(writer);
|
||||
|
||||
Self::write_collection_head(&mut xml_writer, address_book, request, base_href)?;
|
||||
|
||||
// Write contacts if depth > 0
|
||||
if depth != "0" {
|
||||
Self::write_collection_contact_page(&mut xml_writer, contacts, base_href)?;
|
||||
}
|
||||
|
||||
Self::write_carddav_multistatus_end(&mut xml_writer)
|
||||
}
|
||||
|
||||
/// Multistatus opening (DAV + CardDAV + CalendarServer namespaces)
|
||||
/// plus the address book's own `D:response` — the head of a depth-1
|
||||
/// collection PROPFIND. Streaming emitters call this once, then
|
||||
/// [`Self::write_collection_contact_page`] per cursor page, then
|
||||
/// [`Self::write_carddav_multistatus_end`].
|
||||
pub fn write_collection_head<W: Write>(
|
||||
xml_writer: &mut Writer<W>,
|
||||
address_book: &AddressBookDto,
|
||||
request: &PropFindRequest,
|
||||
base_href: &str,
|
||||
) -> Result<()> {
|
||||
xml_writer.write_event(Event::Start(
|
||||
BytesStart::new("D:multistatus").with_attributes([
|
||||
("xmlns:D", "DAV:"),
|
||||
@@ -285,19 +306,25 @@ impl CardDavAdapter {
|
||||
("xmlns:CS", "http://calendarserver.org/ns/"),
|
||||
]),
|
||||
))?;
|
||||
Self::write_addressbook_response(xml_writer, address_book, request, base_href)
|
||||
}
|
||||
|
||||
// Write the address book itself
|
||||
Self::write_addressbook_response(&mut xml_writer, address_book, request, base_href)?;
|
||||
|
||||
// Write contacts if depth > 0
|
||||
if depth != "0" {
|
||||
for contact in contacts {
|
||||
let contact_href = format!("{}{}.vcf", base_href, contact.uid);
|
||||
Self::write_contact_response(&mut xml_writer, contact, &[], &contact_href)?;
|
||||
}
|
||||
/// One depth-1 collection page of contact entries (standard props;
|
||||
/// href buffer reused across the page).
|
||||
pub fn write_collection_contact_page<W: Write>(
|
||||
xml_writer: &mut Writer<W>,
|
||||
contacts: &[ContactDto],
|
||||
base_href: &str,
|
||||
) -> Result<()> {
|
||||
let mut href = String::with_capacity(base_href.len() + 48);
|
||||
for contact in contacts {
|
||||
href.clear();
|
||||
let _ = std::fmt::Write::write_fmt(
|
||||
&mut href,
|
||||
format_args!("{}{}.vcf", base_href, contact.uid),
|
||||
);
|
||||
Self::write_contact_response(xml_writer, contact, &[], &href)?;
|
||||
}
|
||||
|
||||
xml_writer.write_event(Event::End(BytesEnd::new("D:multistatus")))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -648,32 +675,39 @@ impl CardDavAdapter {
|
||||
}
|
||||
|
||||
/// Generate response for contacts (for REPORT)
|
||||
pub fn generate_contacts_response<W: Write>(
|
||||
writer: W,
|
||||
contacts: &[ContactDto],
|
||||
report: &CardDavReportType,
|
||||
base_href: &str,
|
||||
) -> Result<()> {
|
||||
let mut xml_writer = Writer::new(writer);
|
||||
|
||||
/// REPORT `<D:multistatus>` opening tag (DAV + CardDAV namespaces).
|
||||
/// Streaming emitters call this once, then
|
||||
/// [`Self::write_contacts_report_page`] per cursor page, then
|
||||
/// [`Self::write_carddav_multistatus_end`].
|
||||
pub fn write_report_multistatus_start<W: Write>(xml_writer: &mut Writer<W>) -> Result<()> {
|
||||
xml_writer.write_event(Event::Start(
|
||||
BytesStart::new("D:multistatus").with_attributes([
|
||||
("xmlns:D", "DAV:"),
|
||||
("xmlns:CR", "urn:ietf:params:xml:ns:carddav"),
|
||||
]),
|
||||
))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// Borrowed straight out of the request — the old `clone()` copied
|
||||
// the whole Vec of owned QualifiedName strings per REPORT (same
|
||||
// fix the CalDAV surface got in ROUND4).
|
||||
/// Close a multistatus opened by either start writer.
|
||||
pub fn write_carddav_multistatus_end<W: Write>(xml_writer: &mut Writer<W>) -> Result<()> {
|
||||
xml_writer.write_event(Event::End(BytesEnd::new("D:multistatus")))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// One REPORT page of contact responses. Props are borrowed from
|
||||
/// the request; one href buffer is reused across the page.
|
||||
pub fn write_contacts_report_page<W: Write>(
|
||||
xml_writer: &mut Writer<W>,
|
||||
contacts: &[ContactDto],
|
||||
report: &CardDavReportType,
|
||||
base_href: &str,
|
||||
) -> Result<()> {
|
||||
let props = match report {
|
||||
CardDavReportType::AddressbookQuery { props } => props,
|
||||
CardDavReportType::AddressbookMultiget { props, .. } => props,
|
||||
CardDavReportType::SyncCollection { props, .. } => props,
|
||||
};
|
||||
|
||||
// One reused href buffer for the whole listing instead of a
|
||||
// fresh String per contact.
|
||||
let mut href = String::with_capacity(base_href.len() + 48);
|
||||
for contact in contacts {
|
||||
href.clear();
|
||||
@@ -683,13 +717,23 @@ impl CardDavAdapter {
|
||||
);
|
||||
// `write_contact_response` generates the vCard on demand when (and
|
||||
// only when) address-data is actually requested.
|
||||
Self::write_contact_response(&mut xml_writer, contact, props, &href)?;
|
||||
Self::write_contact_response(xml_writer, contact, props, &href)?;
|
||||
}
|
||||
|
||||
xml_writer.write_event(Event::End(BytesEnd::new("D:multistatus")))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn generate_contacts_response<W: Write>(
|
||||
writer: W,
|
||||
contacts: &[ContactDto],
|
||||
report: &CardDavReportType,
|
||||
base_href: &str,
|
||||
) -> Result<()> {
|
||||
let mut xml_writer = Writer::new(writer);
|
||||
Self::write_report_multistatus_start(&mut xml_writer)?;
|
||||
Self::write_contacts_report_page(&mut xml_writer, contacts, report, base_href)?;
|
||||
Self::write_carddav_multistatus_end(&mut xml_writer)
|
||||
}
|
||||
|
||||
/// Write a single contact response element
|
||||
fn write_contact_response<W: Write>(
|
||||
xml_writer: &mut Writer<W>,
|
||||
|
||||
@@ -66,6 +66,12 @@ pub trait ContactStoragePort: Send + Sync + 'static {
|
||||
&self,
|
||||
address_book_id: &Uuid,
|
||||
) -> Result<Vec<Contact>, DomainError>;
|
||||
/// Cursor stream over the book's contacts in listing order — feeds
|
||||
/// the streaming CardDAV emitters.
|
||||
fn stream_contacts_by_book(
|
||||
&self,
|
||||
address_book_id: Uuid,
|
||||
) -> futures::stream::BoxStream<'static, Result<Contact, DomainError>>;
|
||||
async fn get_contacts_by_address_book_paginated(
|
||||
&self,
|
||||
address_book_id: &Uuid,
|
||||
@@ -174,6 +180,15 @@ pub trait ContactUseCase: Send + Sync + 'static {
|
||||
/// List contacts in an address book. `limit`/`offset` bound the
|
||||
/// result for paginated callers (REST API); `None` returns the full
|
||||
/// book, which the CardDAV listing/sync paths rely on.
|
||||
/// Streaming support: cursor over the book's contacts (same Read
|
||||
/// gate as [`Self::list_contacts`], checked once before the cursor
|
||||
/// opens).
|
||||
async fn stream_contacts_by_book(
|
||||
&self,
|
||||
address_book_id: &str,
|
||||
user_id: Uuid,
|
||||
) -> Result<futures::stream::BoxStream<'static, Result<ContactDto, DomainError>>, DomainError>;
|
||||
|
||||
async fn list_contacts(
|
||||
&self,
|
||||
address_book_id: &str,
|
||||
|
||||
@@ -825,6 +825,25 @@ impl ContactUseCase for ContactService {
|
||||
Ok(contacts.into_iter().map(ContactDto::from).collect())
|
||||
}
|
||||
|
||||
async fn stream_contacts_by_book(
|
||||
&self,
|
||||
address_book_id: &str,
|
||||
user_id: Uuid,
|
||||
) -> Result<futures::stream::BoxStream<'static, Result<ContactDto, DomainError>>, DomainError>
|
||||
{
|
||||
use futures::StreamExt;
|
||||
let id = Uuid::parse_str(address_book_id)
|
||||
.map_err(|_| DomainError::validation_error("Invalid address book ID format"))?;
|
||||
// Same Read gate as `list_contacts`, once, before the cursor.
|
||||
self.require_address_book_read_or_public(&id, &user_id)
|
||||
.await?;
|
||||
Ok(Box::pin(
|
||||
self.contact_storage
|
||||
.stream_contacts_by_book(id)
|
||||
.map(|r| r.map(ContactDto::from)),
|
||||
))
|
||||
}
|
||||
|
||||
async fn list_contacts(
|
||||
&self,
|
||||
address_book_id: &str,
|
||||
|
||||
@@ -145,6 +145,12 @@ impl FavoritesUseCase for FavoritesService {
|
||||
// valid (partial success would leak the same oracle we
|
||||
// closed on the single-item path). See
|
||||
// `docs/plan/authz_audit/rest_storage.md`.
|
||||
//
|
||||
// Deliberately serial: a `try_join_all` fan-out measured WORSE
|
||||
// on both the cold (drive_of point-SELECTs) and warm (all-moka)
|
||||
// paths — future orchestration + pool-acquire contention cost
|
||||
// more than the local round trips they overlap. Rejected by
|
||||
// `bench_favorites_authz`; numbers in benches/ROUND6.md.
|
||||
for (item_id, item_type) in items {
|
||||
let resource = Resource::parse(item_type, item_id)?;
|
||||
self.authorization
|
||||
|
||||
@@ -41,55 +41,50 @@ impl NextcloudFileIdService {
|
||||
|
||||
/// Resolve — creating when absent — stable numeric file IDs for many
|
||||
/// UUIDs at once. Cache hits cost nothing; the misses are resolved with a
|
||||
/// single backing query. The returned map is keyed by the caller's
|
||||
/// original id strings; unresolvable inputs are simply absent (mirroring
|
||||
/// the `.ok()` behaviour the callers relied on).
|
||||
pub async fn get_or_create_file_ids(
|
||||
&self,
|
||||
file_ids: &[String],
|
||||
) -> Result<HashMap<String, i64>> {
|
||||
/// single backing query. The returned map is keyed by parsed UUID;
|
||||
/// unparseable/unresolvable inputs are simply absent (mirroring the
|
||||
/// `.ok()` behaviour the callers relied on).
|
||||
pub async fn get_or_create_file_ids(&self, file_ids: &[&str]) -> Result<HashMap<Uuid, i64>> {
|
||||
self.get_or_create_many("file", file_ids).await
|
||||
}
|
||||
|
||||
/// Folder counterpart of [`Self::get_or_create_file_ids`].
|
||||
pub async fn get_or_create_folder_ids(
|
||||
&self,
|
||||
folder_ids: &[String],
|
||||
) -> Result<HashMap<String, i64>> {
|
||||
folder_ids: &[&str],
|
||||
) -> Result<HashMap<Uuid, i64>> {
|
||||
self.get_or_create_many("folder", folder_ids).await
|
||||
}
|
||||
|
||||
async fn get_or_create_many(
|
||||
&self,
|
||||
object_type: &str,
|
||||
raw_ids: &[String],
|
||||
) -> Result<HashMap<String, i64>> {
|
||||
raw_ids: &[&str],
|
||||
) -> Result<HashMap<Uuid, i64>> {
|
||||
let mut result = HashMap::with_capacity(raw_ids.len());
|
||||
// Parsed-UUID → caller's original string; also dedupes the miss list.
|
||||
let mut pending: HashMap<Uuid, String> = HashMap::new();
|
||||
let mut misses: Vec<Uuid> = Vec::new();
|
||||
|
||||
for raw in raw_ids {
|
||||
let Ok(uuid) = Uuid::parse_str(raw) else {
|
||||
continue; // Unparseable ids never had a mapping — skip silently.
|
||||
};
|
||||
if let Some(id) = self.cache.get(&uuid).await {
|
||||
result.insert(raw.clone(), id);
|
||||
result.insert(uuid, id);
|
||||
} else {
|
||||
pending.entry(uuid).or_insert_with(|| raw.clone());
|
||||
misses.push(uuid);
|
||||
}
|
||||
}
|
||||
|
||||
if !pending.is_empty() {
|
||||
let misses: Vec<Uuid> = pending.keys().copied().collect();
|
||||
if !misses.is_empty() {
|
||||
misses.sort_unstable();
|
||||
misses.dedup();
|
||||
let resolved = self
|
||||
.repo()?
|
||||
.get_or_create_many(object_type, &misses)
|
||||
.await?;
|
||||
for (uuid, id) in resolved {
|
||||
self.cache.insert(uuid, id).await;
|
||||
if let Some(original) = pending.get(&uuid) {
|
||||
result.insert(original.clone(), id);
|
||||
}
|
||||
result.insert(uuid, id);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -184,10 +179,7 @@ mod tests {
|
||||
#[tokio::test]
|
||||
async fn test_get_or_create_file_ids_skips_unparseable() {
|
||||
let svc = NextcloudFileIdService::new_stub();
|
||||
let map = svc
|
||||
.get_or_create_file_ids(&["not-a-uuid".to_string()])
|
||||
.await
|
||||
.unwrap();
|
||||
let map = svc.get_or_create_file_ids(&["not-a-uuid"]).await.unwrap();
|
||||
assert!(map.is_empty());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -170,11 +170,43 @@ pub fn i64_str(buf: &mut [u8; 21], v: i64) -> &str {
|
||||
std::str::from_utf8(&buf[start..]).expect("ascii")
|
||||
}
|
||||
|
||||
/// Lower-case hex of `bytes` into one preallocated `String`.
|
||||
///
|
||||
/// Replaces the `.map(|b| format!("{b:02x}")).collect()` shape, which heap-
|
||||
/// allocates a 2-byte `String` per digest byte (16 for MD5, 32 for SHA-256)
|
||||
/// before collect concatenates them.
|
||||
pub fn hex_lower(bytes: &[u8]) -> String {
|
||||
const HEX: &[u8; 16] = b"0123456789abcdef";
|
||||
let mut out = String::with_capacity(bytes.len() * 2);
|
||||
for &b in bytes {
|
||||
out.push(HEX[(b >> 4) as usize] as char);
|
||||
out.push(HEX[(b & 0x0f) as usize] as char);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use chrono::{TimeZone, Utc};
|
||||
|
||||
/// `hex_lower` must match the `format!("{b:02x}")`-per-byte shape it
|
||||
/// replaced, byte for byte.
|
||||
#[test]
|
||||
fn hex_lower_matches_format() {
|
||||
let cases: [&[u8]; 5] = [
|
||||
&[],
|
||||
&[0x00],
|
||||
&[0xff, 0x00, 0xab],
|
||||
&(0u8..=255).collect::<Vec<u8>>(),
|
||||
b"The quick brown fox",
|
||||
];
|
||||
for bytes in cases {
|
||||
let reference: String = bytes.iter().map(|b| format!("{b:02x}")).collect();
|
||||
assert_eq!(hex_lower(bytes), reference);
|
||||
}
|
||||
}
|
||||
|
||||
/// Edge-heavy corpus: epoch, single-digit day (padding!), leap day,
|
||||
/// end-of-year, DST-irrelevant midsummer, far future, max in-range.
|
||||
const CASES: [i64; 12] = [
|
||||
|
||||
@@ -25,6 +25,14 @@ pub trait ContactRepository: Send + Sync + 'static {
|
||||
address_book_id: &Uuid,
|
||||
uids: &[String],
|
||||
) -> ContactRepositoryResult<Vec<Contact>>;
|
||||
/// Cursor stream over every contact of the book in the listing
|
||||
/// order (`full_name, first_name, last_name`) — ONE scan+sort on
|
||||
/// the server; the streaming CardDAV emitters page over it.
|
||||
fn stream_contacts_by_book(
|
||||
&self,
|
||||
address_book_id: Uuid,
|
||||
) -> futures::stream::BoxStream<'static, ContactRepositoryResult<Contact>>;
|
||||
|
||||
async fn get_contacts_by_address_book(
|
||||
&self,
|
||||
address_book_id: &Uuid,
|
||||
|
||||
@@ -146,6 +146,14 @@ impl ContactStoragePort for ContactStorageAdapter {
|
||||
.await
|
||||
}
|
||||
|
||||
fn stream_contacts_by_book(
|
||||
&self,
|
||||
address_book_id: Uuid,
|
||||
) -> futures::stream::BoxStream<'static, Result<Contact, DomainError>> {
|
||||
self.contact_repository
|
||||
.stream_contacts_by_book(address_book_id)
|
||||
}
|
||||
|
||||
async fn get_contacts_by_address_book_paginated(
|
||||
&self,
|
||||
address_book_id: &Uuid,
|
||||
|
||||
@@ -278,6 +278,45 @@ impl ContactRepository for ContactPgRepository {
|
||||
Ok(contacts)
|
||||
}
|
||||
|
||||
fn stream_contacts_by_book(
|
||||
&self,
|
||||
address_book_id: Uuid,
|
||||
) -> futures::stream::BoxStream<'static, ContactRepositoryResult<Contact>> {
|
||||
// ONE ordered scan served through a PG cursor — the CardDAV
|
||||
// multistatus emitters page over this stream so only a page of
|
||||
// contacts is resident (same design as the CalDAV round-5
|
||||
// cursor; contacts have no master/exception bundling, so pages
|
||||
// can cut anywhere).
|
||||
let pool = self.pool.clone();
|
||||
let stream: futures::stream::BoxStream<'static, ContactRepositoryResult<Contact>> =
|
||||
Box::pin(async_stream::try_stream! {
|
||||
let mut conn = pool.acquire().await.map_err(|e| {
|
||||
DomainError::database_error(format!("Failed to acquire connection: {}", e))
|
||||
})?;
|
||||
let mut rows = sqlx::query(
|
||||
r#"
|
||||
SELECT
|
||||
id, address_book_id, uid, full_name, first_name, last_name, nickname,
|
||||
email, phone, address, organization, title, notes, photo_url,
|
||||
birthday, anniversary, vcard, etag, created_at, updated_at
|
||||
FROM carddav.contacts
|
||||
WHERE address_book_id = $1
|
||||
ORDER BY full_name, first_name, last_name
|
||||
"#,
|
||||
)
|
||||
.bind(address_book_id)
|
||||
.fetch(&mut *conn);
|
||||
|
||||
use futures::TryStreamExt;
|
||||
while let Some(row) = rows.try_next().await.map_err(|e| {
|
||||
DomainError::database_error(format!("Failed to stream contacts: {}", e))
|
||||
})? {
|
||||
yield Self::row_to_contact(&row)?;
|
||||
}
|
||||
});
|
||||
stream
|
||||
}
|
||||
|
||||
async fn get_contacts_by_address_book(
|
||||
&self,
|
||||
address_book_id: &Uuid,
|
||||
|
||||
@@ -259,8 +259,9 @@ impl FavoritesRepositoryPort for FavoritesPgRepository {
|
||||
return Ok(HashSet::new());
|
||||
}
|
||||
|
||||
// Collect just the IDs for the IN clause
|
||||
let ids: Vec<String> = item_ids.iter().map(|(id, _)| id.to_string()).collect();
|
||||
// Collect just the IDs for the IN clause — sqlx binds `&[&str]` as
|
||||
// text[], so no per-id String is needed.
|
||||
let ids: Vec<&str> = item_ids.iter().map(|(id, _)| *id).collect();
|
||||
|
||||
let rows = sqlx::query(
|
||||
"SELECT item_id FROM auth.user_favorites WHERE user_id = $1 AND item_id = ANY($2)",
|
||||
|
||||
@@ -11,9 +11,9 @@
|
||||
/// Post-D7-step-6: `storage.files.user_id` dropped, so it's no
|
||||
/// longer projected.
|
||||
type MediaFileRow = (
|
||||
String, // id
|
||||
Uuid, // id (binary decode; benches/ROUND6.md §10)
|
||||
String, // name
|
||||
Option<String>, // folder_id
|
||||
Option<Uuid>, // folder_id
|
||||
Option<String>, // folder path
|
||||
i64, // size
|
||||
String, // mime_type
|
||||
@@ -83,9 +83,9 @@ const CALLER_CAN_READ_DRIVE: &str = "EXISTS (\
|
||||
/// longer part of the tuple; `row_to_file` populates the entity's
|
||||
/// legacy `user_id` field with `None`.
|
||||
type FileRow = (
|
||||
Uuid,
|
||||
String,
|
||||
String,
|
||||
Option<String>,
|
||||
Option<Uuid>,
|
||||
Option<String>,
|
||||
i64,
|
||||
String,
|
||||
@@ -269,7 +269,7 @@ impl FileBlobReadRepository {
|
||||
|
||||
let where_clause = conditions.join(" AND ");
|
||||
let sql = format!(
|
||||
"SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path, \
|
||||
"SELECT fi.id, fi.name, fi.folder_id, fo.path, \
|
||||
fi.size, fi.mime_type, \
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint, \
|
||||
@@ -319,7 +319,7 @@ impl FileBlobReadRepository {
|
||||
}
|
||||
|
||||
let rows = sqlx::query_as::<_, FileRow>(
|
||||
"SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path, \
|
||||
"SELECT fi.id, fi.name, fi.folder_id, fo.path, \
|
||||
fi.size, fi.mime_type, \
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint, \
|
||||
@@ -415,9 +415,9 @@ impl FileBlobReadRepository {
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn row_to_file(
|
||||
id: String,
|
||||
id: Uuid,
|
||||
name: String,
|
||||
folder_id: Option<String>,
|
||||
folder_id: Option<Uuid>,
|
||||
folder_path: Option<String>,
|
||||
size: i64,
|
||||
mime_type: String,
|
||||
@@ -428,12 +428,12 @@ impl FileBlobReadRepository {
|
||||
updated_by: Option<Uuid>,
|
||||
) -> Result<File, DomainError> {
|
||||
File::from_materialized_row(
|
||||
id,
|
||||
id.to_string(),
|
||||
name,
|
||||
folder_path.as_deref(),
|
||||
size as u64,
|
||||
mime_type,
|
||||
folder_id,
|
||||
folder_id.map(|u| u.to_string()),
|
||||
created_at as u64,
|
||||
modified_at as u64,
|
||||
blob_hash,
|
||||
@@ -550,7 +550,7 @@ impl FileBlobReadRepository {
|
||||
AND (g.expires_at IS NULL OR g.expires_at > NOW())
|
||||
AND (d.policies->>'include_in_photo_index')::boolean = true
|
||||
)
|
||||
SELECT top.id::text, top.name, top.folder_id::text, fo.path,
|
||||
SELECT top.id, top.name, top.folder_id, fo.path,
|
||||
top.size, top.mime_type,
|
||||
EXTRACT(EPOCH FROM top.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM top.updated_at)::bigint,
|
||||
@@ -679,9 +679,9 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
let row = sqlx::query_as::<
|
||||
_,
|
||||
(
|
||||
String, // id
|
||||
Uuid, // id (binary decode)
|
||||
String, // name
|
||||
Option<String>, // folder_id
|
||||
Option<Uuid>, // folder_id
|
||||
Option<String>, // folder path
|
||||
i64, // size
|
||||
String, // mime_type
|
||||
@@ -693,7 +693,7 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
),
|
||||
>(
|
||||
r#"
|
||||
SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path,
|
||||
SELECT fi.id, fi.name, fi.folder_id, fo.path,
|
||||
fi.size, fi.mime_type,
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint,
|
||||
@@ -726,9 +726,9 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
let row = sqlx::query_as::<
|
||||
_,
|
||||
(
|
||||
Uuid,
|
||||
String,
|
||||
String,
|
||||
Option<String>,
|
||||
Option<Uuid>,
|
||||
Option<String>,
|
||||
i64,
|
||||
String,
|
||||
@@ -740,7 +740,7 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
),
|
||||
>(
|
||||
r#"
|
||||
SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path,
|
||||
SELECT fi.id, fi.name, fi.folder_id, fo.path,
|
||||
fi.size, fi.mime_type,
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint,
|
||||
@@ -768,7 +768,7 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
let rows: Vec<FileRow> = if let Some(fid) = folder_id {
|
||||
sqlx::query_as(
|
||||
r#"
|
||||
SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path,
|
||||
SELECT fi.id, fi.name, fi.folder_id, fo.path,
|
||||
fi.size, fi.mime_type,
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint,
|
||||
@@ -787,7 +787,7 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
} else {
|
||||
sqlx::query_as(
|
||||
r#"
|
||||
SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path,
|
||||
SELECT fi.id, fi.name, fi.folder_id, fo.path,
|
||||
fi.size, fi.mime_type,
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint,
|
||||
@@ -848,7 +848,7 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
};
|
||||
let sql = format!(
|
||||
r#"
|
||||
SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path,
|
||||
SELECT fi.id, fi.name, fi.folder_id, fo.path,
|
||||
fi.size, fi.mime_type,
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint,
|
||||
@@ -1014,9 +1014,9 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
sqlx::query_as::<
|
||||
_,
|
||||
(
|
||||
Uuid,
|
||||
String,
|
||||
String,
|
||||
Option<String>,
|
||||
Option<Uuid>,
|
||||
Option<String>,
|
||||
i64,
|
||||
String,
|
||||
@@ -1028,7 +1028,7 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
),
|
||||
>(
|
||||
r#"
|
||||
SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path,
|
||||
SELECT fi.id, fi.name, fi.folder_id, fo.path,
|
||||
fi.size, fi.mime_type,
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint,
|
||||
@@ -1052,9 +1052,9 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
sqlx::query_as::<
|
||||
_,
|
||||
(
|
||||
Uuid,
|
||||
String,
|
||||
String,
|
||||
Option<String>,
|
||||
Option<Uuid>,
|
||||
Option<String>,
|
||||
i64,
|
||||
String,
|
||||
@@ -1066,7 +1066,7 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
),
|
||||
>(
|
||||
r#"
|
||||
SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path,
|
||||
SELECT fi.id, fi.name, fi.folder_id, fo.path,
|
||||
fi.size, fi.mime_type,
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint,
|
||||
@@ -1107,12 +1107,12 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
|
||||
let stream = async_stream::try_stream! {
|
||||
let mut row_stream = sqlx::query_as::<_, (
|
||||
String, String, Option<String>, Option<String>,
|
||||
Uuid, String, Option<Uuid>, Option<String>,
|
||||
i64, String, i64, i64, String,
|
||||
Option<Uuid>, Option<Uuid>, // created_by, updated_by (§14)
|
||||
)>(
|
||||
r#"
|
||||
SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path,
|
||||
SELECT fi.id, fi.name, fi.folder_id, fo.path,
|
||||
fi.size, fi.mime_type,
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint,
|
||||
@@ -1195,7 +1195,7 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
let offset_bind = bind_idx + 2;
|
||||
|
||||
let sql = format!(
|
||||
"SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path, \
|
||||
"SELECT fi.id, fi.name, fi.folder_id, fo.path, \
|
||||
fi.size, fi.mime_type, \
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint, \
|
||||
@@ -1214,9 +1214,9 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
let mut query = sqlx::query_as::<
|
||||
_,
|
||||
(
|
||||
Uuid,
|
||||
String,
|
||||
String,
|
||||
Option<String>,
|
||||
Option<Uuid>,
|
||||
Option<String>,
|
||||
i64,
|
||||
String,
|
||||
@@ -1327,7 +1327,7 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
|
||||
// ── Single query with COUNT(*) OVER() ──
|
||||
let sql = format!(
|
||||
"SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path, \
|
||||
"SELECT fi.id, fi.name, fi.folder_id, fo.path, \
|
||||
fi.size, fi.mime_type, \
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint, \
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint, \
|
||||
@@ -1346,9 +1346,9 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
let mut query = sqlx::query_as::<
|
||||
_,
|
||||
(
|
||||
Uuid,
|
||||
String,
|
||||
String,
|
||||
Option<String>,
|
||||
Option<Uuid>,
|
||||
Option<String>,
|
||||
i64,
|
||||
String,
|
||||
@@ -1420,7 +1420,7 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
let rows: Vec<FileRow> = if let Some(fid) = folder_id {
|
||||
sqlx::query_as(
|
||||
r#"
|
||||
SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path,
|
||||
SELECT fi.id, fi.name, fi.folder_id, fo.path,
|
||||
fi.size, fi.mime_type,
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint,
|
||||
@@ -1450,7 +1450,7 @@ impl FileReadPort for FileBlobReadRepository {
|
||||
} else {
|
||||
sqlx::query_as(
|
||||
r#"
|
||||
SELECT fi.id::text, fi.name, fi.folder_id::text, fo.path,
|
||||
SELECT fi.id, fi.name, fi.folder_id, fo.path,
|
||||
fi.size, fi.mime_type,
|
||||
EXTRACT(EPOCH FROM fi.created_at)::bigint,
|
||||
EXTRACT(EPOCH FROM fi.updated_at)::bigint,
|
||||
|
||||
@@ -22,7 +22,8 @@ use axum::{
|
||||
http::{HeaderName, Request, StatusCode, header},
|
||||
response::Response,
|
||||
};
|
||||
use bytes::Buf;
|
||||
use bytes::{Buf, Bytes};
|
||||
use quick_xml::Writer;
|
||||
use std::sync::Arc;
|
||||
|
||||
use crate::application::adapters::carddav_adapter::{
|
||||
@@ -31,7 +32,7 @@ use crate::application::adapters::carddav_adapter::{
|
||||
use crate::application::adapters::uid_from_multiget_href;
|
||||
use crate::application::adapters::webdav_adapter::{PropFindRequest, PropFindType};
|
||||
use crate::application::dtos::address_book_dto::{CreateAddressBookDto, UpdateAddressBookDto};
|
||||
use crate::application::dtos::contact_dto::CreateContactVCardDto;
|
||||
use crate::application::dtos::contact_dto::{ContactDto, CreateContactVCardDto};
|
||||
use crate::application::ports::carddav_ports::{AddressBookUseCase, ContactUseCase};
|
||||
use crate::application::services::contact_service::ContactService;
|
||||
use crate::common::di::AppState;
|
||||
@@ -187,6 +188,164 @@ fn get_addressbook_service(state: &AppState) -> Result<&Arc<ContactService>, App
|
||||
})
|
||||
}
|
||||
|
||||
/// Rows per emitted page for the streaming CardDAV emitters — contacts
|
||||
/// carry no master/exception bundling, so pages cut anywhere.
|
||||
const CARDDAV_STREAM_PAGE_CONTACTS: usize = 500;
|
||||
|
||||
/// Streamed multistatus REPORT: header, one chunk per cursor page,
|
||||
/// footer. Byte-compatible with the buffered
|
||||
/// `generate_contacts_response` output; TTFB becomes the first page and
|
||||
/// the whole-book DTO Vec is never materialised.
|
||||
fn build_streaming_contacts_report(
|
||||
contact_svc: Arc<ContactService>,
|
||||
address_book_id: String,
|
||||
report: CardDavReportType,
|
||||
base_href: String,
|
||||
user_id: uuid::Uuid,
|
||||
) -> Response<Body> {
|
||||
let stream = async_stream::try_stream! {
|
||||
let mut buf = Vec::with_capacity(160);
|
||||
{
|
||||
let mut w = Writer::new(&mut buf);
|
||||
CardDavAdapter::write_report_multistatus_start(&mut w)
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))?;
|
||||
}
|
||||
yield Bytes::from(buf);
|
||||
|
||||
{
|
||||
use futures::TryStreamExt;
|
||||
let mut rows = contact_svc
|
||||
.stream_contacts_by_book(&address_book_id, user_id)
|
||||
.await
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))?;
|
||||
let mut page: Vec<ContactDto> =
|
||||
Vec::with_capacity(CARDDAV_STREAM_PAGE_CONTACTS);
|
||||
loop {
|
||||
let next = rows
|
||||
.try_next()
|
||||
.await
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))?;
|
||||
let flush = match &next {
|
||||
Some(_) => page.len() >= CARDDAV_STREAM_PAGE_CONTACTS,
|
||||
None => !page.is_empty(),
|
||||
};
|
||||
if flush {
|
||||
let mut chunk = Vec::with_capacity(page.len() * 256 + 64);
|
||||
{
|
||||
let mut w = Writer::new(&mut chunk);
|
||||
CardDavAdapter::write_contacts_report_page(
|
||||
&mut w, &page, &report, &base_href,
|
||||
)
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))?;
|
||||
}
|
||||
page.clear();
|
||||
yield Bytes::from(chunk);
|
||||
}
|
||||
match next {
|
||||
Some(c) => page.push(c),
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut buf = Vec::with_capacity(32);
|
||||
{
|
||||
let mut w = Writer::new(&mut buf);
|
||||
CardDavAdapter::write_carddav_multistatus_end(&mut w)
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))?;
|
||||
}
|
||||
yield Bytes::from(buf);
|
||||
};
|
||||
|
||||
use futures::TryStreamExt;
|
||||
let stream = stream
|
||||
.map_err(|e: std::io::Error| -> Box<dyn std::error::Error + Send + Sync> { Box::new(e) });
|
||||
|
||||
Response::builder()
|
||||
.status(StatusCode::MULTI_STATUS)
|
||||
.header(header::CONTENT_TYPE, "application/xml; charset=utf-8")
|
||||
.body(Body::from_stream(stream))
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
/// Streamed depth-1 address-book PROPFIND: head (multistatus + the
|
||||
/// book's own response), one chunk per cursor page, footer.
|
||||
fn build_streaming_book_propfind(
|
||||
contact_svc: Arc<ContactService>,
|
||||
address_book: crate::application::dtos::address_book_dto::AddressBookDto,
|
||||
propfind_request: PropFindRequest,
|
||||
address_book_id: String,
|
||||
base_href: String,
|
||||
user_id: uuid::Uuid,
|
||||
) -> Response<Body> {
|
||||
let stream = async_stream::try_stream! {
|
||||
let mut buf = Vec::with_capacity(2048);
|
||||
{
|
||||
let mut w = Writer::new(&mut buf);
|
||||
CardDavAdapter::write_collection_head(
|
||||
&mut w,
|
||||
&address_book,
|
||||
&propfind_request,
|
||||
&base_href,
|
||||
)
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))?;
|
||||
}
|
||||
yield Bytes::from(buf);
|
||||
|
||||
{
|
||||
use futures::TryStreamExt;
|
||||
let mut rows = contact_svc
|
||||
.stream_contacts_by_book(&address_book_id, user_id)
|
||||
.await
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))?;
|
||||
let mut page: Vec<ContactDto> =
|
||||
Vec::with_capacity(CARDDAV_STREAM_PAGE_CONTACTS);
|
||||
loop {
|
||||
let next = rows
|
||||
.try_next()
|
||||
.await
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))?;
|
||||
let flush = match &next {
|
||||
Some(_) => page.len() >= CARDDAV_STREAM_PAGE_CONTACTS,
|
||||
None => !page.is_empty(),
|
||||
};
|
||||
if flush {
|
||||
let mut chunk = Vec::with_capacity(page.len() * 512 + 64);
|
||||
{
|
||||
let mut w = Writer::new(&mut chunk);
|
||||
CardDavAdapter::write_collection_contact_page(&mut w, &page, &base_href)
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))?;
|
||||
}
|
||||
page.clear();
|
||||
yield Bytes::from(chunk);
|
||||
}
|
||||
match next {
|
||||
Some(c) => page.push(c),
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut buf = Vec::with_capacity(32);
|
||||
{
|
||||
let mut w = Writer::new(&mut buf);
|
||||
CardDavAdapter::write_carddav_multistatus_end(&mut w)
|
||||
.map_err(|e| std::io::Error::other(e.to_string()))?;
|
||||
}
|
||||
yield Bytes::from(buf);
|
||||
};
|
||||
|
||||
use futures::TryStreamExt;
|
||||
let stream = stream
|
||||
.map_err(|e: std::io::Error| -> Box<dyn std::error::Error + Send + Sync> { Box::new(e) });
|
||||
|
||||
Response::builder()
|
||||
.status(StatusCode::MULTI_STATUS)
|
||||
.header(header::CONTENT_TYPE, "application/xml; charset=utf-8")
|
||||
.body(Body::from_stream(stream))
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
fn get_contact_service(state: &AppState) -> Result<&Arc<ContactService>, AppError> {
|
||||
state.contact_use_case.as_ref().ok_or_else(|| {
|
||||
AppError::new(
|
||||
@@ -334,14 +493,19 @@ async fn handle_propfind(
|
||||
.await
|
||||
.map_err(|e| AppError::not_found(format!("Address book not found: {}", e)))?;
|
||||
|
||||
let contacts = if depth != "0" {
|
||||
contact_svc
|
||||
.list_contacts(address_book_id, None, None, user.id)
|
||||
.await
|
||||
.unwrap_or_default()
|
||||
} else {
|
||||
vec![]
|
||||
};
|
||||
// Depth-1 streams the contact listing page by page; depth-0
|
||||
// has no contact section and keeps the tiny buffered path.
|
||||
if depth != "0" {
|
||||
let base_href = format!("/carddav/{}/", address_book_id);
|
||||
return Ok(build_streaming_book_propfind(
|
||||
contact_svc.clone(),
|
||||
address_book,
|
||||
propfind_request,
|
||||
address_book_id.to_string(),
|
||||
base_href,
|
||||
user.id,
|
||||
));
|
||||
}
|
||||
|
||||
let base_href = &format!("/carddav/{}/", address_book_id);
|
||||
let mut response_body = Vec::new();
|
||||
@@ -349,7 +513,7 @@ async fn handle_propfind(
|
||||
CardDavAdapter::generate_addressbook_collection_propfind(
|
||||
&mut response_body,
|
||||
&address_book,
|
||||
&contacts,
|
||||
&[],
|
||||
&propfind_request,
|
||||
base_href,
|
||||
&depth,
|
||||
@@ -423,11 +587,25 @@ async fn handle_report(
|
||||
return Err(AppError::bad_request("Address book ID required in path"));
|
||||
}
|
||||
|
||||
// Whole-book shapes stream; bounded multiget keeps the buffered path.
|
||||
if matches!(
|
||||
&report,
|
||||
CardDavReportType::AddressbookQuery { .. } | CardDavReportType::SyncCollection { .. }
|
||||
) {
|
||||
let base_href = format!("/carddav/{}/", address_book_id);
|
||||
return Ok(build_streaming_contacts_report(
|
||||
contact_svc.clone(),
|
||||
address_book_id.to_string(),
|
||||
report,
|
||||
base_href,
|
||||
user.id,
|
||||
));
|
||||
}
|
||||
|
||||
let contacts = match &report {
|
||||
CardDavReportType::AddressbookQuery { .. } => contact_svc
|
||||
.list_contacts(address_book_id, None, None, user.id)
|
||||
.await
|
||||
.map_err(AppError::from)?,
|
||||
CardDavReportType::AddressbookQuery { .. } => {
|
||||
unreachable!("addressbook-query streams above")
|
||||
}
|
||||
CardDavReportType::AddressbookMultiget { hrefs, .. } => {
|
||||
// Indexed batch lookup (`uid = ANY(...)`) — a multiget for a
|
||||
// handful of contacts must not pay for listing the whole
|
||||
@@ -442,10 +620,9 @@ async fn handle_report(
|
||||
.await
|
||||
.map_err(AppError::from)?
|
||||
}
|
||||
CardDavReportType::SyncCollection { .. } => contact_svc
|
||||
.list_contacts(address_book_id, None, None, user.id)
|
||||
.await
|
||||
.map_err(AppError::from)?,
|
||||
CardDavReportType::SyncCollection { .. } => {
|
||||
unreachable!("sync-collection streams above")
|
||||
}
|
||||
};
|
||||
|
||||
let base_href = &format!("/carddav/{}/", address_book_id);
|
||||
|
||||
@@ -232,17 +232,18 @@ pub async fn access_shared_item(
|
||||
Path(token): Path<String>,
|
||||
headers: HeaderMap,
|
||||
) -> impl IntoResponse {
|
||||
// Register the access
|
||||
let _ = share_use_case.register_shared_link_access(&token).await;
|
||||
|
||||
// Honour an unlock cookie if one was issued by a prior `/verify` call.
|
||||
let unlock_jwt = unlock_jwt_from_headers(&headers, &token);
|
||||
|
||||
// Get the shared link
|
||||
match share_use_case
|
||||
.get_shared_link_with_unlock(&token, unlock_jwt.as_deref())
|
||||
.await
|
||||
{
|
||||
// The access-count increment doesn't gate the fetch — run both
|
||||
// round-trips concurrently instead of serially (one RTT saved on
|
||||
// every public share landing).
|
||||
let (_, item) = tokio::join!(
|
||||
share_use_case.register_shared_link_access(&token),
|
||||
share_use_case.get_shared_link_with_unlock(&token, unlock_jwt.as_deref()),
|
||||
);
|
||||
|
||||
match item {
|
||||
Ok(item) => (StatusCode::OK, Json(item)).into_response(),
|
||||
Err(err) => {
|
||||
// Special handling for share access errors
|
||||
|
||||
@@ -411,8 +411,8 @@ pub async fn handle_search(
|
||||
|
||||
// Pre-resolve numeric ids for every file result in a single batch query
|
||||
// (was one INSERT round-trip per result).
|
||||
let file_uuids: Vec<String> = results.files.iter().map(|f| f.id.clone()).collect();
|
||||
let file_id_map: HashMap<String, i64> = match file_id_svc {
|
||||
let file_uuids: Vec<&str> = results.files.iter().map(|f| f.id.as_str()).collect();
|
||||
let file_id_map: HashMap<uuid::Uuid, i64> = match file_id_svc {
|
||||
Some(svc) => svc
|
||||
.get_or_create_file_ids(&file_uuids)
|
||||
.await
|
||||
@@ -435,7 +435,8 @@ pub async fn handle_search(
|
||||
crate::interfaces::nextcloud::webdav_handler::strip_drive_root_segment(&file.path);
|
||||
let display_path = format!("/{}", display_path);
|
||||
|
||||
let numeric_id = file_id_map.get(&file.id).copied();
|
||||
let numeric_id =
|
||||
crate::interfaces::nextcloud::webdav_handler::nc_id_of(&file_id_map, &file.id);
|
||||
|
||||
let thumbnail_url = match numeric_id {
|
||||
Some(nid) => format!("/index.php/core/preview?fileId={}&x=32&y=32", nid),
|
||||
|
||||
@@ -26,7 +26,7 @@ use crate::interfaces::api::handlers::webdav_handler::{
|
||||
};
|
||||
use crate::interfaces::errors::AppError;
|
||||
use crate::interfaces::nextcloud::webdav_handler::{
|
||||
batch_resolve_ids, format_oc_id, nc_href, write_file_response, write_folder_response,
|
||||
batch_resolve_ids, format_oc_id, nc_href, nc_id_of, write_file_response, write_folder_response,
|
||||
};
|
||||
|
||||
/// Handle WebDAV REPORT and SEARCH methods for Nextcloud compatibility.
|
||||
@@ -150,8 +150,8 @@ async fn handle_filter_files(
|
||||
}
|
||||
|
||||
// Pass 2: resolve every oc:fileid in two batch queries (was one per item).
|
||||
let file_uuids: Vec<String> = files.iter().map(|f| f.id.clone()).collect();
|
||||
let folder_uuids: Vec<String> = folders.iter().map(|f| f.id.clone()).collect();
|
||||
let file_uuids: Vec<&str> = files.iter().map(|f| f.id.as_str()).collect();
|
||||
let folder_uuids: Vec<&str> = folders.iter().map(|f| f.id.as_str()).collect();
|
||||
let (file_id_map, folder_id_map) =
|
||||
batch_resolve_ids(file_id_svc, &file_uuids, &folder_uuids).await;
|
||||
|
||||
@@ -184,7 +184,7 @@ async fn handle_filter_files(
|
||||
continue;
|
||||
};
|
||||
let href = nc_href(url_user, subpath);
|
||||
let fid = file_id_map.get(&file.id).copied();
|
||||
let fid = nc_id_of(&file_id_map, &file.id);
|
||||
let oc_id = fid.map(|id| format_oc_id(id, file_id_svc));
|
||||
let dead = dead_props_for(&file.id, &file_deads);
|
||||
write_file_response(
|
||||
@@ -210,7 +210,7 @@ async fn handle_filter_files(
|
||||
continue;
|
||||
};
|
||||
let href = format!("{}/", nc_href(url_user, subpath));
|
||||
let fid = folder_id_map.get(&folder.id).copied();
|
||||
let fid = nc_id_of(&folder_id_map, &folder.id);
|
||||
let oc_id = fid.map(|id| format_oc_id(id, file_id_svc));
|
||||
let dead = dead_props_for(&folder.id, &folder_deads);
|
||||
write_folder_response(
|
||||
@@ -297,8 +297,8 @@ async fn handle_search(
|
||||
// (was one INSERT round-trip per result).
|
||||
let files: Vec<FileDto> = results.files.iter().map(file_dto_from_search).collect();
|
||||
let folders: Vec<FolderDto> = results.folders.iter().map(folder_dto_from_search).collect();
|
||||
let file_uuids: Vec<String> = files.iter().map(|f| f.id.clone()).collect();
|
||||
let folder_uuids: Vec<String> = folders.iter().map(|f| f.id.clone()).collect();
|
||||
let file_uuids: Vec<&str> = files.iter().map(|f| f.id.as_str()).collect();
|
||||
let folder_uuids: Vec<&str> = folders.iter().map(|f| f.id.as_str()).collect();
|
||||
let (file_id_map, folder_id_map) =
|
||||
batch_resolve_ids(file_id_svc, &file_uuids, &folder_uuids).await;
|
||||
|
||||
@@ -325,7 +325,7 @@ async fn handle_search(
|
||||
continue;
|
||||
};
|
||||
let href = nc_href(url_user, subpath);
|
||||
let fid = file_id_map.get(&file.id).copied();
|
||||
let fid = nc_id_of(&file_id_map, &file.id);
|
||||
let oc_id = fid.map(|id| format_oc_id(id, file_id_svc));
|
||||
let dead = dead_props_for(&file.id, &file_deads);
|
||||
write_file_response(
|
||||
@@ -352,7 +352,7 @@ async fn handle_search(
|
||||
continue;
|
||||
};
|
||||
let href = format!("{}/", nc_href(url_user, subpath));
|
||||
let fid = folder_id_map.get(&folder.id).copied();
|
||||
let fid = nc_id_of(&folder_id_map, &folder.id);
|
||||
let oc_id = fid.map(|id| format_oc_id(id, file_id_svc));
|
||||
let dead = dead_props_for(&folder.id, &folder_deads);
|
||||
write_folder_response(
|
||||
|
||||
@@ -15,7 +15,7 @@ use crate::application::ports::trash_ports::TrashUseCase;
|
||||
use crate::common::di::AppState;
|
||||
use crate::interfaces::errors::AppError;
|
||||
use crate::interfaces::nextcloud::webdav_handler::{
|
||||
batch_resolve_ids, extract_nc_subpath_from_dest, format_oc_id, nc_to_internal_path,
|
||||
batch_resolve_ids, extract_nc_subpath_from_dest, format_oc_id, nc_id_of, nc_to_internal_path,
|
||||
write_text_element,
|
||||
};
|
||||
|
||||
@@ -308,6 +308,7 @@ fn strip_home_prefix<'a>(
|
||||
use crate::application::dtos::trash_dto::TrashedItemDto;
|
||||
use crate::application::services::nextcloud_file_id_service::NextcloudFileIdService;
|
||||
use std::collections::HashMap;
|
||||
use uuid::Uuid;
|
||||
|
||||
/// Generate a complete Nextcloud-compatible multistatus XML response for the trashbin.
|
||||
///
|
||||
@@ -337,14 +338,14 @@ async fn write_trashbin_multistatus<W: std::io::Write>(
|
||||
|
||||
// Pre-resolve every oc:fileid in two batch queries by object type (was one
|
||||
// INSERT round-trip per item). File and folder UUIDs are disjoint, so the
|
||||
// two maps merge cleanly into one keyed by original_id.
|
||||
let mut file_uuids: Vec<String> = Vec::new();
|
||||
let mut folder_uuids: Vec<String> = Vec::new();
|
||||
// two maps merge cleanly into one keyed by parsed original-id UUID.
|
||||
let mut file_uuids: Vec<&str> = Vec::new();
|
||||
let mut folder_uuids: Vec<&str> = Vec::new();
|
||||
for item in items {
|
||||
if item.item_type == "folder" {
|
||||
folder_uuids.push(item.original_id.clone());
|
||||
folder_uuids.push(item.original_id.as_str());
|
||||
} else {
|
||||
file_uuids.push(item.original_id.clone());
|
||||
file_uuids.push(item.original_id.as_str());
|
||||
}
|
||||
}
|
||||
let (mut id_map, folder_id_map) =
|
||||
@@ -427,7 +428,7 @@ fn write_trash_item_response<W: std::io::Write>(
|
||||
username: &str,
|
||||
chroot: &crate::application::dtos::folder_dto::FolderDto,
|
||||
file_id_svc: Option<&Arc<NextcloudFileIdService>>,
|
||||
id_map: &HashMap<String, i64>,
|
||||
id_map: &HashMap<Uuid, i64>,
|
||||
) -> Result<(), String> {
|
||||
xml.write_event(Event::Start(BytesStart::new("d:response")))
|
||||
.map_err(|e| e.to_string())?;
|
||||
@@ -475,7 +476,7 @@ fn write_trash_item_response<W: std::io::Write>(
|
||||
write_text_element(xml, "d:getcontentlength", "0")?;
|
||||
|
||||
// oc:fileid and oc:id — resolved up front in a batch query.
|
||||
let file_id = id_map.get(&item.original_id).copied();
|
||||
let file_id = nc_id_of(id_map, &item.original_id);
|
||||
if let Some(id) = file_id {
|
||||
write_text_element(xml, "oc:fileid", &id.to_string())?;
|
||||
let oc_id = format_oc_id(id, file_id_svc);
|
||||
|
||||
@@ -1445,8 +1445,7 @@ async fn write_nc_file_multistatus<W: std::io::Write>(
|
||||
extras: (&HashSet<String>, &[(QualifiedName, Option<String>)]),
|
||||
) -> Result<(), String> {
|
||||
let (favorite_ids, dead_props) = extras;
|
||||
let (file_id_map, _) =
|
||||
batch_resolve_ids(file_id_svc, std::slice::from_ref(&file.id), &[]).await;
|
||||
let (file_id_map, _) = batch_resolve_ids(file_id_svc, &[file.id.as_str()], &[]).await;
|
||||
|
||||
let mut xml = Writer::new(writer);
|
||||
write_nc_multistatus_open(&mut xml)?;
|
||||
@@ -1457,7 +1456,7 @@ async fn write_nc_file_multistatus<W: std::io::Write>(
|
||||
// shares the requested URL's prefix. `username` is the canonical
|
||||
// identity for the `oc:owner-id` field.
|
||||
let href = nc_href(url_user, subpath);
|
||||
let file_id = file_id_map.get(&file.id).copied();
|
||||
let file_id = nc_id_of(&file_id_map, &file.id);
|
||||
let oc_id = file_id.map(|id| format_oc_id(id, file_id_svc));
|
||||
write_file_response(
|
||||
&mut xml,
|
||||
@@ -1509,7 +1508,7 @@ fn build_nc_streaming_propfind(
|
||||
HashSet::new()
|
||||
};
|
||||
let (_, folder_id_map) =
|
||||
batch_resolve_ids(file_id_svc, &[], std::slice::from_ref(&folder.id)).await;
|
||||
batch_resolve_ids(file_id_svc, &[], &[folder.id.as_str()]).await;
|
||||
let folder_dead = folder_dead_props(&state.webdav_dead_props, &folder).await;
|
||||
|
||||
let mut buf = Vec::with_capacity(4096);
|
||||
@@ -1517,7 +1516,7 @@ fn build_nc_streaming_propfind(
|
||||
let mut xml = Writer::new(&mut buf);
|
||||
write_nc_multistatus_open(&mut xml).map_err(std::io::Error::other)?;
|
||||
let href = nc_collection_href(&username, &subpath);
|
||||
let fid = folder_id_map.get(&folder.id).copied();
|
||||
let fid = nc_id_of(&folder_id_map, &folder.id);
|
||||
let oc_id = fid.map(|id| format_oc_id(id, file_id_svc));
|
||||
write_folder_response(&mut xml, &folder, &href, (fid, oc_id.as_deref()), &username, &folder_favs, quota, &folder_dead)
|
||||
.map_err(std::io::Error::other)?;
|
||||
@@ -1565,7 +1564,7 @@ fn build_nc_streaming_propfind(
|
||||
} else {
|
||||
HashSet::new()
|
||||
};
|
||||
let file_uuids: Vec<String> = batch.iter().map(|f| f.id.clone()).collect();
|
||||
let file_uuids: Vec<&str> = batch.iter().map(|f| f.id.as_str()).collect();
|
||||
let (file_id_map, _) = batch_resolve_ids(file_id_svc, &file_uuids, &[]).await;
|
||||
// One batched dead-props query per page, not one per child
|
||||
// (benches/DEAD-PROPS.md).
|
||||
@@ -1582,7 +1581,7 @@ fn build_nc_streaming_propfind(
|
||||
// re-encoded both for every child).
|
||||
let href =
|
||||
format!("{}{}", child_href_prefix, urlencoding::encode(&file.name));
|
||||
let fid = file_id_map.get(&file.id).copied();
|
||||
let fid = nc_id_of(&file_id_map, &file.id);
|
||||
let oc_id = fid.map(|id| format_oc_id(id, file_id_svc));
|
||||
write_file_response(&mut xml, file, &href, (fid, oc_id.as_deref()), &username, &favs, dead)
|
||||
.map_err(std::io::Error::other)?;
|
||||
@@ -1622,7 +1621,7 @@ fn build_nc_streaming_propfind(
|
||||
} else {
|
||||
HashSet::new()
|
||||
};
|
||||
let folder_uuids: Vec<String> = batch.iter().map(|sf| sf.id.clone()).collect();
|
||||
let folder_uuids: Vec<&str> = batch.iter().map(|sf| sf.id.as_str()).collect();
|
||||
let (_, sub_id_map) = batch_resolve_ids(file_id_svc, &[], &folder_uuids).await;
|
||||
// Batched — see benches/DEAD-PROPS.md.
|
||||
let sub_deads =
|
||||
@@ -1637,7 +1636,7 @@ fn build_nc_streaming_propfind(
|
||||
// precomputed once like the file loop above.
|
||||
let href =
|
||||
format!("{}{}/", child_href_prefix, urlencoding::encode(&sf.name));
|
||||
let fid = sub_id_map.get(&sf.id).copied();
|
||||
let fid = nc_id_of(&sub_id_map, &sf.id);
|
||||
let oc_id = fid.map(|id| format_oc_id(id, file_id_svc));
|
||||
write_folder_response(&mut xml, sf, &href, (fid, oc_id.as_deref()), &username, &favs, quota, dead)
|
||||
.map_err(std::io::Error::other)?;
|
||||
@@ -1949,14 +1948,15 @@ pub fn write_text_element<W: std::io::Write>(
|
||||
|
||||
/// Resolve every `oc:fileid` for a listing in two batch queries (one per
|
||||
/// object type) instead of one INSERT round-trip per child. Returns
|
||||
/// `(file_map, folder_map)` keyed by object UUID; entries are absent when the
|
||||
/// service is disabled or an id can't be resolved, mirroring the previous
|
||||
/// per-call `Option` behaviour. The two batches run concurrently.
|
||||
/// `(file_map, folder_map)` keyed by parsed object UUID; entries are absent
|
||||
/// when the service is disabled or an id can't be resolved, mirroring the
|
||||
/// previous per-call `Option` behaviour. The two batches run concurrently.
|
||||
/// Borrowed inputs + `Uuid` keys keep the whole resolution alloc-free.
|
||||
pub async fn batch_resolve_ids(
|
||||
svc: Option<&Arc<NextcloudFileIdService>>,
|
||||
file_uuids: &[String],
|
||||
folder_uuids: &[String],
|
||||
) -> (HashMap<String, i64>, HashMap<String, i64>) {
|
||||
file_uuids: &[&str],
|
||||
folder_uuids: &[&str],
|
||||
) -> (HashMap<Uuid, i64>, HashMap<Uuid, i64>) {
|
||||
let Some(svc) = svc else {
|
||||
return (HashMap::new(), HashMap::new());
|
||||
};
|
||||
@@ -1967,6 +1967,11 @@ pub async fn batch_resolve_ids(
|
||||
(files.unwrap_or_default(), folders.unwrap_or_default())
|
||||
}
|
||||
|
||||
/// Look up a batch-resolved `oc:fileid` by a DTO's string UUID.
|
||||
pub fn nc_id_of(map: &HashMap<Uuid, i64>, id: &str) -> Option<i64> {
|
||||
Uuid::parse_str(id).ok().and_then(|u| map.get(&u).copied())
|
||||
}
|
||||
|
||||
pub fn format_oc_id(id: i64, svc: Option<&Arc<NextcloudFileIdService>>) -> String {
|
||||
match svc {
|
||||
Some(s) => s.format_oc_id(id),
|
||||
|
||||
@@ -419,8 +419,8 @@ impl IncrementalHasher {
|
||||
|
||||
fn finalize_hex(self) -> String {
|
||||
match self {
|
||||
Self::Md5(h) => h.finalize().iter().map(|b| format!("{b:02x}")).collect(),
|
||||
Self::Sha256(h) => h.finalize().iter().map(|b| format!("{b:02x}")).collect(),
|
||||
Self::Md5(h) => crate::common::fmt::hex_lower(&h.finalize()),
|
||||
Self::Sha256(h) => crate::common::fmt::hex_lower(&h.finalize()),
|
||||
Self::Blake3(h) => h.finalize().to_hex().to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user