add /api/folders/{id}/ancestors
this API to iterate parent up to the drive root or the shared folder
this will help UI to build the breadcrumb in 1 API call
and to identify the root element (is it a drive users has access to or
a shared folder ?)
ui: now only 1 API call is now required to build the breadcrumb
- provide is_shared and is_favorite information in DTO, information propagated as badge/buttons per items
regarding performances I try to be minimalis on SQL to prevent any perf regression
- remove old set of sharedids and favoriteids (was not functionnal anymore)
- fix date picker (no past selection) in grant
- fix contextmenu close on /files section
- rustfmt the integration-test FolderService::new callsites added during
the upstream merge (long single-line args wrapped).
- Cargo.lock: rebuild from upstream/main's lock so only the
testcontainers-modules dev-dep subtree and the memmap2 0.9.10→0.9.11
security bump differ (avoids churning upstream's pins).
- .cargo/audit.toml: ignore the four astral-tokio-tar tar-extraction
advisories — dev-only, transitive via testcontainers-modules
(integration-test harness), never in the production binary.
Resolve conflicts between the external-file-mounts feature and upstream's
D5/D7 refactor (per-file provenance, keyset pagination, cross-drive move
gates, resource-access hook, folder-cascade lifecycle hook).
Key resolutions:
- FolderService::new now takes (repo, authz, file_lifecycle, mount_router);
all callers + DI updated.
- FileRetrievalService / FileManagementService keep both the mount_router
and the new resource_access_hook / drive_repo / storage_usage wiring.
- list_files_batch_with_perms: adapt the mount branch from offset- to
keyset (after_name) pagination, mirroring paginate_mount_entries.
- download_file_impl: keep upstream's &HeaderMap + `impl IntoResponse + use<>`
signature, retain the mount-download branch.
- Mount DTOs: the retired `owner_id` field maps onto created_by/updated_by
(the mount owner) — the fields the frontend now uses for owner display.
- admin/+page.svelte: keep upstream's user-delete modal + the 'mounts' tab.
- Bump memmap2 0.9.10 -> 0.9.11 (RUSTSEC critical advisory fix) and
regenerate Cargo.lock against the merged Cargo.toml.
Seven behaviour-preserving allocation / copy / bandwidth cuts, each behind a
counting-allocator BEFORE/AFTER gate that exit(1)s unless AFTER allocates
strictly fewer than BEFORE (benches/ROUND29.md, examples/bench_round29_micro.rs).
- [B] Content-cache serve fast path (optimized_inner Tier 1 +
get_file_range_preloaded — the video-scrub hot path): probe the cache with a
borrow first and build the owned get_or_load args (quoted-etag / key / id
Strings) only on a miss, instead of allocating them before every probe and
discarding them on a hit. 6 -> 0 allocs per cache hit. Splits get_or_load into
get + load_and_cache so the miss path is not re-probed and the hit/miss stat
counters stay byte-identical. Also drops the unconditional content_hash/name
clones that ran for the >=10 MB streaming tier that used neither.
- [A] NextCloud REPORT emit loops: per-row href String (and format! per folder
row) -> one reused href_buf via nc_href_into / nc_collection_href_into with the
URL-encoded user computed once per page. 1497 fewer allocs on a 500-row page.
- [C] read_full: a single-frame blob is returned zero-copy instead of a second
whole-payload memcpy into a fresh BytesMut; multi-frame path unchanged.
- [D] login-lockout key: to_lowercase()+format! -> one pre-sized ASCII buffer
(non-ASCII keeps str::to_lowercase). 3 -> 1 alloc/req, byte-identical key.
- [E] NC composite-username parse: owned clone/to_string -> &str borrow of the
already-owned raw_username. 1 -> 0 alloc on the common no-marker path.
- [F] get_contacts_in_group: stop SELECTing the discarded multi-KB vcard column
(the live method ROUND25 §Q2 missed; ContactDto has no vcard field).
- [G] count_admin_users: add count_users_by_role -> scalar COUNT(*) instead of
hydrating every admin's full row (incl. up-to-512 KiB avatar + ui_preferences
JSONB) only to .len() it, on a bootstrap-polled status endpoint.
All seven gates pass; cargo fmt --check and cargo clippy --all-features
--all-targets -D warnings clean. §F/§G additionally validated against a live
PostgreSQL 16 with the full migration set (query validity, result equivalence,
600000 -> 8 byte wire delta on the admin count).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LhpDZxSQTAGnAqCHUdtG5N
Three benchmark-gated optimizations from the ROUND25 backlog (benches/ROUND26.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- P1 drive_pg_repository policy reads: decode d.policies through
sqlx::types::Json<DrivePolicies> (one from_slice over the raw JSONB bytes)
instead of a throwaway serde_json::Value DOM + DrivePolicies::from_value —
6 -> 0 allocs/read, 2.77x wall. A shared policies_from_row helper preserves
the lenient unwrap_or_default fallback (malformed bag -> all-false).
- D1 CachedBlobBackend: pre-create the 256 {00..ff} shard dirs at initialize()
(mirroring LocalBlobBackend, reusing HEX_PREFIXES) and drop the redundant
per-write create_dir_all on already-existing shards — ~45us + a blocking-pool
dispatch removed per cache write on cached-remote deployments.
- G1 delta-upload have/need hash sets (distinct_hashes, authorize_chunk_download):
SipHash -> foldhash::quality::RandomState — a fast hasher that stays
DoS-resistant via a per-instance random seed, the required property for the
attacker-controlled 64-hex client hashes — 2.37x wall on a 40k-hash
negotiation. foldhash was already in the lockfile transitively (hashbrown).
Tested and REVERTED (kept as-is): moving the moka eviction unlink off the reactor
via spawn_blocking. The benchmark refuted it — on the local cache dir the
spawn_blocking dispatch (~20us) costs more than the inline unlink (~7us) it would
replace. See ROUND26.md §D2.
Adds bench_round26_{micro,diskio,hasher} (counting allocator / async wall / wall).
Verified: cargo fmt clean, cargo clippy --features bench -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Five benchmark-gated optimizations from a fresh six-way audit (benches/ROUND25.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- M1 EncryptedBlobBackend::decrypt_bytes: replace split_off (a fresh Vec + full
ciphertext memcpy on every decrypted chunk, contradicting its own "in place"
doc) with in-place detached decrypt + a zero-copy Bytes::slice past the nonce.
Peak RAM per read drops from ~2x to ~1x the payload (-262KB/op at 256KiB;
scales with blob size). Plaintext byte-identical; tamper/wrong-key tests pass.
- M2 delta commit: move-unzip the owned chunk list instead of a third
per-occurrence hash clone (-4000 allocs on a 4000-chunk commit).
- M3 folder ZIP download: drop the dead Query<HashMap> extractor it never read
(byte-identical response; 5->0 allocs/request).
- Q1 public-playlist listing: fold the per-playlist COUNT(*) N+1 into one
LEFT JOIN ... GROUP BY via a new inherent repo method (101 -> 1 round-trips,
36x wall on a 100-playlist page).
- Q2 contact REST listings (paginated/search/by-group): stop over-fetching the
multi-KB vcard TEXT the ContactDto discards, via a shared lite row mapper and
narrowed SELECTs (6.4x wall on 1000 contacts with 8KiB vcards). The whole-book
vCard export and CardDAV sync paths keep the column.
Adds bench_round25_micro (counting allocator tracking count+bytes) and
bench_round25_queries (live Postgres), both with equivalence gates and a
rollback exit(1). Verified: cargo fmt clean, cargo clippy -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
The ROUND23-deferred download_zip N+1, given its own validated pass. The
individually-selected files were authorized + fetched one at a time via
get_file_with_perms (require + get = 2 serial round-trips/file) before any
streaming — a 200-file selection was 400 serial round-trips. AFTER routes the
whole multi-select through the new
FileRetrievalService::get_files_by_ids_with_perms: one check_files_read_batch
(the PgAclEngine resolves every file's drive in ONE query and primes the
resource->drive cache) + one get_files_by_ids. 2N round-trips -> 2.
Authorization is unchanged and still enforced BEFORE any ZIP entry is written:
- add_file_entry_streamed writes the entry header (the filename) before it opens
the authorized stream, so the pre-filter is load-bearing — a denied file must
never reach it or its name leaks into the archive. AFTER a denied/missing id
is absent from the authorized map and is skipped in the same input order,
exactly as the old loop skipped a denied get_file_with_perms; it never reaches
the entry write. The authz moved from a per-file require to one batch check
EARLIER in the same function, not into or after the stream.
- The stream open keeps its own per-file Read check (now a primed-cache hit) +
Recents recording; check_files_read_batch is documented + gated as identical
to looping require.
Because the change is authorization-sensitive, the gate is the security property
itself. bench_round24_zip_authz drives the real PgAclEngine over a seeded,
interleaved mix of owned (granted drive) + denied (other drive) + missing ids
and asserts: the batch inclusion set AND input order are identical to the
per-file require loop; the included set is exactly the caller's owned files; no
denied or missing id is ever included (the authz-regression tripwire); and the
batch fetch returns exactly the owned files. Latency (cold, 600-item 1/3-owned
selection): 559 -> 267 ms (2.10x; the realistic all-owned selection is O(1) ->
a larger win). See benches/ROUND24.md.
The folder selections are left as-is (root counts are small and there is no
check_folders_read_batch primitive to batch through).
Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo
fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0
failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
Benchmark-gated, same rule as ROUND2-22: BEFORE/AFTER with a value-equivalence
gate and rollback-on-regression. Two harnesses — bench_round23_micro (no
Postgres; deterministic allocation gate) and bench_round23_queries (live
Postgres; p50 latency + strict equivalence gate against seeded fixtures). See
benches/ROUND23.md.
- J1: contact_pg_repository::row_to_contact (+ the inlined contact_group sibling)
decode the 3 JSONB columns via sqlx::types::Json<T> (one from_slice pass)
instead of row.get::<serde_json::Value> + from_value (a throwaway Value DOM
per column, walked a second time). Per contact row of every list / multiget /
CardDAV sync. Micro 84 -> 33 allocs/op (2.15x); PG 3794 -> 2360 ns/contact
(1.61x) on 500 real rows.
- J2: DrivePolicies::from_value deserializes straight from the borrow
(T::deserialize(&Value)) instead of from_value(value.clone()) — dropping the
full-DOM clone on every drive-policy read (move/copy, share, grant); one-line
body change, all 7 callers unchanged. Micro 5 -> 0 allocs/op (11.51x).
- P1: get_user_profile overlaps the two independent caller+target reads with
tokio::join! (self-case still a single fetch; caller-error precedence
preserved via caller_res? first) instead of two serial round-trips. PG
577 -> 312 us/call (1.85x).
- G1: subject_group remove_member computes the child's transitive-user recursive
CTE once and reuses it for both the would-empty pre-check and the cache
invalidation, instead of running the identical CTE twice (the edge delete is
above the child, so its descendants can't change). PG 829 -> 412 us/removal
(2.01x).
- U1: dedup_service (store_loose_chunks final registration + the ingest
run_rollback) reshapes the owned, dead-after Vec<(String,i64)> via
into_iter().unzip() instead of cloning every 64-byte hash for the
sync_blobs(&[String]) + UNNEST bind. Micro 256 -> 0 hash clones.
Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo
fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0
failed. The PG benches run against a local PostgreSQL 16 (schema applied from
migrations/); every equivalence gate passes.
The download_zip per-item N+1 (the audit's highest raw-latency candidate) is
deferred to a dedicated pass: its fix moves the sole authorization inside the
stream call, so it needs an AuthZ-ordering + anti-enumeration proof, not a perf
banner.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
Benchmark-gated (benches/ROUND20.md), same rule as rounds 2-19: every change
ships with a BEFORE/AFTER counting-allocator micro-benchmark and a byte-value
equivalence gate; a non-winning AFTER is rolled back (never applied). The
rollback rule is encoded in the harness (GATE FAIL exit). All 8 sections pass.
Reproduce: cargo run --release --features bench --example bench_round20_micro
- A1 CalendarEvent iCal parse: replace the throwaway per-property
HashMap<String,Vec<String>> (DTSTART/DTEND/RECURRENCE-ID) with a direct
VALUE=DATE scan; prop_with_params kept #[cfg(test)] (6->2 allocs/event, 4.2x)
- A2 UserDto::from: add User::into_parts and MOVE image (<=512 KiB data URI)
+ ui_preferences JSON instead of cloning on every /api/auth/me (27->14 allocs)
- A3 parse_vcard: drop the per-line to_ascii_uppercase copy + the lines Vec;
promote ascii_ci_contains to common::text and share it (8->1 allocs/contact)
- A4 Calendar/AddressBook DTO: into_parts move incl. custom_properties map (18->10)
- I1 file-listing repos: collect::<Result<Vec>>() size-hints to 0 and grows from
capacity 0; pre-size with Vec::with_capacity (8->1 container reallocs, 4 sites)
- I4 plaintext_stream: lazy emit iterator instead of eager Vec collect (43x wall)
- C1 NC write_etag_element: borrowed pre-escaped quote events, no owned quoted
String/escape re-alloc; byte-identical output (3->0 allocs/PROPFIND row)
- C3 NC favorites REPORT: map.remove() move instead of get().clone() (~7 allocs/fav)
Deferred (documented in ROUND20.md): NC oc:id/trashbin buffer reuse, I1 sibling
CardDAV/CalDAV listing paths, Contact JSONB Json<Vec<_>> decode, dedup
settle_batch &str bind, and a fast DoS-resistant hasher for hot trusted-key maps
(needs a dependency decision).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JsJjcVX9RoN96DMa35Wqzd
Benchmark-gated (examples/bench_round19_micro.rs, benches/ROUND19.md): every
section ships a BEFORE/AFTER counting-allocator arm with a byte/-value
equivalence gate and a GATE-FAIL-rollback exit. All eight pass. No Postgres.
- M1 verify_basic_auth cache key: blake3::hash(format!("{u}:{p}")) → incremental
Hasher (byte-identical key, 2→0 allocs on every Basic-auth DAV request)
- M2 WopiTokenService: prebuild Validation/DecodingKey/EncodingKey in new()
instead of per-call (mirrors JwtTokenService; 16→12 allocs/validate)
- V1/V2 vCard emit (contact_to_vcard/generate_vcard): FN fallback drops the
throwaway to_string, NOTE skips the escape copy for newline-free notes, REV
uses new common::fmt::compact_ical_utc stack renderer (11.5× vs chrono
strftime, 3→0 allocs); per-contact 9→4 allocs
- M4 trash_service::row_to_item_dto: move name/path/blob_hash out of the owned
row instead of cloning (3 clones/file row gone)
- M5 search cache key: Uuid::hyphenated().encode_lower stack buffer instead of
to_string (identical u64 key, 1→0 allocs/request)
- M6 streaming PROPFIND: reuse one href buffer across the page instead of a
format! per child (native + NC handlers; 192→3 allocs on a 64-child page)
- M7 nextcloud extract_url_user: return Cow instead of forcing into_owned
(zero-alloc on the common ASCII-username path)
common::fmt::compact_ical_utc added with chrono-parity unit tests (CASES +
60-year sweep). cargo fmt + clippy --all-targets clean; 526 lib unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ront9bk7YMoffVQkGG47gh
Targets the content-addressable dedup write path (the ROUND15-deferred
"dedup_service hash-String re-allocations") from both ends — the streaming
ingest loop and the delta-commit verification read — plus a CardDAV vCard
micro-cut. Every change is benchmark-gated with a hard rollback rule; no
PostgreSQL needed for any arm (benches/ROUND17.md).
Backend (counting-allocator, examples/bench_round17_micro.rs):
- D2 chunk-ingest (store_from_stream, the hottest write path — every chunk of
every upload): the 64-char hex hash String was allocated 3x per chunk
(to_hex + chunk_hashes clone + session_seen insert-clone, the last dropped on
a duplicate). The intra-upload dedup set now keys on the raw 32-byte BLAKE3
digest ([u8;32], Copy, no heap) and the manifest push is branch-split so a
duplicate moves the hex in: 3 -> 2 allocs/new chunk, 3 -> 1/duplicate.
Measured 214 -> 149 allocs/op (1.14x wall) on a 64-chunk 1-in-2-dup batch;
smaller/faster set too (32B inline keys vs 64B heap Strings).
- D1 hash_chunk_sequence (delta-commit verification): took chunks by
&[(String,u64)] and fed the backend stream with iter().cloned(), re-cloning
every chunk hash a second time on top of the owned Vec the caller already
built. Take the Vec by value + into_iter(): 65 -> 0 internal allocs/op, ~2.5us
of clone work removed per verify.
- V1 vCard TYPE tokens (contact_to_vcard + generate_vcard, 5 sites): each
EMAIL/TEL/ADR TYPE= param used ty.to_uppercase() — a throw-away String per
token per contact. New shared fmt::push_upper writes the upper-cased chars
straight into the buffer (byte-identical to str::to_uppercase, unit-tested):
13 -> 5 allocs/op, 1.19x wall.
Gates: each section asserts byte/-value equivalence (D2 the ordered manifest +
sizes + write-set; D1 the removed clone is a pure copy; V1 the full vCard) and
exits non-zero if an AFTER arm fails to reduce allocations. push_upper is
unit-tested byte-equal to str::to_uppercase (fmt::tests). Verified end-to-end:
cargo fmt clean, clippy --release --all-targets --features bench -D warnings
clean, and the harness prints GATE PASS against the built release lib.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XMmt7vNETYUbEG3Hc17LDx
Benchmark-gated (benches/ROUND14.md); every change ships a BEFORE/AFTER
benchmark with an equivalence gate and is rolled back on regression (the
rule is encoded as a GATE FAIL exit / threshold expect).
Backend
- Q1 faces_for_file → narrow face_boxes_for_file(id, person_id, bbox) with the
caller filter pushed into SQL: drops the 2 KiB embedding BYTEA + 6 unused
columns per face. 15-face lightbox open 0.312→0.219 ms, 32 KB→840 B/req.
- A1 cookie auth uses the borrow-only extract_cookie_str (already backs CSRF)
instead of extract_cookie_value's owned String: -1 alloc/cookie request.
- A2 compute_relevance ASCII case-fold fast path vs name.to_lowercase() per
result row (Unicode fallback preserved): 1.40x, 12→3 allocs/page.
- A3 sub pre-parsed to Uuid at decode time (TokenClaims.sub_id) vs re-parsing
the 36-char claim on every request incl. cache hits: 22.7→0.7 ns.
- A4 auth + NextCloud middlewares borrow request.headers() instead of taking
axum's HeaderMap extractor (a full map clone): 2→0 allocs/authed request.
- A5 CalDAV getlastmodified via the stack rfc2822_utc (byte-identical to
chrono) vs a per-event to_rfc2822() heap String: 5→0 allocs.
- A6 CalDAV per-event href + quoted etag written into reused page buffers vs a
fresh format! pair per event: 3.48x, 240→6 allocs/40-event page.
Frontend
- F1 t() shares one frozen EMPTY_PARAMS for the no-interpolation call forms vs
a throwaway {} per call: -1 alloc/call.
- F2 favorites favoriteIds is a persistent SvelteSet with per-page add (clear
on reset) vs a brand-new set over the whole accumulated list each page:
22.3x over a 40-page drain (O(N^2)→O(N)).
Verified: cargo check --all-targets, cargo clippy -D warnings, both bench
packs (GATE PASS), frontend npm run check + vitest (4/4). ROUND14.md also
records the investigated-but-deferred backlog (music N+1, contact vcard
over-fetch, CachedBlobBackend syscalls, ResourceList.sections builder, etc.).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PymgCdK78NzUF3oRAQCJfN
The app-level ETag re-check before the write still left a gap between the check and the actual UPDATE for a concurrent writer to land in.
Push the check into the write path itself: swap_blob_hash now takes an expected_hash and only applies the SET under the same FOR UPDATE row lock it already held, closing the race instead of just narrowing it. Adds ErrorKind::PreconditionFailed (412) for the CAS-miss path; PUT/WOPI/chunked-upload keep blind-overwrite semantics by passing None
Benchmark-gated (BEFORE/AFTER + equivalence/safety gate per change), same
discipline as rounds 2-12. Full write-up in benches/ROUND13.md.
Shipped:
- V1 Grouped views windowed (files route + ResourceList). The grid arm was
the last unwindowed path (trash is grouped-by-default in grid): each
swimlane now feeds its own VirtualList, outer container a flex stack.
vitest gate: 800-item grouped grid mounts <120 .file-item (was 800).
- Q1 get_users_by_ids drops the <=512 KiB avatar image + ui_preferences
JSONB (notification path never reads them). 30-member fan-out 8.60 ->
0.25 ms (34.3x), ~7.7 MB off the wire.
- Q2 Login provisioning is_empty() -> SELECT EXISTS for calendar + address
book (every login). 0.193 -> 0.170 ms, widens with owned-row count.
- Q3 Recent-access prunes only when the upsert inserted (RETURNING xmax=0)
— a re-access can't grow the set. 0.567 -> 0.324 ms (1.75x).
- L1 Locale supported-codes precomputed once vs rebuilt per anonymous
request. 616 -> 17.3 ns (35.7x), 18 -> 1 allocs.
- H1 Duplicate /api TraceLayer removed (global stack already wraps it).
1.86 -> 1.42 us/request, -6 allocs.
- H2 client_ip span field: borrow-only ClientIpDisplay vs owned String.
187 -> 173 ns, -1 alloc.
Not shipped (discipline): the "media hooks read the blob 3x" lead was a
correctness bug, not a perf dup — the raw-path metadata/faces readers
resolve only for local+unencrypted+single-chunk blobs and silently produce
nothing otherwise. Flagged for maintainers; routing through read_blob_bytes
is a correctness fix (perf-neutral-to-negative), not a benchmark-gated
perf change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
Benchmark-gated round (benches/ROUND12.md; every change ships with a
BEFORE/AFTER harness + equivalence gates, one candidate rejected by its
own bench):
DB / query shapes (bench_round12_queries):
- NC sharee search: username-only projection instead of the 21-column row
(incl. the <=512 KiB avatar) per match, + gin_trgm_ops indexes on
auth.users for the leading-wildcard ILIKE (4.98x; 54.7x with index).
- Password login: delete the redundant full-row update_user — create_session
already stamps last_login_at in its own txn (4.45x per login).
- Email-verified stamp: narrow conditional UPDATE (8.9x); OIDC repeat login
now compares profile state in memory and issues ZERO queries when nothing
changed (was: full 17-column rewrite per login).
- Refresh rotation: revoke+insert+stamp fused into one transaction via new
rotate_session port method (1.18x).
- WOPI CheckFileInfo / authorize_wopi_access: require(Read) + get_file +
check(Update) overlapped with tokio::join!, original result precedence
(cold 1.34x).
- Upload quota gate: user-envelope + drive-cap checks fused into ONE
round-trip (check_upload_quotas) — the NC chunked PUT pays this per
chunk (1.81x, 2 -> 1 queries/chunk); shared verdict evaluators keep
error shapes byte-identical.
CPU / allocs (bench_round12_micro):
- sized_json: pre-sized listing serialization replacing axum Json's 128 B
seed + doubling-realloc chain on files/folder-resources/photos/search
responses (1.40x, 13 -> 2 allocs per 500-row page; byte-identical).
- Security headers: 4 SetResponseHeaderLayer folded into the CSP middleware
pass (5 layers -> 1; 1.43x per request, -26 allocs; header set gated
byte-identical incl. 304s).
- Media capture-metadata: single-read extraction — nom-exif now parses the
buffer kamadak already read (zero-copy Bytes) and videos open once with a
kind() dispatch; per-image opens 2-3 -> 1 (1.44x warm geomean, 1.6-3.2x
cold cache; extraction outputs gated identical incl. the MIME-mislabel
track fallback).
- Chunked-upload session ops: owner gate folded into the operation's own
DashMap lookup + stack-encoded uuid compare (5 -> 3 lookups, -2 allocs,
1.28x per chunk).
Blob cache (bench_blob_cache_index + round-3 regression guard):
- CachedBlobBackend index: tokio::sync::Mutex<LruCache> -> moka::sync::Cache
with byte weigher. The mutex serialized every cached chunk read and scaled
NEGATIVELY (2.08 -> 1.07 Mops/s from 1 -> 2 readers); moka probes are
lock-free (2.17x at K=2). Byte budget now enforced by moka (manual
current_size + collect_evictions machinery deleted); eviction listener
unlinks size-evicted files only (Replaced entries keep their file —
gated). Single-flight miss gate unchanged (16 concurrent misses -> 1
fetch re-verified via the round-3 harness).
- put_blob now populates the cache BEFORE the inner backend consumes the
source file (the old order failed 100% of the time — local renames,
S3/Azure delete the source — so the first read after a whole-file put
re-downloaded from the remote); inner-put failure invalidates the entry.
Frontend (vitest gates):
- List-view thumbnails request the 150px icon rendition instead of 400px
preview into a 40px slot (~7.1x fewer pixels, ~4-5x fewer bytes per
thumbnail across list views); grid keeps preview.
Rejected by its own bench (kept as evidence in bench_round12_micro §2):
- Single-pass compression predicate: the monomorphized And-chain already
costs ~4.6 ns / 0 allocs total; the fused node measured within noise.
New migration: 20260719000000_users_search_trgm.sql (trgm indexes).
Deferred with prepared design: grouped file/grid view virtualization
(single-VirtualRows flatten, the photos pattern) — next round's headline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
Backend (each change benchmark-gated with BEFORE replicas + equivalence
gates; see examples/bench_round11_micro.rs, bench_round11_queries.rs,
bench_log_writer.rs and benches/ROUND11.md — final numbers land in the
follow-up doc commit):
- StoragePath re-representation: single canonical joined String, segments
derived on demand; File/Folder drop the duplicated path_string field
(4000→1000 allocs per 500-row listing page)
- Display classifier fusion: classify_display shares one stack-lowered
extension across the three decision trees; call sites in FileDto,
folder/favorites/recent handlers, trash, path-resolver (+ interning
where Arc::from was still used)
- /status.php and /openapi.json memoized into OnceLock<Bytes> (openapi
rebuilt a 171 KiB spec per request: 2.8 ms → 18 ns)
- NC upload-session PROPFIND: write! + pre-sized body + stack RFC2822
dates (2.3-2.6x, 2582→772 allocs at 256 chunks)
- REST download: dead FileDto clone removed (capture mime/size + move)
- CalendarEventDto/TrashedItem into_parts moves (11 KiB ical_data memcpy
gone per CalDAV row); CardDAV getlastmodified stack render
- 4xx path: borrowed ErrorResponse serialize, ErrorKind::as_str,
not_found/already_exists clone kill
- vCard emit via write!; search page moved out with into_iter skip/take;
content-hit UUIDs parsed once; group last-user check via HashSet
- RateLimiter: lock-free get + insert (and_upsert_with variant REJECTED
by benchmark); CSRF token borrow-compare + borrowed cookie extraction
- Thumbnail/preview ETags built from as_str (Debug-identical bytes)
- Encrypted backend: encrypt_in_place_detached single-buffer write path,
chunk-sized reserve in collect_stream; retry labels made lazy
- PG: deferred upload registration 3→1 round-trips (persist_file CTE
template); direct_grant_cache for Calendar/AddressBook/Playlist authz
(single-flight + set_role/clear_role invalidation); expand_user
tokio::join!; geo clusters min(uuid)::text; recluster face assignment
batched into one UNNEST update
- People recluster cosine: norms precomputed once (bit-identical gate)
- NC capabilities poll logs demoted to debug; tracing-appender dep added
for the log-writer benchmark
Frontend:
- ResourceList.selectedEntries O(N)-per-toggle → id-index projection
O(k log k); favorites/recent consume the batchToolbar snippet param and
drop their duplicate filter + dead selectedIds mirror
- Recent: star state via new favoriteIds prop — a star click no longer
rebuilds all N entries
- admin timeAgo >30d fallback uses the cached Intl.DateTimeFormat
- vitest gates in src/lib/components/round11.bench.test.ts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
Benchmark-gated (equivalence + BEFORE/AFTER; results + reproduce commands in
benches/ROUND7.md):
- Photos timeline re-grouped + re-laid-out the whole accumulated library on
every 60-item page (both `groups` and `photoRows` were $derived over the
full list), Σ ≈ O(N²/60) main-thread work during a scroll. Pages arrive
newest-first so grouping is append-only: the new PhotoTimeline
(lib/utils/photoTimeline.ts) re-buckets only the fresh page and re-lays-out
only changed groups, reusing untouched groups' cached rows, falling back to
a full rebuild on any config/deletion/non-append change. The pure
buildPhotoRows is the verbatim reference the gate holds it equal to at every
page. 50×60 drain: 76 500 → 3 000 grouping ops (25.5x), 23.0 → 2.2 ms
(10.6x).
- Range downloads paid authz + access-notify twice: download_file_impl
resolves the file via get_file_with_perms, then the Range branch re-ran
require_file + notify_file_accessed per request. Media/PDF viewers fetch
exclusively via Range (one request per seek), so every seek in a scrub
re-authorized an already-cleared file. Now routed through the non-perms
get_file_range_preloaded (matching the share-landing + WebDAV range paths);
the unused _with_perms range method is removed. The request-level gate still
denies before the branch runs (bench asserts member granted, outsider
denied). Per seek removed: WARM 0.67 µs, COLD 1362.66 µs — a grant-cascade
drive-resolve query per seek for a shared-drive recipient on a cold cache.
- /api/folders/{id}/resources row→DTO mapping cloned row.name into the DTO
though the row is owned; folders move it (fixed icons), files compute the
name-derived icon/category classes first then move it. 500-row page:
10.004 → 9.004 allocs/row (500 clones removed), output identical.
Deferred with rationale in ROUND7.md: thumbnail ACL-before-304 (security
posture — needs a security review, not a perf tweak), batch_operations
Arc<str>→String widening, list-view O(N²) on smaller lists, and the serial→
join! pairs (decide-by-bench with injected latency, per the round-6 rejection).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
Benchmark-gated (equivalence + BEFORE/AFTER in examples/bench_*, results
and reproduce commands in benches/ROUND6.md):
- CardDAV whole-book REPORT + depth-1 PROPFIND stream through a PG
cursor (stream_contacts_by_book, 500-contact pages) instead of
materialising every vCard twice: 8 000 contacts TTFB 37.4 → 7.6 ms
(4.9x), peak heap 19.0 → 7.0 MiB (2.7x), wall -23%; REPORT and
PROPFIND byte-identical to the buffered writers.
- NC numeric-id chain fully borrowed: get_or_create_file_ids/folder_ids
take &[&str] and return HashMap<Uuid, i64>; batch_resolve_ids callers
(PROPFIND pages, REPORT, trashbin, OCS search) pass id slices and look
up via nc_id_of. 2.006 → 0.006 allocs/child (334x), 1.53x wall per
500-child page. batch_check_favorites binds &[&str] as text[].
- file_blob_read_repository listing SELECTs drop id::text/folder_id::text
server casts: rows decode binary Uuid (16 vs 36 bytes on the wire) and
render once in row_to_file. A/B on 500-row pages: 1.225 → 1.044 ms
mean (1.17x), p95 1.686 → 1.345 (bench_uuid_text_cast; single-row,
param and min() sites left as-is deliberately).
- IncrementalHasher::finalize_hex renders through common::fmt::hex_lower
instead of one format! per digest byte: 18 → 1 (md5) / 35 → 1 (sha256)
allocs per chunk finalize, 14-15x wall.
- Share landing overlaps the access-count UPDATE with the unlock fetch
via tokio::join! (one round-trip off every public link hit).
- REJECTED by benchmark and reverted: try_join_all fan-out of the
batch-favorites authz pre-check — 42.6 → 56.4 ms cold, 0.15 → 0.23 ms
warm against local-socket PG (bench_favorites_authz kept as evidence).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
this fix https://github.com/AtalayaLabs/OxiCloud/issues/607
which was introduced by commit 12dc648cff
when a user does activity in a drive, admin can invalidate cache via the internal
call /api/admin/internal/trigger-sweep
this permit end 2 end test to validte immediately that used_bytes corresponds to the expected result
this fix https://github.com/AtalayaLabs/OxiCloud/issues/607
which was introduced by commit 12dc648cff
when a user rename a root folder, this invalidate the cache
still some UX effect displaying phantom drive is grant is revoked,
cache is 30s of TTL so this UX glitch is acceptable
- nc chunked-upload MOVE assembly (#12): both branches now funnel through
update_file_streaming_with_perms, whose internal fork enforces Update on
the existing file OR Create on the parent folder / drive root. Pre-fix,
the create branch went through plain upload_file_streaming with no
authz.require — a Viewer on a shared drive could MKCOL → PUT chunks →
MOVE and land a brand-new file. Error mapping switched to AppError::from
so denials keep the graduated 403/404 shape.
- trash empty-for-drive: route through authz.require(Delete, Drive) instead
of the bespoke drives_with_delete_for check + hardcoded not_found. Viewer
now gets 403 (has Read), outsider stays 404 (no Read, anti-enum). Emits
the standard authz.denied event with visibility field instead of the
ad-hoc trash.empty_drive_rejected.
- tests/api/trash_per_drive.hurl: flip Viewer/Editor asserts 404 → 403;
new Step 11b regression pin for finding #10 (Editor restore + delete
attempts must 403 AND body must not contain "success":true — trips if
the historical substring-match-on-"not found" hack ever comes back).
Edouard Vanbelle