- provide is_shared and is_favorite information in DTO, information propagated as badge/buttons per items
regarding performances I try to be minimalis on SQL to prevent any perf regression
- remove old set of sharedids and favoriteids (was not functionnal anymore)
- fix date picker (no past selection) in grant
- fix contextmenu close on /files section
Resolve conflicts between the external-file-mounts feature and upstream's
D5/D7 refactor (per-file provenance, keyset pagination, cross-drive move
gates, resource-access hook, folder-cascade lifecycle hook).
Key resolutions:
- FolderService::new now takes (repo, authz, file_lifecycle, mount_router);
all callers + DI updated.
- FileRetrievalService / FileManagementService keep both the mount_router
and the new resource_access_hook / drive_repo / storage_usage wiring.
- list_files_batch_with_perms: adapt the mount branch from offset- to
keyset (after_name) pagination, mirroring paginate_mount_entries.
- download_file_impl: keep upstream's &HeaderMap + `impl IntoResponse + use<>`
signature, retain the mount-download branch.
- Mount DTOs: the retired `owner_id` field maps onto created_by/updated_by
(the mount owner) — the fields the frontend now uses for owner display.
- admin/+page.svelte: keep upstream's user-delete modal + the 'mounts' tab.
- Bump memmap2 0.9.10 -> 0.9.11 (RUSTSEC critical advisory fix) and
regenerate Cargo.lock against the merged Cargo.toml.
Seven behaviour-preserving allocation / copy / bandwidth cuts, each behind a
counting-allocator BEFORE/AFTER gate that exit(1)s unless AFTER allocates
strictly fewer than BEFORE (benches/ROUND29.md, examples/bench_round29_micro.rs).
- [B] Content-cache serve fast path (optimized_inner Tier 1 +
get_file_range_preloaded — the video-scrub hot path): probe the cache with a
borrow first and build the owned get_or_load args (quoted-etag / key / id
Strings) only on a miss, instead of allocating them before every probe and
discarding them on a hit. 6 -> 0 allocs per cache hit. Splits get_or_load into
get + load_and_cache so the miss path is not re-probed and the hit/miss stat
counters stay byte-identical. Also drops the unconditional content_hash/name
clones that ran for the >=10 MB streaming tier that used neither.
- [A] NextCloud REPORT emit loops: per-row href String (and format! per folder
row) -> one reused href_buf via nc_href_into / nc_collection_href_into with the
URL-encoded user computed once per page. 1497 fewer allocs on a 500-row page.
- [C] read_full: a single-frame blob is returned zero-copy instead of a second
whole-payload memcpy into a fresh BytesMut; multi-frame path unchanged.
- [D] login-lockout key: to_lowercase()+format! -> one pre-sized ASCII buffer
(non-ASCII keeps str::to_lowercase). 3 -> 1 alloc/req, byte-identical key.
- [E] NC composite-username parse: owned clone/to_string -> &str borrow of the
already-owned raw_username. 1 -> 0 alloc on the common no-marker path.
- [F] get_contacts_in_group: stop SELECTing the discarded multi-KB vcard column
(the live method ROUND25 §Q2 missed; ContactDto has no vcard field).
- [G] count_admin_users: add count_users_by_role -> scalar COUNT(*) instead of
hydrating every admin's full row (incl. up-to-512 KiB avatar + ui_preferences
JSONB) only to .len() it, on a bootstrap-polled status endpoint.
All seven gates pass; cargo fmt --check and cargo clippy --all-features
--all-targets -D warnings clean. §F/§G additionally validated against a live
PostgreSQL 16 with the full migration set (query validity, result equivalence,
600000 -> 8 byte wire delta on the admin count).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LhpDZxSQTAGnAqCHUdtG5N
Three benchmark-gated optimizations from the ROUND25 backlog (benches/ROUND26.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- P1 drive_pg_repository policy reads: decode d.policies through
sqlx::types::Json<DrivePolicies> (one from_slice over the raw JSONB bytes)
instead of a throwaway serde_json::Value DOM + DrivePolicies::from_value —
6 -> 0 allocs/read, 2.77x wall. A shared policies_from_row helper preserves
the lenient unwrap_or_default fallback (malformed bag -> all-false).
- D1 CachedBlobBackend: pre-create the 256 {00..ff} shard dirs at initialize()
(mirroring LocalBlobBackend, reusing HEX_PREFIXES) and drop the redundant
per-write create_dir_all on already-existing shards — ~45us + a blocking-pool
dispatch removed per cache write on cached-remote deployments.
- G1 delta-upload have/need hash sets (distinct_hashes, authorize_chunk_download):
SipHash -> foldhash::quality::RandomState — a fast hasher that stays
DoS-resistant via a per-instance random seed, the required property for the
attacker-controlled 64-hex client hashes — 2.37x wall on a 40k-hash
negotiation. foldhash was already in the lockfile transitively (hashbrown).
Tested and REVERTED (kept as-is): moving the moka eviction unlink off the reactor
via spawn_blocking. The benchmark refuted it — on the local cache dir the
spawn_blocking dispatch (~20us) costs more than the inline unlink (~7us) it would
replace. See ROUND26.md §D2.
Adds bench_round26_{micro,diskio,hasher} (counting allocator / async wall / wall).
Verified: cargo fmt clean, cargo clippy --features bench -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Five benchmark-gated optimizations from a fresh six-way audit (benches/ROUND25.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- M1 EncryptedBlobBackend::decrypt_bytes: replace split_off (a fresh Vec + full
ciphertext memcpy on every decrypted chunk, contradicting its own "in place"
doc) with in-place detached decrypt + a zero-copy Bytes::slice past the nonce.
Peak RAM per read drops from ~2x to ~1x the payload (-262KB/op at 256KiB;
scales with blob size). Plaintext byte-identical; tamper/wrong-key tests pass.
- M2 delta commit: move-unzip the owned chunk list instead of a third
per-occurrence hash clone (-4000 allocs on a 4000-chunk commit).
- M3 folder ZIP download: drop the dead Query<HashMap> extractor it never read
(byte-identical response; 5->0 allocs/request).
- Q1 public-playlist listing: fold the per-playlist COUNT(*) N+1 into one
LEFT JOIN ... GROUP BY via a new inherent repo method (101 -> 1 round-trips,
36x wall on a 100-playlist page).
- Q2 contact REST listings (paginated/search/by-group): stop over-fetching the
multi-KB vcard TEXT the ContactDto discards, via a shared lite row mapper and
narrowed SELECTs (6.4x wall on 1000 contacts with 8KiB vcards). The whole-book
vCard export and CardDAV sync paths keep the column.
Adds bench_round25_micro (counting allocator tracking count+bytes) and
bench_round25_queries (live Postgres), both with equivalence gates and a
rollback exit(1). Verified: cargo fmt clean, cargo clippy -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Benchmark-gated, same rule as ROUND2-22: BEFORE/AFTER with a value-equivalence
gate and rollback-on-regression. Two harnesses — bench_round23_micro (no
Postgres; deterministic allocation gate) and bench_round23_queries (live
Postgres; p50 latency + strict equivalence gate against seeded fixtures). See
benches/ROUND23.md.
- J1: contact_pg_repository::row_to_contact (+ the inlined contact_group sibling)
decode the 3 JSONB columns via sqlx::types::Json<T> (one from_slice pass)
instead of row.get::<serde_json::Value> + from_value (a throwaway Value DOM
per column, walked a second time). Per contact row of every list / multiget /
CardDAV sync. Micro 84 -> 33 allocs/op (2.15x); PG 3794 -> 2360 ns/contact
(1.61x) on 500 real rows.
- J2: DrivePolicies::from_value deserializes straight from the borrow
(T::deserialize(&Value)) instead of from_value(value.clone()) — dropping the
full-DOM clone on every drive-policy read (move/copy, share, grant); one-line
body change, all 7 callers unchanged. Micro 5 -> 0 allocs/op (11.51x).
- P1: get_user_profile overlaps the two independent caller+target reads with
tokio::join! (self-case still a single fetch; caller-error precedence
preserved via caller_res? first) instead of two serial round-trips. PG
577 -> 312 us/call (1.85x).
- G1: subject_group remove_member computes the child's transitive-user recursive
CTE once and reuses it for both the would-empty pre-check and the cache
invalidation, instead of running the identical CTE twice (the edge delete is
above the child, so its descendants can't change). PG 829 -> 412 us/removal
(2.01x).
- U1: dedup_service (store_loose_chunks final registration + the ingest
run_rollback) reshapes the owned, dead-after Vec<(String,i64)> via
into_iter().unzip() instead of cloning every 64-byte hash for the
sync_blobs(&[String]) + UNNEST bind. Micro 256 -> 0 hash clones.
Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo
fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0
failed. The PG benches run against a local PostgreSQL 16 (schema applied from
migrations/); every equivalence gate passes.
The download_zip per-item N+1 (the audit's highest raw-latency candidate) is
deferred to a dedicated pass: its fix moves the sole authorization inside the
stream call, so it needs an AuthZ-ordering + anti-enumeration proof, not a perf
banner.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
Round 21 of the benchmark-gated perf sweep. Six behaviour-preserving,
allocation-reducing changes, each with a BEFORE/AFTER counting-allocator
section in examples/bench_round21_micro.rs and a byte/-value equivalence
gate; all six pass their deterministic alloc gate (a non-winning AFTER
exits 1 = rollback).
- R1: pre-size the 16 CalDAV/CardDAV row-mapper Vecs (+1 HashMap) with
Vec::with_capacity(rows.len()) — the ROUND20 §I1 file-side pattern
extended to the calendar/contact repos it deferred. 7 → 1 allocs/op.
- R2: settle_batch binds a borrowed Vec<&str> instead of cloning every
chunk hash into a Vec<String> (sqlx encodes &[&str] as text[]
identically; favorites_pg_repository.rs:271 precedent). 33 → 1 allocs/op,
39x wall.
- R3: store_loose_chunks keys its intra-request dedup set on the raw
[u8;32] BLAKE3 digest and moves the hex on a duplicate (the ROUND17 §D2
pattern applied to the delta-upload sibling). 401 → 209 allocs/op.
- R4: CardDAV getetag emits borrowed pre-escaped " quotes via a shared
write_quoted_etag helper (ROUND20 §C1 pattern, all 4 CardDAV etag sites).
3 → 0 allocs/op.
- R5: BDAY stamped via the new fmt::compact_date stack renderer instead of
chrono's strftime interpreter (chrono fallback out of the 4-digit-year
range; byte-identical, unit-tested vs chrono). 2 → 0 allocs/op, 10.5x wall.
- R6: NC trashbin folder content-type via Cow::Borrowed instead of
.to_string() on the constant (ROUND16 §M1 pattern). 1 → 0 allocs/op.
See benches/ROUND21.md for the full write-up and the deferred-items list
(HeaderMap-clone hot handlers, Query→typed-struct, WebDAV dead-props
HashSet, and others surfaced by the audit that want their own validated
pass). Validated: cargo fmt, cargo clippy --features bench --all-targets
-D warnings, cargo test --lib --features test_utils (529 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015gHVq5Wy2TzdWeSqtEmK6m
Benchmark-gated (benches/ROUND20.md), same rule as rounds 2-19: every change
ships with a BEFORE/AFTER counting-allocator micro-benchmark and a byte-value
equivalence gate; a non-winning AFTER is rolled back (never applied). The
rollback rule is encoded in the harness (GATE FAIL exit). All 8 sections pass.
Reproduce: cargo run --release --features bench --example bench_round20_micro
- A1 CalendarEvent iCal parse: replace the throwaway per-property
HashMap<String,Vec<String>> (DTSTART/DTEND/RECURRENCE-ID) with a direct
VALUE=DATE scan; prop_with_params kept #[cfg(test)] (6->2 allocs/event, 4.2x)
- A2 UserDto::from: add User::into_parts and MOVE image (<=512 KiB data URI)
+ ui_preferences JSON instead of cloning on every /api/auth/me (27->14 allocs)
- A3 parse_vcard: drop the per-line to_ascii_uppercase copy + the lines Vec;
promote ascii_ci_contains to common::text and share it (8->1 allocs/contact)
- A4 Calendar/AddressBook DTO: into_parts move incl. custom_properties map (18->10)
- I1 file-listing repos: collect::<Result<Vec>>() size-hints to 0 and grows from
capacity 0; pre-size with Vec::with_capacity (8->1 container reallocs, 4 sites)
- I4 plaintext_stream: lazy emit iterator instead of eager Vec collect (43x wall)
- C1 NC write_etag_element: borrowed pre-escaped quote events, no owned quoted
String/escape re-alloc; byte-identical output (3->0 allocs/PROPFIND row)
- C3 NC favorites REPORT: map.remove() move instead of get().clone() (~7 allocs/fav)
Deferred (documented in ROUND20.md): NC oc:id/trashbin buffer reuse, I1 sibling
CardDAV/CalDAV listing paths, Contact JSONB Json<Vec<_>> decode, dedup
settle_batch &str bind, and a fast DoS-resistant hasher for hot trusted-key maps
(needs a dependency decision).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JsJjcVX9RoN96DMa35Wqzd
Targets the content-addressable dedup write path (the ROUND15-deferred
"dedup_service hash-String re-allocations") from both ends — the streaming
ingest loop and the delta-commit verification read — plus a CardDAV vCard
micro-cut. Every change is benchmark-gated with a hard rollback rule; no
PostgreSQL needed for any arm (benches/ROUND17.md).
Backend (counting-allocator, examples/bench_round17_micro.rs):
- D2 chunk-ingest (store_from_stream, the hottest write path — every chunk of
every upload): the 64-char hex hash String was allocated 3x per chunk
(to_hex + chunk_hashes clone + session_seen insert-clone, the last dropped on
a duplicate). The intra-upload dedup set now keys on the raw 32-byte BLAKE3
digest ([u8;32], Copy, no heap) and the manifest push is branch-split so a
duplicate moves the hex in: 3 -> 2 allocs/new chunk, 3 -> 1/duplicate.
Measured 214 -> 149 allocs/op (1.14x wall) on a 64-chunk 1-in-2-dup batch;
smaller/faster set too (32B inline keys vs 64B heap Strings).
- D1 hash_chunk_sequence (delta-commit verification): took chunks by
&[(String,u64)] and fed the backend stream with iter().cloned(), re-cloning
every chunk hash a second time on top of the owned Vec the caller already
built. Take the Vec by value + into_iter(): 65 -> 0 internal allocs/op, ~2.5us
of clone work removed per verify.
- V1 vCard TYPE tokens (contact_to_vcard + generate_vcard, 5 sites): each
EMAIL/TEL/ADR TYPE= param used ty.to_uppercase() — a throw-away String per
token per contact. New shared fmt::push_upper writes the upper-cased chars
straight into the buffer (byte-identical to str::to_uppercase, unit-tested):
13 -> 5 allocs/op, 1.19x wall.
Gates: each section asserts byte/-value equivalence (D2 the ordered manifest +
sizes + write-set; D1 the removed clone is a pure copy; V1 the full vCard) and
exits non-zero if an AFTER arm fails to reduce allocations. push_upper is
unit-tested byte-equal to str::to_uppercase (fmt::tests). Verified end-to-end:
cargo fmt clean, clippy --release --all-targets --features bench -D warnings
clean, and the harness prints GATE PASS against the built release lib.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XMmt7vNETYUbEG3Hc17LDx
Edouard Vanbelle