add /api/folders/{id}/ancestors
this API to iterate parent up to the drive root or the shared folder
this will help UI to build the breadcrumb in 1 API call
and to identify the root element (is it a drive users has access to or
a shared folder ?)
ui: now only 1 API call is now required to build the breadcrumb
- provide is_shared and is_favorite information in DTO, information propagated as badge/buttons per items
regarding performances I try to be minimalis on SQL to prevent any perf regression
- remove old set of sharedids and favoriteids (was not functionnal anymore)
- fix date picker (no past selection) in grant
- fix contextmenu close on /files section
- rustfmt the integration-test FolderService::new callsites added during
the upstream merge (long single-line args wrapped).
- Cargo.lock: rebuild from upstream/main's lock so only the
testcontainers-modules dev-dep subtree and the memmap2 0.9.10→0.9.11
security bump differ (avoids churning upstream's pins).
- .cargo/audit.toml: ignore the four astral-tokio-tar tar-extraction
advisories — dev-only, transitive via testcontainers-modules
(integration-test harness), never in the production binary.
Resolve conflicts between the external-file-mounts feature and upstream's
D5/D7 refactor (per-file provenance, keyset pagination, cross-drive move
gates, resource-access hook, folder-cascade lifecycle hook).
Key resolutions:
- FolderService::new now takes (repo, authz, file_lifecycle, mount_router);
all callers + DI updated.
- FileRetrievalService / FileManagementService keep both the mount_router
and the new resource_access_hook / drive_repo / storage_usage wiring.
- list_files_batch_with_perms: adapt the mount branch from offset- to
keyset (after_name) pagination, mirroring paginate_mount_entries.
- download_file_impl: keep upstream's &HeaderMap + `impl IntoResponse + use<>`
signature, retain the mount-download branch.
- Mount DTOs: the retired `owner_id` field maps onto created_by/updated_by
(the mount owner) — the fields the frontend now uses for owner display.
- admin/+page.svelte: keep upstream's user-delete modal + the 'mounts' tab.
- Bump memmap2 0.9.10 -> 0.9.11 (RUSTSEC critical advisory fix) and
regenerate Cargo.lock against the merged Cargo.toml.
Seven behaviour-preserving allocation / copy / bandwidth cuts, each behind a
counting-allocator BEFORE/AFTER gate that exit(1)s unless AFTER allocates
strictly fewer than BEFORE (benches/ROUND29.md, examples/bench_round29_micro.rs).
- [B] Content-cache serve fast path (optimized_inner Tier 1 +
get_file_range_preloaded — the video-scrub hot path): probe the cache with a
borrow first and build the owned get_or_load args (quoted-etag / key / id
Strings) only on a miss, instead of allocating them before every probe and
discarding them on a hit. 6 -> 0 allocs per cache hit. Splits get_or_load into
get + load_and_cache so the miss path is not re-probed and the hit/miss stat
counters stay byte-identical. Also drops the unconditional content_hash/name
clones that ran for the >=10 MB streaming tier that used neither.
- [A] NextCloud REPORT emit loops: per-row href String (and format! per folder
row) -> one reused href_buf via nc_href_into / nc_collection_href_into with the
URL-encoded user computed once per page. 1497 fewer allocs on a 500-row page.
- [C] read_full: a single-frame blob is returned zero-copy instead of a second
whole-payload memcpy into a fresh BytesMut; multi-frame path unchanged.
- [D] login-lockout key: to_lowercase()+format! -> one pre-sized ASCII buffer
(non-ASCII keeps str::to_lowercase). 3 -> 1 alloc/req, byte-identical key.
- [E] NC composite-username parse: owned clone/to_string -> &str borrow of the
already-owned raw_username. 1 -> 0 alloc on the common no-marker path.
- [F] get_contacts_in_group: stop SELECTing the discarded multi-KB vcard column
(the live method ROUND25 §Q2 missed; ContactDto has no vcard field).
- [G] count_admin_users: add count_users_by_role -> scalar COUNT(*) instead of
hydrating every admin's full row (incl. up-to-512 KiB avatar + ui_preferences
JSONB) only to .len() it, on a bootstrap-polled status endpoint.
All seven gates pass; cargo fmt --check and cargo clippy --all-features
--all-targets -D warnings clean. §F/§G additionally validated against a live
PostgreSQL 16 with the full migration set (query validity, result equivalence,
600000 -> 8 byte wire delta on the admin count).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LhpDZxSQTAGnAqCHUdtG5N
Adds application/adapters/param_encoding.rs: encode/decode for the three
RFC 6868 caret-escape sequences (^n newline, ^^ literal caret, ^' literal
DQUOTE) plus render_param_value, which quotes and encodes a parameter
value only when the vCard/iCalendar param-value grammar requires it
(comma/semicolon/colon triggers quoting; an embedded DQUOTE/caret/newline
needs escaping even once quoted).
Wires render_param_value into the two vCard generators' TYPE parameter
emission (EMAIL/TEL/ADR) in carddav_adapter.rs::contact_to_vcard and
contact_service.rs::generate_vcard — contact `type` fields are free text,
not a fixed enum, so a value containing a comma or quote previously
produced outright broken vCard syntax (no quoting/escaping was applied
at all).
Scope note: this branch only fixes the generation side. The existing
vCard parser is a naive line-by-line substring scanner (`line.contains
("TYPE=WORK")`) with no real per-parameter tokenizing, so there's no
integration point yet for the decode half without a larger parser
rewrite — that's part of the vCard 4.0 work (RFC 6350), which depends on
this branch for the shared encoding utility.
report_handler's four REPORT emit loops (the filter-files favorites REPORT and the
search REPORT, each a file loop + a folder loop) shared the same per-row oc:id
String that ROUND27 §H1 replaced in the two PROPFIND page loops. Apply the
identical, already-validated transformation: hoist one oc_buf per handler (reused
across both its loops) and compute the id into it via format_oc_id_into instead of
a fresh format_oc_id String per child. 1 String/row -> 0 (amortized). The
write_{file,folder}_response fns already take Option<&str>, so their signatures are
unchanged and the emitted oc:id bytes are byte-identical.
Same transformation benchmarked in ROUND27 §H1 (bench_round27_micro: 998 -> 0
per-row allocs, 2.16x wall), so no new bench. Verified: cargo fmt clean, cargo
clippy --features bench -D warnings clean, cargo test --lib --features bench =
529 passed / 0 failed across 5 consecutive runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Two behaviour-preserving allocation cuts (benches/ROUND27.md), each with a
counting-allocator BEFORE/AFTER gate that rolls back if AFTER does not allocate
fewer than BEFORE:
- H1 NextCloud PROPFIND: the streaming page loops built oc:id as a fresh String
per child (format_oc_id -> format!("{:08}{}", id, instance)). Add
format_oc_id_into(&mut buf, id, svc) and compute into one oc_buf reused across
the page (next to the existing href buffer) — 1 String/row -> 0. 998->0
per-row allocs on a 500-row page, 2.16x wall. The write fns still take
Option<&str>, so no signature change; oc:id bytes identical. Scoped to the two
PROPFIND page loops (the hot directory-listing path); REPORT/trashbin deferred.
- P2 contact create/update: bind sqlx::types::Json(&dtos) (Encode runs to_writer
straight into the JSONB buffer) instead of serde_json::to_value(&dtos) + bind,
which built a throwaway Value DOM per JSONB column. Write-side twin of ROUND23
J1. 21->2 allocs, 4.68x wall for a 3-entry column. Behaviour-preserving:
to_value sorts keys and direct serialize keeps struct order, but Postgres
normalizes JSONB key order so the stored value is identical (verified via psql:
'{...alpha...}'::jsonb = '{...struct...}'::jsonb -> t), and reads decode by
field name; the etag comes from the domain entity, not the stored JSONB.
Adds bench_round27_micro. Verified: cargo fmt clean, cargo clippy --features
bench -D warnings clean (real exit), cargo test --lib --features bench = 529
passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Three benchmark-gated optimizations from the ROUND25 backlog (benches/ROUND26.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- P1 drive_pg_repository policy reads: decode d.policies through
sqlx::types::Json<DrivePolicies> (one from_slice over the raw JSONB bytes)
instead of a throwaway serde_json::Value DOM + DrivePolicies::from_value —
6 -> 0 allocs/read, 2.77x wall. A shared policies_from_row helper preserves
the lenient unwrap_or_default fallback (malformed bag -> all-false).
- D1 CachedBlobBackend: pre-create the 256 {00..ff} shard dirs at initialize()
(mirroring LocalBlobBackend, reusing HEX_PREFIXES) and drop the redundant
per-write create_dir_all on already-existing shards — ~45us + a blocking-pool
dispatch removed per cache write on cached-remote deployments.
- G1 delta-upload have/need hash sets (distinct_hashes, authorize_chunk_download):
SipHash -> foldhash::quality::RandomState — a fast hasher that stays
DoS-resistant via a per-instance random seed, the required property for the
attacker-controlled 64-hex client hashes — 2.37x wall on a 40k-hash
negotiation. foldhash was already in the lockfile transitively (hashbrown).
Tested and REVERTED (kept as-is): moving the moka eviction unlink off the reactor
via spawn_blocking. The benchmark refuted it — on the local cache dir the
spawn_blocking dispatch (~20us) costs more than the inline unlink (~7us) it would
replace. See ROUND26.md §D2.
Adds bench_round26_{micro,diskio,hasher} (counting allocator / async wall / wall).
Verified: cargo fmt clean, cargo clippy --features bench -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Five benchmark-gated optimizations from a fresh six-way audit (benches/ROUND25.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- M1 EncryptedBlobBackend::decrypt_bytes: replace split_off (a fresh Vec + full
ciphertext memcpy on every decrypted chunk, contradicting its own "in place"
doc) with in-place detached decrypt + a zero-copy Bytes::slice past the nonce.
Peak RAM per read drops from ~2x to ~1x the payload (-262KB/op at 256KiB;
scales with blob size). Plaintext byte-identical; tamper/wrong-key tests pass.
- M2 delta commit: move-unzip the owned chunk list instead of a third
per-occurrence hash clone (-4000 allocs on a 4000-chunk commit).
- M3 folder ZIP download: drop the dead Query<HashMap> extractor it never read
(byte-identical response; 5->0 allocs/request).
- Q1 public-playlist listing: fold the per-playlist COUNT(*) N+1 into one
LEFT JOIN ... GROUP BY via a new inherent repo method (101 -> 1 round-trips,
36x wall on a 100-playlist page).
- Q2 contact REST listings (paginated/search/by-group): stop over-fetching the
multi-KB vcard TEXT the ContactDto discards, via a shared lite row mapper and
narrowed SELECTs (6.4x wall on 1000 contacts with 8KiB vcards). The whole-book
vCard export and CardDAV sync paths keep the column.
Adds bench_round25_micro (counting allocator tracking count+bytes) and
bench_round25_queries (live Postgres), both with equivalence gates and a
rollback exit(1). Verified: cargo fmt clean, cargo clippy -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
The ROUND23-deferred download_zip N+1, given its own validated pass. The
individually-selected files were authorized + fetched one at a time via
get_file_with_perms (require + get = 2 serial round-trips/file) before any
streaming — a 200-file selection was 400 serial round-trips. AFTER routes the
whole multi-select through the new
FileRetrievalService::get_files_by_ids_with_perms: one check_files_read_batch
(the PgAclEngine resolves every file's drive in ONE query and primes the
resource->drive cache) + one get_files_by_ids. 2N round-trips -> 2.
Authorization is unchanged and still enforced BEFORE any ZIP entry is written:
- add_file_entry_streamed writes the entry header (the filename) before it opens
the authorized stream, so the pre-filter is load-bearing — a denied file must
never reach it or its name leaks into the archive. AFTER a denied/missing id
is absent from the authorized map and is skipped in the same input order,
exactly as the old loop skipped a denied get_file_with_perms; it never reaches
the entry write. The authz moved from a per-file require to one batch check
EARLIER in the same function, not into or after the stream.
- The stream open keeps its own per-file Read check (now a primed-cache hit) +
Recents recording; check_files_read_batch is documented + gated as identical
to looping require.
Because the change is authorization-sensitive, the gate is the security property
itself. bench_round24_zip_authz drives the real PgAclEngine over a seeded,
interleaved mix of owned (granted drive) + denied (other drive) + missing ids
and asserts: the batch inclusion set AND input order are identical to the
per-file require loop; the included set is exactly the caller's owned files; no
denied or missing id is ever included (the authz-regression tripwire); and the
batch fetch returns exactly the owned files. Latency (cold, 600-item 1/3-owned
selection): 559 -> 267 ms (2.10x; the realistic all-owned selection is O(1) ->
a larger win). See benches/ROUND24.md.
The folder selections are left as-is (root counts are small and there is no
check_folders_read_batch primitive to batch through).
Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo
fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0
failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
Benchmark-gated, same rule as ROUND2-22: BEFORE/AFTER with a value-equivalence
gate and rollback-on-regression. Two harnesses — bench_round23_micro (no
Postgres; deterministic allocation gate) and bench_round23_queries (live
Postgres; p50 latency + strict equivalence gate against seeded fixtures). See
benches/ROUND23.md.
- J1: contact_pg_repository::row_to_contact (+ the inlined contact_group sibling)
decode the 3 JSONB columns via sqlx::types::Json<T> (one from_slice pass)
instead of row.get::<serde_json::Value> + from_value (a throwaway Value DOM
per column, walked a second time). Per contact row of every list / multiget /
CardDAV sync. Micro 84 -> 33 allocs/op (2.15x); PG 3794 -> 2360 ns/contact
(1.61x) on 500 real rows.
- J2: DrivePolicies::from_value deserializes straight from the borrow
(T::deserialize(&Value)) instead of from_value(value.clone()) — dropping the
full-DOM clone on every drive-policy read (move/copy, share, grant); one-line
body change, all 7 callers unchanged. Micro 5 -> 0 allocs/op (11.51x).
- P1: get_user_profile overlaps the two independent caller+target reads with
tokio::join! (self-case still a single fetch; caller-error precedence
preserved via caller_res? first) instead of two serial round-trips. PG
577 -> 312 us/call (1.85x).
- G1: subject_group remove_member computes the child's transitive-user recursive
CTE once and reuses it for both the would-empty pre-check and the cache
invalidation, instead of running the identical CTE twice (the edge delete is
above the child, so its descendants can't change). PG 829 -> 412 us/removal
(2.01x).
- U1: dedup_service (store_loose_chunks final registration + the ingest
run_rollback) reshapes the owned, dead-after Vec<(String,i64)> via
into_iter().unzip() instead of cloning every 64-byte hash for the
sync_blobs(&[String]) + UNNEST bind. Micro 256 -> 0 hash clones.
Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo
fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0
failed. The PG benches run against a local PostgreSQL 16 (schema applied from
migrations/); every equivalence gate passes.
The download_zip per-item N+1 (the audit's highest raw-latency candidate) is
deferred to a dedicated pass: its fix moves the sole authorization inside the
stream call, so it needs an AuthZ-ordering + anti-enumeration proof, not a perf
banner.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
Benchmark-gated, same rule as ROUND2-21: every change ships with a
BEFORE/AFTER counting-allocator benchmark and a byte/-value equivalence
gate; an AFTER that fails to reduce allocations exits non-zero (rollback).
See benches/ROUND22.md and examples/bench_round22_micro.rs. All arms
no-Postgres.
- H1: the hot GET handlers (get_thumbnail, download_file, list_files_query,
list_photos, NextCloud preview, public-share download/access) take
`req: Request` last and read `req.headers()` by borrow instead of axum's
HeaderMap extractor, whose FromRequestParts impl clones the whole request
header table just to read 1-3 headers (the ROUND14 §A4 middleware pattern,
finally propagated to the handlers). 2 -> 0 allocs/req · 9.95x wall.
- W1: native WebDAV write_etag_quoted — the etag emitter for every /webdav/
PROPFIND row (per file AND per folder, up to 500/page) — emits the quotes
as borrowed pre-escaped " text events instead of escaping a "{etag}"
String (the ROUND20 §C1 / ROUND21 §R4 pattern). 3 -> 0 allocs/row.
- C1: CalDAV getetag routed through a shared write_quoted_etag helper across
all 5 sites (3 per-event + 2 per-calendar); the now-dead etag: &mut String
buffer threaded through write_event_response/standard/requested props + the
two per-page buffers removed. 2 -> 0 allocs/row.
- D1: FileDto::from reuses the moved parts.blob_hash instead of cloning it
via the content_hash() getter (the ROUND19/20 move-not-clone sweep missed
it — hash/etag are read before into_parts()). Per file row of every
listing. 1 -> 0 allocs/row.
- E1: CalendarEvent::update_time_range/update_all_day stamp timed
DTSTART/DTEND via fmt::compact_ical_utc stack render (chrono fallback out
of range) instead of the %Y%m%dT%H%M%SZ strftime interpreter. 4 -> 0.
- S1: ShareItemType::try_from uses eq_ignore_ascii_case instead of a
throwaway to_lowercase() String. 1 -> 0 allocs/parse.
Verified: cargo clippy --features bench --all-targets -D warnings clean,
cargo fmt --all --check clean, cargo test --lib --features bench = 529
passed / 0 failed (incl. the OpenAPI-spec-validity test guarding the H1
utoipa-handler signature change).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
Round 21 of the benchmark-gated perf sweep. Six behaviour-preserving,
allocation-reducing changes, each with a BEFORE/AFTER counting-allocator
section in examples/bench_round21_micro.rs and a byte/-value equivalence
gate; all six pass their deterministic alloc gate (a non-winning AFTER
exits 1 = rollback).
- R1: pre-size the 16 CalDAV/CardDAV row-mapper Vecs (+1 HashMap) with
Vec::with_capacity(rows.len()) — the ROUND20 §I1 file-side pattern
extended to the calendar/contact repos it deferred. 7 → 1 allocs/op.
- R2: settle_batch binds a borrowed Vec<&str> instead of cloning every
chunk hash into a Vec<String> (sqlx encodes &[&str] as text[]
identically; favorites_pg_repository.rs:271 precedent). 33 → 1 allocs/op,
39x wall.
- R3: store_loose_chunks keys its intra-request dedup set on the raw
[u8;32] BLAKE3 digest and moves the hex on a duplicate (the ROUND17 §D2
pattern applied to the delta-upload sibling). 401 → 209 allocs/op.
- R4: CardDAV getetag emits borrowed pre-escaped " quotes via a shared
write_quoted_etag helper (ROUND20 §C1 pattern, all 4 CardDAV etag sites).
3 → 0 allocs/op.
- R5: BDAY stamped via the new fmt::compact_date stack renderer instead of
chrono's strftime interpreter (chrono fallback out of the 4-digit-year
range; byte-identical, unit-tested vs chrono). 2 → 0 allocs/op, 10.5x wall.
- R6: NC trashbin folder content-type via Cow::Borrowed instead of
.to_string() on the constant (ROUND16 §M1 pattern). 1 → 0 allocs/op.
See benches/ROUND21.md for the full write-up and the deferred-items list
(HeaderMap-clone hot handlers, Query→typed-struct, WebDAV dead-props
HashSet, and others surfaced by the audit that want their own validated
pass). Validated: cargo fmt, cargo clippy --features bench --all-targets
-D warnings, cargo test --lib --features test_utils (529 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015gHVq5Wy2TzdWeSqtEmK6m
Benchmark-gated (benches/ROUND20.md), same rule as rounds 2-19: every change
ships with a BEFORE/AFTER counting-allocator micro-benchmark and a byte-value
equivalence gate; a non-winning AFTER is rolled back (never applied). The
rollback rule is encoded in the harness (GATE FAIL exit). All 8 sections pass.
Reproduce: cargo run --release --features bench --example bench_round20_micro
- A1 CalendarEvent iCal parse: replace the throwaway per-property
HashMap<String,Vec<String>> (DTSTART/DTEND/RECURRENCE-ID) with a direct
VALUE=DATE scan; prop_with_params kept #[cfg(test)] (6->2 allocs/event, 4.2x)
- A2 UserDto::from: add User::into_parts and MOVE image (<=512 KiB data URI)
+ ui_preferences JSON instead of cloning on every /api/auth/me (27->14 allocs)
- A3 parse_vcard: drop the per-line to_ascii_uppercase copy + the lines Vec;
promote ascii_ci_contains to common::text and share it (8->1 allocs/contact)
- A4 Calendar/AddressBook DTO: into_parts move incl. custom_properties map (18->10)
- I1 file-listing repos: collect::<Result<Vec>>() size-hints to 0 and grows from
capacity 0; pre-size with Vec::with_capacity (8->1 container reallocs, 4 sites)
- I4 plaintext_stream: lazy emit iterator instead of eager Vec collect (43x wall)
- C1 NC write_etag_element: borrowed pre-escaped quote events, no owned quoted
String/escape re-alloc; byte-identical output (3->0 allocs/PROPFIND row)
- C3 NC favorites REPORT: map.remove() move instead of get().clone() (~7 allocs/fav)
Deferred (documented in ROUND20.md): NC oc:id/trashbin buffer reuse, I1 sibling
CardDAV/CalDAV listing paths, Contact JSONB Json<Vec<_>> decode, dedup
settle_batch &str bind, and a fast DoS-resistant hasher for hot trusted-key maps
(needs a dependency decision).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JsJjcVX9RoN96DMa35Wqzd
Benchmark-gated (examples/bench_round19_micro.rs, benches/ROUND19.md): every
section ships a BEFORE/AFTER counting-allocator arm with a byte/-value
equivalence gate and a GATE-FAIL-rollback exit. All eight pass. No Postgres.
- M1 verify_basic_auth cache key: blake3::hash(format!("{u}:{p}")) → incremental
Hasher (byte-identical key, 2→0 allocs on every Basic-auth DAV request)
- M2 WopiTokenService: prebuild Validation/DecodingKey/EncodingKey in new()
instead of per-call (mirrors JwtTokenService; 16→12 allocs/validate)
- V1/V2 vCard emit (contact_to_vcard/generate_vcard): FN fallback drops the
throwaway to_string, NOTE skips the escape copy for newline-free notes, REV
uses new common::fmt::compact_ical_utc stack renderer (11.5× vs chrono
strftime, 3→0 allocs); per-contact 9→4 allocs
- M4 trash_service::row_to_item_dto: move name/path/blob_hash out of the owned
row instead of cloning (3 clones/file row gone)
- M5 search cache key: Uuid::hyphenated().encode_lower stack buffer instead of
to_string (identical u64 key, 1→0 allocs/request)
- M6 streaming PROPFIND: reuse one href buffer across the page instead of a
format! per child (native + NC handlers; 192→3 allocs on a 64-child page)
- M7 nextcloud extract_url_user: return Cow instead of forcing into_owned
(zero-alloc on the common ASCII-username path)
common::fmt::compact_ical_utc added with chrono-parity unit tests (CASES +
60-year sweep). cargo fmt + clippy --all-targets clean; 526 lib unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ront9bk7YMoffVQkGG47gh
Two items from the ROUND17 deferred list, each benchmark-gated with a
BEFORE/AFTER equivalence gate and a rollback-on-regression check (ROUND2–17
discipline). See benches/ROUND18.md.
[C1] backend — CalendarEvent::update_ical_property / remove_ical_property
rewrote the ENTIRE ical_data body with format!("{}{}{}") on every call and
allocated two search needles per call. calendar_storage_adapter::update_event
fans a multi-field edit out into one call per changed field, so a full REST
edit paid one full-body (up to ~11 KB) allocation per property. The body is
now mutated in place (replace_range for an existing property, four insert/
insert_str for a new one, byte-identical spans) and the single "\nNAME:"
needle is built on the stack (the "\r\nNAME:" needle was redundant — the LF
form is its suffix). bench_round18_micro [C1]: 70 -> 2 allocs/op (68 fewer),
2.46x wall, emitted body byte-identical.
[F1] frontend — ResourceList itemIndexById rebuilt a fresh Map over the whole
accumulated list every infinite-scroll page (O(N)/page, O(N^2) drain) and,
being a new instance each page, re-fired the reap-stale effect (another O(N)
id Set/page). New ItemIndexBuilder extends a persistent Map with the fresh
page only and reuses the reference across appends; the reap-stale effect now
tests membership against it. round18.bench.test.ts [F1]: 40x50 drain 74.1 ->
6.4 ms (11.5x), deep-equal to the reference at every page, reference-contract
gate (same-ref append / new-ref rebuild).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoxFtikahM1N4PE5s3ZVH3
Targets the content-addressable dedup write path (the ROUND15-deferred
"dedup_service hash-String re-allocations") from both ends — the streaming
ingest loop and the delta-commit verification read — plus a CardDAV vCard
micro-cut. Every change is benchmark-gated with a hard rollback rule; no
PostgreSQL needed for any arm (benches/ROUND17.md).
Backend (counting-allocator, examples/bench_round17_micro.rs):
- D2 chunk-ingest (store_from_stream, the hottest write path — every chunk of
every upload): the 64-char hex hash String was allocated 3x per chunk
(to_hex + chunk_hashes clone + session_seen insert-clone, the last dropped on
a duplicate). The intra-upload dedup set now keys on the raw 32-byte BLAKE3
digest ([u8;32], Copy, no heap) and the manifest push is branch-split so a
duplicate moves the hex in: 3 -> 2 allocs/new chunk, 3 -> 1/duplicate.
Measured 214 -> 149 allocs/op (1.14x wall) on a 64-chunk 1-in-2-dup batch;
smaller/faster set too (32B inline keys vs 64B heap Strings).
- D1 hash_chunk_sequence (delta-commit verification): took chunks by
&[(String,u64)] and fed the backend stream with iter().cloned(), re-cloning
every chunk hash a second time on top of the owned Vec the caller already
built. Take the Vec by value + into_iter(): 65 -> 0 internal allocs/op, ~2.5us
of clone work removed per verify.
- V1 vCard TYPE tokens (contact_to_vcard + generate_vcard, 5 sites): each
EMAIL/TEL/ADR TYPE= param used ty.to_uppercase() — a throw-away String per
token per contact. New shared fmt::push_upper writes the upper-cased chars
straight into the buffer (byte-identical to str::to_uppercase, unit-tested):
13 -> 5 allocs/op, 1.19x wall.
Gates: each section asserts byte/-value equivalence (D2 the ordered manifest +
sizes + write-set; D1 the removed clone is a pure copy; V1 the full vCard) and
exits non-zero if an AFTER arm fails to reduce allocations. push_upper is
unit-tested byte-equal to str::to_uppercase (fmt::tests). Verified end-to-end:
cargo fmt clean, clippy --release --all-targets --features bench -D warnings
clean, and the harness prints GATE PASS against the built release lib.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XMmt7vNETYUbEG3Hc17LDx
Benchmark-gated, same rule as rounds 2–14: every change ships with a
BEFORE/AFTER benchmark and an equivalence/safety gate; an AFTER that doesn't
beat its BEFORE is rolled back. The rule is encoded per harness (GATE FAIL
non-zero exit in the Rust examples, threshold expect() in vitest).
F1 — Grouped listings (trash / recent / favorites / shared-with-me)
re-bucketed the WHOLE accumulated list on every infinite-scroll page.
ResourceSectionsBuilder (new, off the reactive graph) re-buckets only the
fresh page and hands VirtualList the same rows array reference for untouched
buckets. 50×50 (2 500-item) drain: 63 750 → 2 500 bucketOf calls (25.5×),
12.5 → 1.3 ms wall (9.9×); O(N²/page) → O(N). Deep-equal to the full-rebuild
reference at every page for both a contiguous (date) and a non-contiguous
(trash-by-drive) group-by; reference-stability + fallback gated.
B1 — exif Make/Model: the display String was thrown away to allocate the
trimmed copy; display_value_trimmed trims in place (drain + truncate), 2 → 1
alloc per field (8 → 4 allocs/op, 1.26×).
B2 — content-index worker: text_extractor::supports (lowercases MIME +
extension) was called twice per file per drain batch; classify once into a
Vec<bool> and thread it through both uses. 256-file batch: 704 → 353 allocs,
34.5 → 16.7 µs (2.07×).
B3 — tantivy: skip SnippetGenerator::create on a zero-hit content search
(return Ok(vec![]) once top_docs.is_empty()); the per-hit loop was empty.
400-doc index: 1 575.6 → 1 237.2 ns (1.27×), widens with index size.
Harnesses: examples/bench_round15_micro.rs, examples/bench_round15_tantivy.rs,
frontend resourceSections.bench.test.ts; writeup in benches/ROUND15.md. Also
normalizes two round14 bench examples that were committed unformatted
(cargo fmt --all).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012o47jSrtL7xuNGTHXmtiYL
Benchmark-gated (benches/ROUND14.md); every change ships a BEFORE/AFTER
benchmark with an equivalence gate and is rolled back on regression (the
rule is encoded as a GATE FAIL exit / threshold expect).
Backend
- Q1 faces_for_file → narrow face_boxes_for_file(id, person_id, bbox) with the
caller filter pushed into SQL: drops the 2 KiB embedding BYTEA + 6 unused
columns per face. 15-face lightbox open 0.312→0.219 ms, 32 KB→840 B/req.
- A1 cookie auth uses the borrow-only extract_cookie_str (already backs CSRF)
instead of extract_cookie_value's owned String: -1 alloc/cookie request.
- A2 compute_relevance ASCII case-fold fast path vs name.to_lowercase() per
result row (Unicode fallback preserved): 1.40x, 12→3 allocs/page.
- A3 sub pre-parsed to Uuid at decode time (TokenClaims.sub_id) vs re-parsing
the 36-char claim on every request incl. cache hits: 22.7→0.7 ns.
- A4 auth + NextCloud middlewares borrow request.headers() instead of taking
axum's HeaderMap extractor (a full map clone): 2→0 allocs/authed request.
- A5 CalDAV getlastmodified via the stack rfc2822_utc (byte-identical to
chrono) vs a per-event to_rfc2822() heap String: 5→0 allocs.
- A6 CalDAV per-event href + quoted etag written into reused page buffers vs a
fresh format! pair per event: 3.48x, 240→6 allocs/40-event page.
Frontend
- F1 t() shares one frozen EMPTY_PARAMS for the no-interpolation call forms vs
a throwaway {} per call: -1 alloc/call.
- F2 favorites favoriteIds is a persistent SvelteSet with per-page add (clear
on reset) vs a brand-new set over the whole accumulated list each page:
22.3x over a 40-page drain (O(N^2)→O(N)).
Verified: cargo check --all-targets, cargo clippy -D warnings, both bench
packs (GATE PASS), frontend npm run check + vitest (4/4). ROUND14.md also
records the investigated-but-deferred backlog (music N+1, contact vcard
over-fetch, CachedBlobBackend syscalls, ResourceList.sections builder, etc.).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PymgCdK78NzUF3oRAQCJfN
The app-level ETag re-check before the write still left a gap between the check and the actual UPDATE for a concurrent writer to land in.
Push the check into the write path itself: swap_blob_hash now takes an expected_hash and only applies the SET under the same FOR UPDATE row lock it already held, closing the race instead of just narrowing it. Adds ErrorKind::PreconditionFailed (412) for the CAS-miss path; PUT/WOPI/chunked-upload keep blind-overwrite semantics by passing None
webdav_patch.hurl and nc_webdav_patch.hurl existed with real coverage
since the original PATCH commits but were never added to
tests/api/run.sh, so just api-test/CI silently skipped them. Wire both
in, fix nc_webdav_patch.hurl's header-after-[BasicAuth] ordering bug
that meant it had never actually passed, and add two new
consistency-focused files chaining PATCH operations with
cross-protocol/cross-surface verification:
- webdav_patch_consistency.hurl: chained overwrites with ETag-change
checks, GET/HEAD/PROPFIND cross-protocol agreement, quota-507
leaving the file byte-for-byte unchanged, direct_put_max_bytes
prefix/suffix regression coverage.
- nc_webdav_patch_consistency.hurl: Editor/Viewer/Outsider permission
matrix, cross-surface lock interop, quota-507 via the NC surface.
Running these surfaced two real bugs in the NC PATCH handler, both
fixed here:
- The write step mapped every error (including a legitimate anti-enum
permission denial) to a raw 500 instead of AppError::from(e), unlike
the plain surface. A Viewer without Update permission got a 500
leak instead of the expected 404.
- nc_to_internal_path() didn't strip the leading '/' that chroot.path
carries from StoragePath::to_string(), so a LOCK taken via /webdav/
silently failed to block PATCH via /remote.php/dav/ on the same
file — the exact-string lock-store lookup never matched. Added a
regression unit test.
Fixes to the RFC 5789 PATCH implementation found by review of the
rfc-5789-http-patch branch:
- nextcloud/webdav_handler.rs::handle_patch now enforces storage quota
before committing, matching the plain WebDAV surface (was a quota
bypass via the NextCloud endpoint).
- The plain surface's If-Match/If-None-Match comparison reused a
hand-rolled single-value strong compare that mishandled weak (W/)
validators and multi-value lists. Moved the correct RFC 7232 helpers
(already used by the NC surface) into the shared handler file so both
surfaces use one conformant implementation.
- NC handle_patch resolved the target file via get_file_by_path, which
performs no authorization check, before any permission-gated call —
for a full-file-range patch this could leak size/ETag via 412/416
responses to a caller without Read on that file. Added the same
explicit authz.require(Read, ...) the plain surface already has.
- ingest_range_patch_to_cas capped the whole spliced stream (prefix +
edit + suffix) against direct_put_max_bytes, so PATCH became
permanently unusable on any file at or above that size regardless of
edit size. The cap now only bounds the edit itself.
- NC handle_patch had no active-lock guard, so a LOCK taken via
/webdav/ didn't protect the same file reached through
/remote.php/dav/. Now shares enforce_native_lock with the plain
surface.
- Added a re-check of the file's ETag immediately before the write on
both surfaces, narrowing (not eliminating — that would need
compare-and-swap support in the write path) the window in which two
concurrent PATCHes to disjoint ranges could silently clobber each
other.
- NC handle_patch returned 404 for a PATCH on a directory instead of
409 like the plain surface; now checks folder existence first.
- The Content-Length-vs-X-Update-Range span check only fired when
Content-Length was present, so a chunked-transfer body could silently
diverge from the declared span. ingest_range_patch_to_cas now counts
actual body bytes and validates against the declared span
regardless, discarding the ingested blob on mismatch.
The NextCloud-compatible WebDAV surface (/remote.php/dav/…) had no PATCH
dispatch arm at all — requests fell through to 405 — unlike the plain-file
surface (see the sibling commit on this repo's rfc-5789-http-patch work).
Adds handle_patch to nextcloud/webdav_handler.rs, reusing the plain
surface's X-Update-Range mechanism directly instead of duplicating it:
- api/handlers/webdav_handler.rs::parse_update_range is now pub(crate)
so both surfaces share the same header-parsing/validation logic.
- upload_ingest::ingest_range_patch_to_cas (already surface-agnostic)
splices the request body between the file's untouched prefix/suffix
byte ranges and re-ingests through the same content-addressable
pipeline handle_put uses.
Follows this file's own handle_put conventions rather than the plain
handler's: no active-lock guard (the NC surface has no LOCK/UNLOCK
dispatch arm at all) and no explicit storage-quota check (handle_put
doesn't do one either on this surface) — matching the sibling handler
instead of importing behavior the NC surface doesn't otherwise have.
Adds PATCH to the OPTIONS Allow header.
Also fixes a pre-existing clippy::useless_borrows_in_formatting warning
in thumbnail_service.rs (unrelated to this change, but blocking a clean
clippy run on this branch).
Adds tests/api/nc_webdav_patch.hurl covering explicit-range and append
PATCH, the Content-Range rejection, the missing-header 400, and PATCH
on a nonexistent file.
RFC 4918 §9.7.1 forbids partial updates on PUT; this adds PATCH as the
supported mechanism instead, via an X-Update-Range header (bytes=<start>-<end>
or append). Reuses the existing CAS/dedup pipeline by splicing the request
body between the file's untouched prefix/suffix byte ranges and re-ingesting
as one continuous stream, so unedited chunks dedup for free.
Benchmark-gated (BEFORE/AFTER + equivalence/safety gate per change), same
discipline as rounds 2-12. Full write-up in benches/ROUND13.md.
Shipped:
- V1 Grouped views windowed (files route + ResourceList). The grid arm was
the last unwindowed path (trash is grouped-by-default in grid): each
swimlane now feeds its own VirtualList, outer container a flex stack.
vitest gate: 800-item grouped grid mounts <120 .file-item (was 800).
- Q1 get_users_by_ids drops the <=512 KiB avatar image + ui_preferences
JSONB (notification path never reads them). 30-member fan-out 8.60 ->
0.25 ms (34.3x), ~7.7 MB off the wire.
- Q2 Login provisioning is_empty() -> SELECT EXISTS for calendar + address
book (every login). 0.193 -> 0.170 ms, widens with owned-row count.
- Q3 Recent-access prunes only when the upsert inserted (RETURNING xmax=0)
— a re-access can't grow the set. 0.567 -> 0.324 ms (1.75x).
- L1 Locale supported-codes precomputed once vs rebuilt per anonymous
request. 616 -> 17.3 ns (35.7x), 18 -> 1 allocs.
- H1 Duplicate /api TraceLayer removed (global stack already wraps it).
1.86 -> 1.42 us/request, -6 allocs.
- H2 client_ip span field: borrow-only ClientIpDisplay vs owned String.
187 -> 173 ns, -1 alloc.
Not shipped (discipline): the "media hooks read the blob 3x" lead was a
correctness bug, not a perf dup — the raw-path metadata/faces readers
resolve only for local+unencrypted+single-chunk blobs and silently produce
nothing otherwise. Flagged for maintainers; routing through read_blob_bytes
is a correctness fix (perf-neutral-to-negative), not a benchmark-gated
perf change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
Benchmark-gated round (benches/ROUND12.md; every change ships with a
BEFORE/AFTER harness + equivalence gates, one candidate rejected by its
own bench):
DB / query shapes (bench_round12_queries):
- NC sharee search: username-only projection instead of the 21-column row
(incl. the <=512 KiB avatar) per match, + gin_trgm_ops indexes on
auth.users for the leading-wildcard ILIKE (4.98x; 54.7x with index).
- Password login: delete the redundant full-row update_user — create_session
already stamps last_login_at in its own txn (4.45x per login).
- Email-verified stamp: narrow conditional UPDATE (8.9x); OIDC repeat login
now compares profile state in memory and issues ZERO queries when nothing
changed (was: full 17-column rewrite per login).
- Refresh rotation: revoke+insert+stamp fused into one transaction via new
rotate_session port method (1.18x).
- WOPI CheckFileInfo / authorize_wopi_access: require(Read) + get_file +
check(Update) overlapped with tokio::join!, original result precedence
(cold 1.34x).
- Upload quota gate: user-envelope + drive-cap checks fused into ONE
round-trip (check_upload_quotas) — the NC chunked PUT pays this per
chunk (1.81x, 2 -> 1 queries/chunk); shared verdict evaluators keep
error shapes byte-identical.
CPU / allocs (bench_round12_micro):
- sized_json: pre-sized listing serialization replacing axum Json's 128 B
seed + doubling-realloc chain on files/folder-resources/photos/search
responses (1.40x, 13 -> 2 allocs per 500-row page; byte-identical).
- Security headers: 4 SetResponseHeaderLayer folded into the CSP middleware
pass (5 layers -> 1; 1.43x per request, -26 allocs; header set gated
byte-identical incl. 304s).
- Media capture-metadata: single-read extraction — nom-exif now parses the
buffer kamadak already read (zero-copy Bytes) and videos open once with a
kind() dispatch; per-image opens 2-3 -> 1 (1.44x warm geomean, 1.6-3.2x
cold cache; extraction outputs gated identical incl. the MIME-mislabel
track fallback).
- Chunked-upload session ops: owner gate folded into the operation's own
DashMap lookup + stack-encoded uuid compare (5 -> 3 lookups, -2 allocs,
1.28x per chunk).
Blob cache (bench_blob_cache_index + round-3 regression guard):
- CachedBlobBackend index: tokio::sync::Mutex<LruCache> -> moka::sync::Cache
with byte weigher. The mutex serialized every cached chunk read and scaled
NEGATIVELY (2.08 -> 1.07 Mops/s from 1 -> 2 readers); moka probes are
lock-free (2.17x at K=2). Byte budget now enforced by moka (manual
current_size + collect_evictions machinery deleted); eviction listener
unlinks size-evicted files only (Replaced entries keep their file —
gated). Single-flight miss gate unchanged (16 concurrent misses -> 1
fetch re-verified via the round-3 harness).
- put_blob now populates the cache BEFORE the inner backend consumes the
source file (the old order failed 100% of the time — local renames,
S3/Azure delete the source — so the first read after a whole-file put
re-downloaded from the remote); inner-put failure invalidates the entry.
Frontend (vitest gates):
- List-view thumbnails request the 150px icon rendition instead of 400px
preview into a 40px slot (~7.1x fewer pixels, ~4-5x fewer bytes per
thumbnail across list views); grid keeps preview.
Rejected by its own bench (kept as evidence in bench_round12_micro §2):
- Single-pass compression predicate: the monomorphized And-chain already
costs ~4.6 ns / 0 allocs total; the fused node measured within noise.
New migration: 20260719000000_users_search_trgm.sql (trgm indexes).
Deferred with prepared design: grouped file/grid view virtualization
(single-VirtualRows flatten, the photos pattern) — next round's headline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
Edouard Vanbelle