add /api/folders/{id}/ancestors
this API to iterate parent up to the drive root or the shared folder
this will help UI to build the breadcrumb in 1 API call
and to identify the root element (is it a drive users has access to or
a shared folder ?)
ui: now only 1 API call is now required to build the breadcrumb
- provide is_shared and is_favorite information in DTO, information propagated as badge/buttons per items
regarding performances I try to be minimalis on SQL to prevent any perf regression
- remove old set of sharedids and favoriteids (was not functionnal anymore)
- fix date picker (no past selection) in grant
- fix contextmenu close on /files section
- rustfmt the integration-test FolderService::new callsites added during
the upstream merge (long single-line args wrapped).
- Cargo.lock: rebuild from upstream/main's lock so only the
testcontainers-modules dev-dep subtree and the memmap2 0.9.10→0.9.11
security bump differ (avoids churning upstream's pins).
- .cargo/audit.toml: ignore the four astral-tokio-tar tar-extraction
advisories — dev-only, transitive via testcontainers-modules
(integration-test harness), never in the production binary.
Resolve conflicts between the external-file-mounts feature and upstream's
D5/D7 refactor (per-file provenance, keyset pagination, cross-drive move
gates, resource-access hook, folder-cascade lifecycle hook).
Key resolutions:
- FolderService::new now takes (repo, authz, file_lifecycle, mount_router);
all callers + DI updated.
- FileRetrievalService / FileManagementService keep both the mount_router
and the new resource_access_hook / drive_repo / storage_usage wiring.
- list_files_batch_with_perms: adapt the mount branch from offset- to
keyset (after_name) pagination, mirroring paginate_mount_entries.
- download_file_impl: keep upstream's &HeaderMap + `impl IntoResponse + use<>`
signature, retain the mount-download branch.
- Mount DTOs: the retired `owner_id` field maps onto created_by/updated_by
(the mount owner) — the fields the frontend now uses for owner display.
- admin/+page.svelte: keep upstream's user-delete modal + the 'mounts' tab.
- Bump memmap2 0.9.10 -> 0.9.11 (RUSTSEC critical advisory fix) and
regenerate Cargo.lock against the merged Cargo.toml.
Seven behaviour-preserving allocation / copy / bandwidth cuts, each behind a
counting-allocator BEFORE/AFTER gate that exit(1)s unless AFTER allocates
strictly fewer than BEFORE (benches/ROUND29.md, examples/bench_round29_micro.rs).
- [B] Content-cache serve fast path (optimized_inner Tier 1 +
get_file_range_preloaded — the video-scrub hot path): probe the cache with a
borrow first and build the owned get_or_load args (quoted-etag / key / id
Strings) only on a miss, instead of allocating them before every probe and
discarding them on a hit. 6 -> 0 allocs per cache hit. Splits get_or_load into
get + load_and_cache so the miss path is not re-probed and the hit/miss stat
counters stay byte-identical. Also drops the unconditional content_hash/name
clones that ran for the >=10 MB streaming tier that used neither.
- [A] NextCloud REPORT emit loops: per-row href String (and format! per folder
row) -> one reused href_buf via nc_href_into / nc_collection_href_into with the
URL-encoded user computed once per page. 1497 fewer allocs on a 500-row page.
- [C] read_full: a single-frame blob is returned zero-copy instead of a second
whole-payload memcpy into a fresh BytesMut; multi-frame path unchanged.
- [D] login-lockout key: to_lowercase()+format! -> one pre-sized ASCII buffer
(non-ASCII keeps str::to_lowercase). 3 -> 1 alloc/req, byte-identical key.
- [E] NC composite-username parse: owned clone/to_string -> &str borrow of the
already-owned raw_username. 1 -> 0 alloc on the common no-marker path.
- [F] get_contacts_in_group: stop SELECTing the discarded multi-KB vcard column
(the live method ROUND25 §Q2 missed; ContactDto has no vcard field).
- [G] count_admin_users: add count_users_by_role -> scalar COUNT(*) instead of
hydrating every admin's full row (incl. up-to-512 KiB avatar + ui_preferences
JSONB) only to .len() it, on a bootstrap-polled status endpoint.
All seven gates pass; cargo fmt --check and cargo clippy --all-features
--all-targets -D warnings clean. §F/§G additionally validated against a live
PostgreSQL 16 with the full migration set (query validity, result equivalence,
600000 -> 8 byte wire delta on the admin count).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LhpDZxSQTAGnAqCHUdtG5N
Adds application/adapters/param_encoding.rs: encode/decode for the three
RFC 6868 caret-escape sequences (^n newline, ^^ literal caret, ^' literal
DQUOTE) plus render_param_value, which quotes and encodes a parameter
value only when the vCard/iCalendar param-value grammar requires it
(comma/semicolon/colon triggers quoting; an embedded DQUOTE/caret/newline
needs escaping even once quoted).
Wires render_param_value into the two vCard generators' TYPE parameter
emission (EMAIL/TEL/ADR) in carddav_adapter.rs::contact_to_vcard and
contact_service.rs::generate_vcard — contact `type` fields are free text,
not a fixed enum, so a value containing a comma or quote previously
produced outright broken vCard syntax (no quoting/escaping was applied
at all).
Scope note: this branch only fixes the generation side. The existing
vCard parser is a naive line-by-line substring scanner (`line.contains
("TYPE=WORK")`) with no real per-parameter tokenizing, so there's no
integration point yet for the decode half without a larger parser
rewrite — that's part of the vCard 4.0 work (RFC 6350), which depends on
this branch for the shared encoding utility.
report_handler's four REPORT emit loops (the filter-files favorites REPORT and the
search REPORT, each a file loop + a folder loop) shared the same per-row oc:id
String that ROUND27 §H1 replaced in the two PROPFIND page loops. Apply the
identical, already-validated transformation: hoist one oc_buf per handler (reused
across both its loops) and compute the id into it via format_oc_id_into instead of
a fresh format_oc_id String per child. 1 String/row -> 0 (amortized). The
write_{file,folder}_response fns already take Option<&str>, so their signatures are
unchanged and the emitted oc:id bytes are byte-identical.
Same transformation benchmarked in ROUND27 §H1 (bench_round27_micro: 998 -> 0
per-row allocs, 2.16x wall), so no new bench. Verified: cargo fmt clean, cargo
clippy --features bench -D warnings clean, cargo test --lib --features bench =
529 passed / 0 failed across 5 consecutive runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Two behaviour-preserving allocation cuts (benches/ROUND27.md), each with a
counting-allocator BEFORE/AFTER gate that rolls back if AFTER does not allocate
fewer than BEFORE:
- H1 NextCloud PROPFIND: the streaming page loops built oc:id as a fresh String
per child (format_oc_id -> format!("{:08}{}", id, instance)). Add
format_oc_id_into(&mut buf, id, svc) and compute into one oc_buf reused across
the page (next to the existing href buffer) — 1 String/row -> 0. 998->0
per-row allocs on a 500-row page, 2.16x wall. The write fns still take
Option<&str>, so no signature change; oc:id bytes identical. Scoped to the two
PROPFIND page loops (the hot directory-listing path); REPORT/trashbin deferred.
- P2 contact create/update: bind sqlx::types::Json(&dtos) (Encode runs to_writer
straight into the JSONB buffer) instead of serde_json::to_value(&dtos) + bind,
which built a throwaway Value DOM per JSONB column. Write-side twin of ROUND23
J1. 21->2 allocs, 4.68x wall for a 3-entry column. Behaviour-preserving:
to_value sorts keys and direct serialize keeps struct order, but Postgres
normalizes JSONB key order so the stored value is identical (verified via psql:
'{...alpha...}'::jsonb = '{...struct...}'::jsonb -> t), and reads decode by
field name; the etag comes from the domain entity, not the stored JSONB.
Adds bench_round27_micro. Verified: cargo fmt clean, cargo clippy --features
bench -D warnings clean (real exit), cargo test --lib --features bench = 529
passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Three benchmark-gated optimizations from the ROUND25 backlog (benches/ROUND26.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- P1 drive_pg_repository policy reads: decode d.policies through
sqlx::types::Json<DrivePolicies> (one from_slice over the raw JSONB bytes)
instead of a throwaway serde_json::Value DOM + DrivePolicies::from_value —
6 -> 0 allocs/read, 2.77x wall. A shared policies_from_row helper preserves
the lenient unwrap_or_default fallback (malformed bag -> all-false).
- D1 CachedBlobBackend: pre-create the 256 {00..ff} shard dirs at initialize()
(mirroring LocalBlobBackend, reusing HEX_PREFIXES) and drop the redundant
per-write create_dir_all on already-existing shards — ~45us + a blocking-pool
dispatch removed per cache write on cached-remote deployments.
- G1 delta-upload have/need hash sets (distinct_hashes, authorize_chunk_download):
SipHash -> foldhash::quality::RandomState — a fast hasher that stays
DoS-resistant via a per-instance random seed, the required property for the
attacker-controlled 64-hex client hashes — 2.37x wall on a 40k-hash
negotiation. foldhash was already in the lockfile transitively (hashbrown).
Tested and REVERTED (kept as-is): moving the moka eviction unlink off the reactor
via spawn_blocking. The benchmark refuted it — on the local cache dir the
spawn_blocking dispatch (~20us) costs more than the inline unlink (~7us) it would
replace. See ROUND26.md §D2.
Adds bench_round26_{micro,diskio,hasher} (counting allocator / async wall / wall).
Verified: cargo fmt clean, cargo clippy --features bench -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Five benchmark-gated optimizations from a fresh six-way audit (benches/ROUND25.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- M1 EncryptedBlobBackend::decrypt_bytes: replace split_off (a fresh Vec + full
ciphertext memcpy on every decrypted chunk, contradicting its own "in place"
doc) with in-place detached decrypt + a zero-copy Bytes::slice past the nonce.
Peak RAM per read drops from ~2x to ~1x the payload (-262KB/op at 256KiB;
scales with blob size). Plaintext byte-identical; tamper/wrong-key tests pass.
- M2 delta commit: move-unzip the owned chunk list instead of a third
per-occurrence hash clone (-4000 allocs on a 4000-chunk commit).
- M3 folder ZIP download: drop the dead Query<HashMap> extractor it never read
(byte-identical response; 5->0 allocs/request).
- Q1 public-playlist listing: fold the per-playlist COUNT(*) N+1 into one
LEFT JOIN ... GROUP BY via a new inherent repo method (101 -> 1 round-trips,
36x wall on a 100-playlist page).
- Q2 contact REST listings (paginated/search/by-group): stop over-fetching the
multi-KB vcard TEXT the ContactDto discards, via a shared lite row mapper and
narrowed SELECTs (6.4x wall on 1000 contacts with 8KiB vcards). The whole-book
vCard export and CardDAV sync paths keep the column.
Adds bench_round25_micro (counting allocator tracking count+bytes) and
bench_round25_queries (live Postgres), both with equivalence gates and a
rollback exit(1). Verified: cargo fmt clean, cargo clippy -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
The ROUND23-deferred download_zip N+1, given its own validated pass. The
individually-selected files were authorized + fetched one at a time via
get_file_with_perms (require + get = 2 serial round-trips/file) before any
streaming — a 200-file selection was 400 serial round-trips. AFTER routes the
whole multi-select through the new
FileRetrievalService::get_files_by_ids_with_perms: one check_files_read_batch
(the PgAclEngine resolves every file's drive in ONE query and primes the
resource->drive cache) + one get_files_by_ids. 2N round-trips -> 2.
Authorization is unchanged and still enforced BEFORE any ZIP entry is written:
- add_file_entry_streamed writes the entry header (the filename) before it opens
the authorized stream, so the pre-filter is load-bearing — a denied file must
never reach it or its name leaks into the archive. AFTER a denied/missing id
is absent from the authorized map and is skipped in the same input order,
exactly as the old loop skipped a denied get_file_with_perms; it never reaches
the entry write. The authz moved from a per-file require to one batch check
EARLIER in the same function, not into or after the stream.
- The stream open keeps its own per-file Read check (now a primed-cache hit) +
Recents recording; check_files_read_batch is documented + gated as identical
to looping require.
Because the change is authorization-sensitive, the gate is the security property
itself. bench_round24_zip_authz drives the real PgAclEngine over a seeded,
interleaved mix of owned (granted drive) + denied (other drive) + missing ids
and asserts: the batch inclusion set AND input order are identical to the
per-file require loop; the included set is exactly the caller's owned files; no
denied or missing id is ever included (the authz-regression tripwire); and the
batch fetch returns exactly the owned files. Latency (cold, 600-item 1/3-owned
selection): 559 -> 267 ms (2.10x; the realistic all-owned selection is O(1) ->
a larger win). See benches/ROUND24.md.
The folder selections are left as-is (root counts are small and there is no
check_folders_read_batch primitive to batch through).
Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo
fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0
failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
Benchmark-gated, same rule as ROUND2-22: BEFORE/AFTER with a value-equivalence
gate and rollback-on-regression. Two harnesses — bench_round23_micro (no
Postgres; deterministic allocation gate) and bench_round23_queries (live
Postgres; p50 latency + strict equivalence gate against seeded fixtures). See
benches/ROUND23.md.
- J1: contact_pg_repository::row_to_contact (+ the inlined contact_group sibling)
decode the 3 JSONB columns via sqlx::types::Json<T> (one from_slice pass)
instead of row.get::<serde_json::Value> + from_value (a throwaway Value DOM
per column, walked a second time). Per contact row of every list / multiget /
CardDAV sync. Micro 84 -> 33 allocs/op (2.15x); PG 3794 -> 2360 ns/contact
(1.61x) on 500 real rows.
- J2: DrivePolicies::from_value deserializes straight from the borrow
(T::deserialize(&Value)) instead of from_value(value.clone()) — dropping the
full-DOM clone on every drive-policy read (move/copy, share, grant); one-line
body change, all 7 callers unchanged. Micro 5 -> 0 allocs/op (11.51x).
- P1: get_user_profile overlaps the two independent caller+target reads with
tokio::join! (self-case still a single fetch; caller-error precedence
preserved via caller_res? first) instead of two serial round-trips. PG
577 -> 312 us/call (1.85x).
- G1: subject_group remove_member computes the child's transitive-user recursive
CTE once and reuses it for both the would-empty pre-check and the cache
invalidation, instead of running the identical CTE twice (the edge delete is
above the child, so its descendants can't change). PG 829 -> 412 us/removal
(2.01x).
- U1: dedup_service (store_loose_chunks final registration + the ingest
run_rollback) reshapes the owned, dead-after Vec<(String,i64)> via
into_iter().unzip() instead of cloning every 64-byte hash for the
sync_blobs(&[String]) + UNNEST bind. Micro 256 -> 0 hash clones.
Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo
fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0
failed. The PG benches run against a local PostgreSQL 16 (schema applied from
migrations/); every equivalence gate passes.
The download_zip per-item N+1 (the audit's highest raw-latency candidate) is
deferred to a dedicated pass: its fix moves the sole authorization inside the
stream call, so it needs an AuthZ-ordering + anti-enumeration proof, not a perf
banner.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
Benchmark-gated, same rule as ROUND2-21: every change ships with a
BEFORE/AFTER counting-allocator benchmark and a byte/-value equivalence
gate; an AFTER that fails to reduce allocations exits non-zero (rollback).
See benches/ROUND22.md and examples/bench_round22_micro.rs. All arms
no-Postgres.
- H1: the hot GET handlers (get_thumbnail, download_file, list_files_query,
list_photos, NextCloud preview, public-share download/access) take
`req: Request` last and read `req.headers()` by borrow instead of axum's
HeaderMap extractor, whose FromRequestParts impl clones the whole request
header table just to read 1-3 headers (the ROUND14 §A4 middleware pattern,
finally propagated to the handlers). 2 -> 0 allocs/req · 9.95x wall.
- W1: native WebDAV write_etag_quoted — the etag emitter for every /webdav/
PROPFIND row (per file AND per folder, up to 500/page) — emits the quotes
as borrowed pre-escaped " text events instead of escaping a "{etag}"
String (the ROUND20 §C1 / ROUND21 §R4 pattern). 3 -> 0 allocs/row.
- C1: CalDAV getetag routed through a shared write_quoted_etag helper across
all 5 sites (3 per-event + 2 per-calendar); the now-dead etag: &mut String
buffer threaded through write_event_response/standard/requested props + the
two per-page buffers removed. 2 -> 0 allocs/row.
- D1: FileDto::from reuses the moved parts.blob_hash instead of cloning it
via the content_hash() getter (the ROUND19/20 move-not-clone sweep missed
it — hash/etag are read before into_parts()). Per file row of every
listing. 1 -> 0 allocs/row.
- E1: CalendarEvent::update_time_range/update_all_day stamp timed
DTSTART/DTEND via fmt::compact_ical_utc stack render (chrono fallback out
of range) instead of the %Y%m%dT%H%M%SZ strftime interpreter. 4 -> 0.
- S1: ShareItemType::try_from uses eq_ignore_ascii_case instead of a
throwaway to_lowercase() String. 1 -> 0 allocs/parse.
Verified: cargo clippy --features bench --all-targets -D warnings clean,
cargo fmt --all --check clean, cargo test --lib --features bench = 529
passed / 0 failed (incl. the OpenAPI-spec-validity test guarding the H1
utoipa-handler signature change).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
Round 21 of the benchmark-gated perf sweep. Six behaviour-preserving,
allocation-reducing changes, each with a BEFORE/AFTER counting-allocator
section in examples/bench_round21_micro.rs and a byte/-value equivalence
gate; all six pass their deterministic alloc gate (a non-winning AFTER
exits 1 = rollback).
- R1: pre-size the 16 CalDAV/CardDAV row-mapper Vecs (+1 HashMap) with
Vec::with_capacity(rows.len()) — the ROUND20 §I1 file-side pattern
extended to the calendar/contact repos it deferred. 7 → 1 allocs/op.
- R2: settle_batch binds a borrowed Vec<&str> instead of cloning every
chunk hash into a Vec<String> (sqlx encodes &[&str] as text[]
identically; favorites_pg_repository.rs:271 precedent). 33 → 1 allocs/op,
39x wall.
- R3: store_loose_chunks keys its intra-request dedup set on the raw
[u8;32] BLAKE3 digest and moves the hex on a duplicate (the ROUND17 §D2
pattern applied to the delta-upload sibling). 401 → 209 allocs/op.
- R4: CardDAV getetag emits borrowed pre-escaped " quotes via a shared
write_quoted_etag helper (ROUND20 §C1 pattern, all 4 CardDAV etag sites).
3 → 0 allocs/op.
- R5: BDAY stamped via the new fmt::compact_date stack renderer instead of
chrono's strftime interpreter (chrono fallback out of the 4-digit-year
range; byte-identical, unit-tested vs chrono). 2 → 0 allocs/op, 10.5x wall.
- R6: NC trashbin folder content-type via Cow::Borrowed instead of
.to_string() on the constant (ROUND16 §M1 pattern). 1 → 0 allocs/op.
See benches/ROUND21.md for the full write-up and the deferred-items list
(HeaderMap-clone hot handlers, Query→typed-struct, WebDAV dead-props
HashSet, and others surfaced by the audit that want their own validated
pass). Validated: cargo fmt, cargo clippy --features bench --all-targets
-D warnings, cargo test --lib --features test_utils (529 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015gHVq5Wy2TzdWeSqtEmK6m
Edouard Vanbelle