Commit Graph

765 Commits

Author SHA1 Message Date
Edouard Vanbelle 53077e0fa3 fix(merge): fix merge of mount + is_favorite feature
fix the merge of moun feature and is_favorite + is_shared feature
    together it was raising an impossible build due to missing schema
2026-07-22 08:30:06 +02:00
Dionisio Pozo ef3e3f45b3 Merge pull request #521 from BCNelson/feat/external-file-mounts
feat(mounts): external file mounts — pluggable provider, read-write, WebDAV/NextCloud, admin UI
2026-07-22 06:52:43 +02:00
Dionisio Pozo 86268049ff Merge pull request #641 from EdouardVanbelle/refactor/front-resource-list
feat(fileDto, folderDto): add is_favorite + is_shared
2026-07-22 06:51:28 +02:00
DioCrafts d66956824c Optimize storage, GC, and upload hot paths 2026-07-22 02:06:29 +02:00
Edouard Vanbelle d9987782c4 feat(fileDto, folderDto): add is_favorite + is_shared
- provide is_shared and is_favorite information in DTO, information propagated as badge/buttons per items
      regarding performances I try to be minimalis on SQL to prevent any perf regression

    - remove old set of sharedids and favoriteids (was not functionnal anymore)
    - fix date picker (no past selection) in grant
    - fix contextmenu close on /files section
2026-07-22 01:55:10 +02:00
Bradley Nelson 05ae217022 fix(mounts): post-merge fmt + audit + minimal lockfile
- rustfmt the integration-test FolderService::new callsites added during
  the upstream merge (long single-line args wrapped).
- Cargo.lock: rebuild from upstream/main's lock so only the
  testcontainers-modules dev-dep subtree and the memmap2 0.9.10→0.9.11
  security bump differ (avoids churning upstream's pins).
- .cargo/audit.toml: ignore the four astral-tokio-tar tar-extraction
  advisories — dev-only, transitive via testcontainers-modules
  (integration-test harness), never in the production binary.
2026-07-21 17:25:48 -06:00
Bradley Nelson 9f4bc54113 Merge upstream/main into feat/external-file-mounts
Resolve conflicts between the external-file-mounts feature and upstream's
D5/D7 refactor (per-file provenance, keyset pagination, cross-drive move
gates, resource-access hook, folder-cascade lifecycle hook).

Key resolutions:
- FolderService::new now takes (repo, authz, file_lifecycle, mount_router);
  all callers + DI updated.
- FileRetrievalService / FileManagementService keep both the mount_router
  and the new resource_access_hook / drive_repo / storage_usage wiring.
- list_files_batch_with_perms: adapt the mount branch from offset- to
  keyset (after_name) pagination, mirroring paginate_mount_entries.
- download_file_impl: keep upstream's &HeaderMap + `impl IntoResponse + use<>`
  signature, retain the mount-download branch.
- Mount DTOs: the retired `owner_id` field maps onto created_by/updated_by
  (the mount owner) — the fields the frontend now uses for owner display.
- admin/+page.svelte: keep upstream's user-delete modal + the 'mounts' tab.
- Bump memmap2 0.9.10 -> 0.9.11 (RUSTSEC critical advisory fix) and
  regenerate Cargo.lock against the merged Cargo.toml.
2026-07-21 17:09:36 -06:00
Dionisio Pozo 67fe944c2a Merge pull request #596 from swissiety/rfc-5789-http-patch 2026-07-22 00:32:27 +02:00
Claude 8e55caa8a9 perf(round29): cache-serve borrow-probe, NC REPORT href buffer, auth per-req allocs, DB over-fetch
Seven behaviour-preserving allocation / copy / bandwidth cuts, each behind a
counting-allocator BEFORE/AFTER gate that exit(1)s unless AFTER allocates
strictly fewer than BEFORE (benches/ROUND29.md, examples/bench_round29_micro.rs).

- [B] Content-cache serve fast path (optimized_inner Tier 1 +
  get_file_range_preloaded — the video-scrub hot path): probe the cache with a
  borrow first and build the owned get_or_load args (quoted-etag / key / id
  Strings) only on a miss, instead of allocating them before every probe and
  discarding them on a hit. 6 -> 0 allocs per cache hit. Splits get_or_load into
  get + load_and_cache so the miss path is not re-probed and the hit/miss stat
  counters stay byte-identical. Also drops the unconditional content_hash/name
  clones that ran for the >=10 MB streaming tier that used neither.
- [A] NextCloud REPORT emit loops: per-row href String (and format! per folder
  row) -> one reused href_buf via nc_href_into / nc_collection_href_into with the
  URL-encoded user computed once per page. 1497 fewer allocs on a 500-row page.
- [C] read_full: a single-frame blob is returned zero-copy instead of a second
  whole-payload memcpy into a fresh BytesMut; multi-frame path unchanged.
- [D] login-lockout key: to_lowercase()+format! -> one pre-sized ASCII buffer
  (non-ASCII keeps str::to_lowercase). 3 -> 1 alloc/req, byte-identical key.
- [E] NC composite-username parse: owned clone/to_string -> &str borrow of the
  already-owned raw_username. 1 -> 0 alloc on the common no-marker path.
- [F] get_contacts_in_group: stop SELECTing the discarded multi-KB vcard column
  (the live method ROUND25 §Q2 missed; ContactDto has no vcard field).
- [G] count_admin_users: add count_users_by_role -> scalar COUNT(*) instead of
  hydrating every admin's full row (incl. up-to-512 KiB avatar + ui_preferences
  JSONB) only to .len() it, on a bootstrap-polled status endpoint.

All seven gates pass; cargo fmt --check and cargo clippy --all-features
--all-targets -D warnings clean. §F/§G additionally validated against a live
PostgreSQL 16 with the full migration set (query validity, result equivalence,
600000 -> 8 byte wire delta on the admin count).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LhpDZxSQTAGnAqCHUdtG5N
2026-07-21 10:25:36 +00:00
Claude 9333037fc7 perf(round28): extend the PROPFIND oc:id reused buffer (round27 H1) to the REPORT emit loops
report_handler's four REPORT emit loops (the filter-files favorites REPORT and the
search REPORT, each a file loop + a folder loop) shared the same per-row oc:id
String that ROUND27 §H1 replaced in the two PROPFIND page loops. Apply the
identical, already-validated transformation: hoist one oc_buf per handler (reused
across both its loops) and compute the id into it via format_oc_id_into instead of
a fresh format_oc_id String per child. 1 String/row -> 0 (amortized). The
write_{file,folder}_response fns already take Option<&str>, so their signatures are
unchanged and the emitted oc:id bytes are byte-identical.

Same transformation benchmarked in ROUND27 §H1 (bench_round27_micro: 998 -> 0
per-row allocs, 2.16x wall), so no new bench. Verified: cargo fmt clean, cargo
clippy --features bench -D warnings clean, cargo test --lib --features bench =
529 passed / 0 failed across 5 consecutive runs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
2026-07-21 01:49:37 +00:00
Claude 8c936de50d perf(round27): NextCloud PROPFIND oc:id per-row buffer, contact JSONB write direct-serialize
Two behaviour-preserving allocation cuts (benches/ROUND27.md), each with a
counting-allocator BEFORE/AFTER gate that rolls back if AFTER does not allocate
fewer than BEFORE:

- H1 NextCloud PROPFIND: the streaming page loops built oc:id as a fresh String
  per child (format_oc_id -> format!("{:08}{}", id, instance)). Add
  format_oc_id_into(&mut buf, id, svc) and compute into one oc_buf reused across
  the page (next to the existing href buffer) — 1 String/row -> 0. 998->0
  per-row allocs on a 500-row page, 2.16x wall. The write fns still take
  Option<&str>, so no signature change; oc:id bytes identical. Scoped to the two
  PROPFIND page loops (the hot directory-listing path); REPORT/trashbin deferred.
- P2 contact create/update: bind sqlx::types::Json(&dtos) (Encode runs to_writer
  straight into the JSONB buffer) instead of serde_json::to_value(&dtos) + bind,
  which built a throwaway Value DOM per JSONB column. Write-side twin of ROUND23
  J1. 21->2 allocs, 4.68x wall for a 3-entry column. Behaviour-preserving:
  to_value sorts keys and direct serialize keeps struct order, but Postgres
  normalizes JSONB key order so the stored value is identical (verified via psql:
  '{...alpha...}'::jsonb = '{...struct...}'::jsonb -> t), and reads decode by
  field name; the etag comes from the domain entity, not the stored JSONB.

Adds bench_round27_micro. Verified: cargo fmt clean, cargo clippy --features
bench -D warnings clean (real exit), cargo test --lib --features bench = 529
passed / 0 failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
2026-07-21 01:30:53 +00:00
Claude 5b2bb8f883 perf(round26): drive-policy JSONB decode, CachedBlobBackend shard-dir pre-create, delta-upload foldhash
Three benchmark-gated optimizations from the ROUND25 backlog (benches/ROUND26.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:

- P1 drive_pg_repository policy reads: decode d.policies through
  sqlx::types::Json<DrivePolicies> (one from_slice over the raw JSONB bytes)
  instead of a throwaway serde_json::Value DOM + DrivePolicies::from_value —
  6 -> 0 allocs/read, 2.77x wall. A shared policies_from_row helper preserves
  the lenient unwrap_or_default fallback (malformed bag -> all-false).
- D1 CachedBlobBackend: pre-create the 256 {00..ff} shard dirs at initialize()
  (mirroring LocalBlobBackend, reusing HEX_PREFIXES) and drop the redundant
  per-write create_dir_all on already-existing shards — ~45us + a blocking-pool
  dispatch removed per cache write on cached-remote deployments.
- G1 delta-upload have/need hash sets (distinct_hashes, authorize_chunk_download):
  SipHash -> foldhash::quality::RandomState — a fast hasher that stays
  DoS-resistant via a per-instance random seed, the required property for the
  attacker-controlled 64-hex client hashes — 2.37x wall on a 40k-hash
  negotiation. foldhash was already in the lockfile transitively (hashbrown).

Tested and REVERTED (kept as-is): moving the moka eviction unlink off the reactor
via spawn_blocking. The benchmark refuted it — on the local cache dir the
spawn_blocking dispatch (~20us) costs more than the inline unlink (~7us) it would
replace. See ROUND26.md §D2.

Adds bench_round26_{micro,diskio,hasher} (counting allocator / async wall / wall).
Verified: cargo fmt clean, cargo clippy --features bench -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
2026-07-21 01:05:12 +00:00
Claude e8e4ef4b15 perf(round25): in-place encrypted decrypt, dedup hash move, dead folder-Query, playlist N+1 fold, contact vcard over-fetch
Five benchmark-gated optimizations from a fresh six-way audit (benches/ROUND25.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:

- M1 EncryptedBlobBackend::decrypt_bytes: replace split_off (a fresh Vec + full
  ciphertext memcpy on every decrypted chunk, contradicting its own "in place"
  doc) with in-place detached decrypt + a zero-copy Bytes::slice past the nonce.
  Peak RAM per read drops from ~2x to ~1x the payload (-262KB/op at 256KiB;
  scales with blob size). Plaintext byte-identical; tamper/wrong-key tests pass.
- M2 delta commit: move-unzip the owned chunk list instead of a third
  per-occurrence hash clone (-4000 allocs on a 4000-chunk commit).
- M3 folder ZIP download: drop the dead Query<HashMap> extractor it never read
  (byte-identical response; 5->0 allocs/request).
- Q1 public-playlist listing: fold the per-playlist COUNT(*) N+1 into one
  LEFT JOIN ... GROUP BY via a new inherent repo method (101 -> 1 round-trips,
  36x wall on a 100-playlist page).
- Q2 contact REST listings (paginated/search/by-group): stop over-fetching the
  multi-KB vcard TEXT the ContactDto discards, via a shared lite row mapper and
  narrowed SELECTs (6.4x wall on 1000 contacts with 8KiB vcards). The whole-book
  vCard export and CardDAV sync paths keep the column.

Adds bench_round25_micro (counting allocator tracking count+bytes) and
bench_round25_queries (live Postgres), both with equivalence gates and a
rollback exit(1). Verified: cargo fmt clean, cargo clippy -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
2026-07-21 00:07:44 +00:00
Dionisio Pozo 0d82632f92 Merge pull request #629 from EdouardVanbelle/refactor/front-resource-list
Refactor(front) restore legacy frontend features
2026-07-21 00:16:40 +02:00
Edouard Vanbelle 931e27d09c fix(resources): wire missing created_by and updated_by 2026-07-20 21:04:32 +02:00
Claude ffb536e0ae perf: round 24 — download_zip per-item authz+metadata N+1 → batch (validated authorization pass)
The ROUND23-deferred download_zip N+1, given its own validated pass. The
individually-selected files were authorized + fetched one at a time via
get_file_with_perms (require + get = 2 serial round-trips/file) before any
streaming — a 200-file selection was 400 serial round-trips. AFTER routes the
whole multi-select through the new
FileRetrievalService::get_files_by_ids_with_perms: one check_files_read_batch
(the PgAclEngine resolves every file's drive in ONE query and primes the
resource->drive cache) + one get_files_by_ids. 2N round-trips -> 2.

Authorization is unchanged and still enforced BEFORE any ZIP entry is written:
- add_file_entry_streamed writes the entry header (the filename) before it opens
  the authorized stream, so the pre-filter is load-bearing — a denied file must
  never reach it or its name leaks into the archive. AFTER a denied/missing id
  is absent from the authorized map and is skipped in the same input order,
  exactly as the old loop skipped a denied get_file_with_perms; it never reaches
  the entry write. The authz moved from a per-file require to one batch check
  EARLIER in the same function, not into or after the stream.
- The stream open keeps its own per-file Read check (now a primed-cache hit) +
  Recents recording; check_files_read_batch is documented + gated as identical
  to looping require.

Because the change is authorization-sensitive, the gate is the security property
itself. bench_round24_zip_authz drives the real PgAclEngine over a seeded,
interleaved mix of owned (granted drive) + denied (other drive) + missing ids
and asserts: the batch inclusion set AND input order are identical to the
per-file require loop; the included set is exactly the caller's owned files; no
denied or missing id is ever included (the authz-regression tripwire); and the
batch fetch returns exactly the owned files. Latency (cold, 600-item 1/3-owned
selection): 559 -> 267 ms (2.10x; the realistic all-owned selection is O(1) ->
a larger win). See benches/ROUND24.md.

The folder selections are left as-is (root counts are small and there is no
check_folders_read_batch primitive to batch through).

Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo
fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0
failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
2026-07-20 17:12:38 +00:00
Claude 1ec7030cc7 perf: round 23 — Postgres query-shape pass: typed JSONB decode, drive-policy borrow-deserialize, user-profile join!, subject-group CTE reuse, dedup unzip
Benchmark-gated, same rule as ROUND2-22: BEFORE/AFTER with a value-equivalence
gate and rollback-on-regression. Two harnesses — bench_round23_micro (no
Postgres; deterministic allocation gate) and bench_round23_queries (live
Postgres; p50 latency + strict equivalence gate against seeded fixtures). See
benches/ROUND23.md.

- J1: contact_pg_repository::row_to_contact (+ the inlined contact_group sibling)
  decode the 3 JSONB columns via sqlx::types::Json<T> (one from_slice pass)
  instead of row.get::<serde_json::Value> + from_value (a throwaway Value DOM
  per column, walked a second time). Per contact row of every list / multiget /
  CardDAV sync. Micro 84 -> 33 allocs/op (2.15x); PG 3794 -> 2360 ns/contact
  (1.61x) on 500 real rows.
- J2: DrivePolicies::from_value deserializes straight from the borrow
  (T::deserialize(&Value)) instead of from_value(value.clone()) — dropping the
  full-DOM clone on every drive-policy read (move/copy, share, grant); one-line
  body change, all 7 callers unchanged. Micro 5 -> 0 allocs/op (11.51x).
- P1: get_user_profile overlaps the two independent caller+target reads with
  tokio::join! (self-case still a single fetch; caller-error precedence
  preserved via caller_res? first) instead of two serial round-trips. PG
  577 -> 312 us/call (1.85x).
- G1: subject_group remove_member computes the child's transitive-user recursive
  CTE once and reuses it for both the would-empty pre-check and the cache
  invalidation, instead of running the identical CTE twice (the edge delete is
  above the child, so its descendants can't change). PG 829 -> 412 us/removal
  (2.01x).
- U1: dedup_service (store_loose_chunks final registration + the ingest
  run_rollback) reshapes the owned, dead-after Vec<(String,i64)> via
  into_iter().unzip() instead of cloning every 64-byte hash for the
  sync_blobs(&[String]) + UNNEST bind. Micro 256 -> 0 hash clones.

Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo
fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0
failed. The PG benches run against a local PostgreSQL 16 (schema applied from
migrations/); every equivalence gate passes.

The download_zip per-item N+1 (the audit's highest raw-latency candidate) is
deferred to a dedicated pass: its fix moves the sole authorization inside the
stream call, so it needs an AuthZ-ordering + anti-enumeration proof, not a perf
banner.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
2026-07-20 15:25:42 +00:00
Claude 992bdae898 perf: round 22 — hot-GET HeaderMap borrow, native-WebDAV/CalDAV etag borrowed quotes, FileDto content_hash move, CalendarEvent stamp, ShareItemType case-fold
Benchmark-gated, same rule as ROUND2-21: every change ships with a
BEFORE/AFTER counting-allocator benchmark and a byte/-value equivalence
gate; an AFTER that fails to reduce allocations exits non-zero (rollback).
See benches/ROUND22.md and examples/bench_round22_micro.rs. All arms
no-Postgres.

- H1: the hot GET handlers (get_thumbnail, download_file, list_files_query,
  list_photos, NextCloud preview, public-share download/access) take
  `req: Request` last and read `req.headers()` by borrow instead of axum's
  HeaderMap extractor, whose FromRequestParts impl clones the whole request
  header table just to read 1-3 headers (the ROUND14 §A4 middleware pattern,
  finally propagated to the handlers). 2 -> 0 allocs/req · 9.95x wall.
- W1: native WebDAV write_etag_quoted — the etag emitter for every /webdav/
  PROPFIND row (per file AND per folder, up to 500/page) — emits the quotes
  as borrowed pre-escaped &quot; text events instead of escaping a "{etag}"
  String (the ROUND20 §C1 / ROUND21 §R4 pattern). 3 -> 0 allocs/row.
- C1: CalDAV getetag routed through a shared write_quoted_etag helper across
  all 5 sites (3 per-event + 2 per-calendar); the now-dead etag: &mut String
  buffer threaded through write_event_response/standard/requested props + the
  two per-page buffers removed. 2 -> 0 allocs/row.
- D1: FileDto::from reuses the moved parts.blob_hash instead of cloning it
  via the content_hash() getter (the ROUND19/20 move-not-clone sweep missed
  it — hash/etag are read before into_parts()). Per file row of every
  listing. 1 -> 0 allocs/row.
- E1: CalendarEvent::update_time_range/update_all_day stamp timed
  DTSTART/DTEND via fmt::compact_ical_utc stack render (chrono fallback out
  of range) instead of the %Y%m%dT%H%M%SZ strftime interpreter. 4 -> 0.
- S1: ShareItemType::try_from uses eq_ignore_ascii_case instead of a
  throwaway to_lowercase() String. 1 -> 0 allocs/parse.

Verified: cargo clippy --features bench --all-targets -D warnings clean,
cargo fmt --all --check clean, cargo test --lib --features bench = 529
passed / 0 failed (incl. the OpenAPI-spec-validity test guarding the H1
utoipa-handler signature change).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
2026-07-20 13:48:47 +00:00
Claude 77f13ac643 perf: round 21 — CalDAV/CardDAV row-mapper pre-size, dedup hash-bind & digest-key dedup, CardDAV etag/BDAY emit, NC trashbin content-type
Round 21 of the benchmark-gated perf sweep. Six behaviour-preserving,
allocation-reducing changes, each with a BEFORE/AFTER counting-allocator
section in examples/bench_round21_micro.rs and a byte/-value equivalence
gate; all six pass their deterministic alloc gate (a non-winning AFTER
exits 1 = rollback).

- R1: pre-size the 16 CalDAV/CardDAV row-mapper Vecs (+1 HashMap) with
  Vec::with_capacity(rows.len()) — the ROUND20 §I1 file-side pattern
  extended to the calendar/contact repos it deferred. 7 → 1 allocs/op.
- R2: settle_batch binds a borrowed Vec<&str> instead of cloning every
  chunk hash into a Vec<String> (sqlx encodes &[&str] as text[]
  identically; favorites_pg_repository.rs:271 precedent). 33 → 1 allocs/op,
  39x wall.
- R3: store_loose_chunks keys its intra-request dedup set on the raw
  [u8;32] BLAKE3 digest and moves the hex on a duplicate (the ROUND17 §D2
  pattern applied to the delta-upload sibling). 401 → 209 allocs/op.
- R4: CardDAV getetag emits borrowed pre-escaped &quot; quotes via a shared
  write_quoted_etag helper (ROUND20 §C1 pattern, all 4 CardDAV etag sites).
  3 → 0 allocs/op.
- R5: BDAY stamped via the new fmt::compact_date stack renderer instead of
  chrono's strftime interpreter (chrono fallback out of the 4-digit-year
  range; byte-identical, unit-tested vs chrono). 2 → 0 allocs/op, 10.5x wall.
- R6: NC trashbin folder content-type via Cow::Borrowed instead of
  .to_string() on the constant (ROUND16 §M1 pattern). 1 → 0 allocs/op.

See benches/ROUND21.md for the full write-up and the deferred-items list
(HeaderMap-clone hot handlers, Query→typed-struct, WebDAV dead-props
HashSet, and others surfaced by the audit that want their own validated
pass). Validated: cargo fmt, cargo clippy --features bench --all-targets
-D warnings, cargo test --lib --features test_utils (529 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015gHVq5Wy2TzdWeSqtEmK6m
2026-07-20 08:48:42 +00:00
Claude 867e1fe259 perf: round 20 — iCal/vCard parse allocs, owned-DTO moves, Result-collect pre-size, NC etag/favorites emit
Benchmark-gated (benches/ROUND20.md), same rule as rounds 2-19: every change
ships with a BEFORE/AFTER counting-allocator micro-benchmark and a byte-value
equivalence gate; a non-winning AFTER is rolled back (never applied). The
rollback rule is encoded in the harness (GATE FAIL exit). All 8 sections pass.

Reproduce: cargo run --release --features bench --example bench_round20_micro

- A1 CalendarEvent iCal parse: replace the throwaway per-property
  HashMap<String,Vec<String>> (DTSTART/DTEND/RECURRENCE-ID) with a direct
  VALUE=DATE scan; prop_with_params kept #[cfg(test)] (6->2 allocs/event, 4.2x)
- A2 UserDto::from: add User::into_parts and MOVE image (<=512 KiB data URI)
  + ui_preferences JSON instead of cloning on every /api/auth/me (27->14 allocs)
- A3 parse_vcard: drop the per-line to_ascii_uppercase copy + the lines Vec;
  promote ascii_ci_contains to common::text and share it (8->1 allocs/contact)
- A4 Calendar/AddressBook DTO: into_parts move incl. custom_properties map (18->10)
- I1 file-listing repos: collect::<Result<Vec>>() size-hints to 0 and grows from
  capacity 0; pre-size with Vec::with_capacity (8->1 container reallocs, 4 sites)
- I4 plaintext_stream: lazy emit iterator instead of eager Vec collect (43x wall)
- C1 NC write_etag_element: borrowed pre-escaped quote events, no owned quoted
  String/escape re-alloc; byte-identical output (3->0 allocs/PROPFIND row)
- C3 NC favorites REPORT: map.remove() move instead of get().clone() (~7 allocs/fav)

Deferred (documented in ROUND20.md): NC oc:id/trashbin buffer reuse, I1 sibling
CardDAV/CalDAV listing paths, Contact JSONB Json<Vec<_>> decode, dedup
settle_batch &str bind, and a fast DoS-resistant hasher for hot trusted-key maps
(needs a dependency decision).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JsJjcVX9RoN96DMa35Wqzd
2026-07-20 00:18:54 +00:00
Claude 9754aecfa9 perf: round 19 — auth/WOPI/vCard/PROPFIND per-request & per-row alloc cuts
Benchmark-gated (examples/bench_round19_micro.rs, benches/ROUND19.md): every
section ships a BEFORE/AFTER counting-allocator arm with a byte/-value
equivalence gate and a GATE-FAIL-rollback exit. All eight pass. No Postgres.

- M1 verify_basic_auth cache key: blake3::hash(format!("{u}:{p}")) → incremental
  Hasher (byte-identical key, 2→0 allocs on every Basic-auth DAV request)
- M2 WopiTokenService: prebuild Validation/DecodingKey/EncodingKey in new()
  instead of per-call (mirrors JwtTokenService; 16→12 allocs/validate)
- V1/V2 vCard emit (contact_to_vcard/generate_vcard): FN fallback drops the
  throwaway to_string, NOTE skips the escape copy for newline-free notes, REV
  uses new common::fmt::compact_ical_utc stack renderer (11.5× vs chrono
  strftime, 3→0 allocs); per-contact 9→4 allocs
- M4 trash_service::row_to_item_dto: move name/path/blob_hash out of the owned
  row instead of cloning (3 clones/file row gone)
- M5 search cache key: Uuid::hyphenated().encode_lower stack buffer instead of
  to_string (identical u64 key, 1→0 allocs/request)
- M6 streaming PROPFIND: reuse one href buffer across the page instead of a
  format! per child (native + NC handlers; 192→3 allocs on a 64-child page)
- M7 nextcloud extract_url_user: return Cow instead of forcing into_owned
  (zero-alloc on the common ASCII-username path)

common::fmt::compact_ical_utc added with chrono-parity unit tests (CASES +
60-year sweep). cargo fmt + clippy --all-targets clean; 526 lib unit tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ront9bk7YMoffVQkGG47gh
2026-07-19 22:29:48 +00:00
Claude 6bc09cb3b3 perf: round 18 — calendar-event in-place iCal edit, ResourceList incremental id-index
Two items from the ROUND17 deferred list, each benchmark-gated with a
BEFORE/AFTER equivalence gate and a rollback-on-regression check (ROUND2–17
discipline). See benches/ROUND18.md.

[C1] backend — CalendarEvent::update_ical_property / remove_ical_property
rewrote the ENTIRE ical_data body with format!("{}{}{}") on every call and
allocated two search needles per call. calendar_storage_adapter::update_event
fans a multi-field edit out into one call per changed field, so a full REST
edit paid one full-body (up to ~11 KB) allocation per property. The body is
now mutated in place (replace_range for an existing property, four insert/
insert_str for a new one, byte-identical spans) and the single "\nNAME:"
needle is built on the stack (the "\r\nNAME:" needle was redundant — the LF
form is its suffix). bench_round18_micro [C1]: 70 -> 2 allocs/op (68 fewer),
2.46x wall, emitted body byte-identical.

[F1] frontend — ResourceList itemIndexById rebuilt a fresh Map over the whole
accumulated list every infinite-scroll page (O(N)/page, O(N^2) drain) and,
being a new instance each page, re-fired the reap-stale effect (another O(N)
id Set/page). New ItemIndexBuilder extends a persistent Map with the fresh
page only and reuses the reference across appends; the reap-stale effect now
tests membership against it. round18.bench.test.ts [F1]: 40x50 drain 74.1 ->
6.4 ms (11.5x), deep-equal to the reference at every page, reference-contract
gate (same-ref append / new-ref rebuild).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoxFtikahM1N4PE5s3ZVH3
2026-07-19 20:53:30 +00:00
Claude c207e66f4b perf: round 17 — dedup ingest/verify hash-clone purge, CardDAV vCard TYPE tokens
Targets the content-addressable dedup write path (the ROUND15-deferred
"dedup_service hash-String re-allocations") from both ends — the streaming
ingest loop and the delta-commit verification read — plus a CardDAV vCard
micro-cut. Every change is benchmark-gated with a hard rollback rule; no
PostgreSQL needed for any arm (benches/ROUND17.md).

Backend (counting-allocator, examples/bench_round17_micro.rs):
- D2 chunk-ingest (store_from_stream, the hottest write path — every chunk of
  every upload): the 64-char hex hash String was allocated 3x per chunk
  (to_hex + chunk_hashes clone + session_seen insert-clone, the last dropped on
  a duplicate). The intra-upload dedup set now keys on the raw 32-byte BLAKE3
  digest ([u8;32], Copy, no heap) and the manifest push is branch-split so a
  duplicate moves the hex in: 3 -> 2 allocs/new chunk, 3 -> 1/duplicate.
  Measured 214 -> 149 allocs/op (1.14x wall) on a 64-chunk 1-in-2-dup batch;
  smaller/faster set too (32B inline keys vs 64B heap Strings).
- D1 hash_chunk_sequence (delta-commit verification): took chunks by
  &[(String,u64)] and fed the backend stream with iter().cloned(), re-cloning
  every chunk hash a second time on top of the owned Vec the caller already
  built. Take the Vec by value + into_iter(): 65 -> 0 internal allocs/op, ~2.5us
  of clone work removed per verify.
- V1 vCard TYPE tokens (contact_to_vcard + generate_vcard, 5 sites): each
  EMAIL/TEL/ADR TYPE= param used ty.to_uppercase() — a throw-away String per
  token per contact. New shared fmt::push_upper writes the upper-cased chars
  straight into the buffer (byte-identical to str::to_uppercase, unit-tested):
  13 -> 5 allocs/op, 1.19x wall.

Gates: each section asserts byte/-value equivalence (D2 the ordered manifest +
sizes + write-set; D1 the removed clone is a pure copy; V1 the full vCard) and
exits non-zero if an AFTER arm fails to reduce allocations. push_upper is
unit-tested byte-equal to str::to_uppercase (fmt::tests). Verified end-to-end:
cargo fmt clean, clippy --release --all-targets --features bench -D warnings
clean, and the harness prints GATE PASS against the built release lib.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XMmt7vNETYUbEG3Hc17LDx
2026-07-19 19:33:41 +00:00
Dionisio Pozo a09569b5c2 Merge pull request #625 from AtalayaLabs/claude/performance-optimization-analysis-raoezl
Round 16: incremental lanes/contextMap builders & alloc cuts
2026-07-19 19:35:26 +02:00
Claude 955f4a7b9f perf: round 16 — shares-lane & contextMap incremental builders, folder/href/disposition/preview alloc cuts
Finishes the route-level half of the O(N²/page) grouped-listing class ROUND15
fixed inside ResourceList, plus a backend CPU/alloc micro-pack. Every change is
benchmark-gated with a hard rollback rule; no PostgreSQL needed for any arm
(benches/ROUND16.md).

Frontend (vitest):
- F1 "My shares" lanes: the `lanes` $derived.by re-bucketed the whole
  accumulated grant list on every page and every grant edit. SharedLanesBuilder
  re-emits only the fresh page (fan-out + first-appearance header), reusing
  untouched lanes' array refs. 25.5x fewer emit calls, 8.8x wall, O(N²/page)→O(N).
- F2 contextMap (trash/recent/favorites/shared-with-me): each rebuilt a fresh
  N-key Map, re-hashing every accumulated id, per page. primeContextPage holds a
  persistent SvelteMap primed per page (the shipped favoriteIds shape).
  25.5x fewer entry calls, 7.2x wall.
- Extracted the shared O(1) append test (isAppendExtension); F1's gate re-covers it.

Backend (counting-allocator):
- M1 folder display constants Arc::from -> intern_display (3 sites): 3 -> 0 allocs/row.
- M2 build_content_disposition (every download + Range seek): 3 -> 1 alloc, 6x, 2.67x wall.
- M3 nc_href (every NC PROPFIND/REPORT href): Vec+join+format -> one pre-sized
  buffer, keeping urlencoding::encode (byte-identical). 38 -> 27 allocs/op.
- M4 NC preview fileId: collect-then-parse -> borrow-slice parse. 4 -> 0 allocs.

Gates: sharedLanes/listContext.bench.test.ts, examples/bench_round16_micro.rs
(GATE PASS all sections). Frontend: vitest 331 pass, svelte-check clean.
Backend: clippy -D warnings clean, 524 lib tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0193NjactJVqfU32gxeJDj8m
2026-07-19 17:30:22 +00:00
Edouard Vanbelle dfa4f7a075 admin: show only internal users 2026-07-19 17:04:29 +02:00
Edouard Vanbelle d4ae23ef13 chore: apply clippy recos 2026-07-19 17:04:24 +02:00
Edouard Vanbelle 26b8b530c9 security(users): prevent ext. user to become admin 2026-07-19 16:26:11 +02:00
Edouard Vanbelle 2a08fe83ae feat(user): admin can promote external user + security on deletion
promotion by admin of external user into internal possible
    deletion of a user request admin to enter it's email, this is to prevent any miss click
2026-07-19 16:26:11 +02:00
Edouard Vanbelle e003a8c55b feat(admin): display external users for security reasons 2026-07-19 16:26:11 +02:00
Edouard Vanbelle 05dfda9aa3 feat(drive): add quota update handler
per today: admin only can update quota
    shared drive can have quota updated (personal drive's quota belong to user's quota)
2026-07-19 16:25:41 +02:00
Claude 3be85fa9f0 perf: round 15 — grouped-listing O(N²) rebucket, exif/reseed allocs, tantivy zero-hit snippet skip
Benchmark-gated, same rule as rounds 2–14: every change ships with a
BEFORE/AFTER benchmark and an equivalence/safety gate; an AFTER that doesn't
beat its BEFORE is rolled back. The rule is encoded per harness (GATE FAIL
non-zero exit in the Rust examples, threshold expect() in vitest).

F1 — Grouped listings (trash / recent / favorites / shared-with-me)
re-bucketed the WHOLE accumulated list on every infinite-scroll page.
ResourceSectionsBuilder (new, off the reactive graph) re-buckets only the
fresh page and hands VirtualList the same rows array reference for untouched
buckets. 50×50 (2 500-item) drain: 63 750 → 2 500 bucketOf calls (25.5×),
12.5 → 1.3 ms wall (9.9×); O(N²/page) → O(N). Deep-equal to the full-rebuild
reference at every page for both a contiguous (date) and a non-contiguous
(trash-by-drive) group-by; reference-stability + fallback gated.

B1 — exif Make/Model: the display String was thrown away to allocate the
trimmed copy; display_value_trimmed trims in place (drain + truncate), 2 → 1
alloc per field (8 → 4 allocs/op, 1.26×).

B2 — content-index worker: text_extractor::supports (lowercases MIME +
extension) was called twice per file per drain batch; classify once into a
Vec<bool> and thread it through both uses. 256-file batch: 704 → 353 allocs,
34.5 → 16.7 µs (2.07×).

B3 — tantivy: skip SnippetGenerator::create on a zero-hit content search
(return Ok(vec![]) once top_docs.is_empty()); the per-hit loop was empty.
400-doc index: 1 575.6 → 1 237.2 ns (1.27×), widens with index size.

Harnesses: examples/bench_round15_micro.rs, examples/bench_round15_tantivy.rs,
frontend resourceSections.bench.test.ts; writeup in benches/ROUND15.md. Also
normalizes two round14 bench examples that were committed unformatted
(cargo fmt --all).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012o47jSrtL7xuNGTHXmtiYL
2026-07-19 11:36:47 +00:00
Claude c930f865b0 perf: round 14 — faces narrow projection, auth per-request allocs, CalDAV emit buffers, frontend set churn
Benchmark-gated (benches/ROUND14.md); every change ships a BEFORE/AFTER
benchmark with an equivalence gate and is rolled back on regression (the
rule is encoded as a GATE FAIL exit / threshold expect).

Backend
- Q1 faces_for_file → narrow face_boxes_for_file(id, person_id, bbox) with the
  caller filter pushed into SQL: drops the 2 KiB embedding BYTEA + 6 unused
  columns per face. 15-face lightbox open 0.312→0.219 ms, 32 KB→840 B/req.
- A1 cookie auth uses the borrow-only extract_cookie_str (already backs CSRF)
  instead of extract_cookie_value's owned String: -1 alloc/cookie request.
- A2 compute_relevance ASCII case-fold fast path vs name.to_lowercase() per
  result row (Unicode fallback preserved): 1.40x, 12→3 allocs/page.
- A3 sub pre-parsed to Uuid at decode time (TokenClaims.sub_id) vs re-parsing
  the 36-char claim on every request incl. cache hits: 22.7→0.7 ns.
- A4 auth + NextCloud middlewares borrow request.headers() instead of taking
  axum's HeaderMap extractor (a full map clone): 2→0 allocs/authed request.
- A5 CalDAV getlastmodified via the stack rfc2822_utc (byte-identical to
  chrono) vs a per-event to_rfc2822() heap String: 5→0 allocs.
- A6 CalDAV per-event href + quoted etag written into reused page buffers vs a
  fresh format! pair per event: 3.48x, 240→6 allocs/40-event page.

Frontend
- F1 t() shares one frozen EMPTY_PARAMS for the no-interpolation call forms vs
  a throwaway {} per call: -1 alloc/call.
- F2 favorites favoriteIds is a persistent SvelteSet with per-page add (clear
  on reset) vs a brand-new set over the whole accumulated list each page:
  22.3x over a 40-page drain (O(N^2)→O(N)).

Verified: cargo check --all-targets, cargo clippy -D warnings, both bench
packs (GATE PASS), frontend npm run check + vitest (4/4). ROUND14.md also
records the investigated-but-deferred backlog (music N+1, contact vcard
over-fetch, CachedBlobBackend syscalls, ResourceList.sections builder, etc.).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PymgCdK78NzUF3oRAQCJfN
2026-07-19 10:22:12 +00:00
M.Schmidt a01b0a856e fix error mapping for put; cross surface interop ressource locking; put quota leftovers; 2026-07-19 10:36:26 +02:00
M.Schmidt af74c94028 fix(webdav): make PATCH's concurrency guard a real compare-and-swap
The app-level ETag re-check before the write still left a gap between the check and the actual UPDATE for a concurrent writer to land in.
Push the check into the write path itself: swap_blob_hash now takes an expected_hash and only applies the SET under the same FOR UPDATE row lock it already held, closing the race instead of just narrowing it. Adds ErrorKind::PreconditionFailed (412) for the CAS-miss path; PUT/WOPI/chunked-upload keep blind-overwrite semantics by passing None
2026-07-19 10:36:07 +02:00
M.Schmidt d57f7bfe3a fix(webdav): wire orphaned PATCH tests + fix NC error-mapping and path bugs they caught
webdav_patch.hurl and nc_webdav_patch.hurl existed with real coverage
since the original PATCH commits but were never added to
tests/api/run.sh, so just api-test/CI silently skipped them. Wire both
in, fix nc_webdav_patch.hurl's header-after-[BasicAuth] ordering bug
that meant it had never actually passed, and add two new
consistency-focused files chaining PATCH operations with
cross-protocol/cross-surface verification:

- webdav_patch_consistency.hurl: chained overwrites with ETag-change
  checks, GET/HEAD/PROPFIND cross-protocol agreement, quota-507
  leaving the file byte-for-byte unchanged, direct_put_max_bytes
  prefix/suffix regression coverage.
- nc_webdav_patch_consistency.hurl: Editor/Viewer/Outsider permission
  matrix, cross-surface lock interop, quota-507 via the NC surface.

Running these surfaced two real bugs in the NC PATCH handler, both
fixed here:

- The write step mapped every error (including a legitimate anti-enum
  permission denial) to a raw 500 instead of AppError::from(e), unlike
  the plain surface. A Viewer without Update permission got a 500
  leak instead of the expected 404.
- nc_to_internal_path() didn't strip the leading '/' that chroot.path
  carries from StoragePath::to_string(), so a LOCK taken via /webdav/
  silently failed to block PATCH via /remote.php/dav/ on the same
  file — the exact-string lock-store lookup never matched. Added a
  regression unit test.
2026-07-19 10:35:35 +02:00
M.Schmidt c390a781bb fix(webdav): close PATCH gaps found in review (quota, authz, cap, races)
Fixes to the RFC 5789 PATCH implementation found by review of the
rfc-5789-http-patch branch:

- nextcloud/webdav_handler.rs::handle_patch now enforces storage quota
  before committing, matching the plain WebDAV surface (was a quota
  bypass via the NextCloud endpoint).
- The plain surface's If-Match/If-None-Match comparison reused a
  hand-rolled single-value strong compare that mishandled weak (W/)
  validators and multi-value lists. Moved the correct RFC 7232 helpers
  (already used by the NC surface) into the shared handler file so both
  surfaces use one conformant implementation.
- NC handle_patch resolved the target file via get_file_by_path, which
  performs no authorization check, before any permission-gated call —
  for a full-file-range patch this could leak size/ETag via 412/416
  responses to a caller without Read on that file. Added the same
  explicit authz.require(Read, ...) the plain surface already has.
- ingest_range_patch_to_cas capped the whole spliced stream (prefix +
  edit + suffix) against direct_put_max_bytes, so PATCH became
  permanently unusable on any file at or above that size regardless of
  edit size. The cap now only bounds the edit itself.
- NC handle_patch had no active-lock guard, so a LOCK taken via
  /webdav/ didn't protect the same file reached through
  /remote.php/dav/. Now shares enforce_native_lock with the plain
  surface.
- Added a re-check of the file's ETag immediately before the write on
  both surfaces, narrowing (not eliminating — that would need
  compare-and-swap support in the write path) the window in which two
  concurrent PATCHes to disjoint ranges could silently clobber each
  other.
- NC handle_patch returned 404 for a PATCH on a directory instead of
  409 like the plain surface; now checks folder existence first.
- The Content-Length-vs-X-Update-Range span check only fired when
  Content-Length was present, so a chunked-transfer body could silently
  diverge from the declared span. ingest_range_patch_to_cas now counts
  actual body bytes and validates against the declared span
  regardless, discarding the ingested blob on mismatch.
2026-07-19 10:35:35 +02:00
M.Schmidt 93ae7ab142 feat(webdav): extend HTTP PATCH to the NextCloud surface (RFC 5789)
The NextCloud-compatible WebDAV surface (/remote.php/dav/…) had no PATCH
dispatch arm at all — requests fell through to 405 — unlike the plain-file
surface (see the sibling commit on this repo's rfc-5789-http-patch work).

Adds handle_patch to nextcloud/webdav_handler.rs, reusing the plain
surface's X-Update-Range mechanism directly instead of duplicating it:
- api/handlers/webdav_handler.rs::parse_update_range is now pub(crate)
  so both surfaces share the same header-parsing/validation logic.
- upload_ingest::ingest_range_patch_to_cas (already surface-agnostic)
  splices the request body between the file's untouched prefix/suffix
  byte ranges and re-ingests through the same content-addressable
  pipeline handle_put uses.

Follows this file's own handle_put conventions rather than the plain
handler's: no active-lock guard (the NC surface has no LOCK/UNLOCK
dispatch arm at all) and no explicit storage-quota check (handle_put
doesn't do one either on this surface) — matching the sibling handler
instead of importing behavior the NC surface doesn't otherwise have.

Adds PATCH to the OPTIONS Allow header.

Also fixes a pre-existing clippy::useless_borrows_in_formatting warning
in thumbnail_service.rs (unrelated to this change, but blocking a clean
clippy run on this branch).

Adds tests/api/nc_webdav_patch.hurl covering explicit-range and append
PATCH, the Content-Range rejection, the missing-header 400, and PATCH
on a nonexistent file.
2026-07-19 10:35:35 +02:00
M.Schmidt d3546305f6 feat(webdav): implement HTTP PATCH for partial content updates (RFC 5789)
RFC 4918 §9.7.1 forbids partial updates on PUT; this adds PATCH as the
supported mechanism instead, via an X-Update-Range header (bytes=<start>-<end>
or append). Reuses the existing CAS/dedup pipeline by splicing the request
body between the file's untouched prefix/suffix byte ranges and re-ingesting
as one continuous stream, so unedited chunks dedup for free.
2026-07-19 10:35:35 +02:00
Claude f58d72a780 perf: round 13 — grouped-view virtualization, notification/login query narrowing, HTTP dedup, locale precompute
Benchmark-gated (BEFORE/AFTER + equivalence/safety gate per change), same
discipline as rounds 2-12. Full write-up in benches/ROUND13.md.

Shipped:
- V1 Grouped views windowed (files route + ResourceList). The grid arm was
  the last unwindowed path (trash is grouped-by-default in grid): each
  swimlane now feeds its own VirtualList, outer container a flex stack.
  vitest gate: 800-item grouped grid mounts <120 .file-item (was 800).
- Q1 get_users_by_ids drops the <=512 KiB avatar image + ui_preferences
  JSONB (notification path never reads them). 30-member fan-out 8.60 ->
  0.25 ms (34.3x), ~7.7 MB off the wire.
- Q2 Login provisioning is_empty() -> SELECT EXISTS for calendar + address
  book (every login). 0.193 -> 0.170 ms, widens with owned-row count.
- Q3 Recent-access prunes only when the upsert inserted (RETURNING xmax=0)
  — a re-access can't grow the set. 0.567 -> 0.324 ms (1.75x).
- L1 Locale supported-codes precomputed once vs rebuilt per anonymous
  request. 616 -> 17.3 ns (35.7x), 18 -> 1 allocs.
- H1 Duplicate /api TraceLayer removed (global stack already wraps it).
  1.86 -> 1.42 us/request, -6 allocs.
- H2 client_ip span field: borrow-only ClientIpDisplay vs owned String.
  187 -> 173 ns, -1 alloc.

Not shipped (discipline): the "media hooks read the blob 3x" lead was a
correctness bug, not a perf dup — the raw-path metadata/faces readers
resolve only for local+unencrypted+single-chunk blobs and silently produce
nothing otherwise. Flagged for maintainers; routing through read_blob_bytes
is a correctness fix (perf-neutral-to-negative), not a benchmark-gated
perf change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
2026-07-19 08:17:48 +00:00
Claude 50eca0627f perf: round 12 — auth write-path narrowing, fused quota gate, moka blob-cache index, media single-read, sized listing JSON
Benchmark-gated round (benches/ROUND12.md; every change ships with a
BEFORE/AFTER harness + equivalence gates, one candidate rejected by its
own bench):

DB / query shapes (bench_round12_queries):
- NC sharee search: username-only projection instead of the 21-column row
  (incl. the <=512 KiB avatar) per match, + gin_trgm_ops indexes on
  auth.users for the leading-wildcard ILIKE (4.98x; 54.7x with index).
- Password login: delete the redundant full-row update_user — create_session
  already stamps last_login_at in its own txn (4.45x per login).
- Email-verified stamp: narrow conditional UPDATE (8.9x); OIDC repeat login
  now compares profile state in memory and issues ZERO queries when nothing
  changed (was: full 17-column rewrite per login).
- Refresh rotation: revoke+insert+stamp fused into one transaction via new
  rotate_session port method (1.18x).
- WOPI CheckFileInfo / authorize_wopi_access: require(Read) + get_file +
  check(Update) overlapped with tokio::join!, original result precedence
  (cold 1.34x).
- Upload quota gate: user-envelope + drive-cap checks fused into ONE
  round-trip (check_upload_quotas) — the NC chunked PUT pays this per
  chunk (1.81x, 2 -> 1 queries/chunk); shared verdict evaluators keep
  error shapes byte-identical.

CPU / allocs (bench_round12_micro):
- sized_json: pre-sized listing serialization replacing axum Json's 128 B
  seed + doubling-realloc chain on files/folder-resources/photos/search
  responses (1.40x, 13 -> 2 allocs per 500-row page; byte-identical).
- Security headers: 4 SetResponseHeaderLayer folded into the CSP middleware
  pass (5 layers -> 1; 1.43x per request, -26 allocs; header set gated
  byte-identical incl. 304s).
- Media capture-metadata: single-read extraction — nom-exif now parses the
  buffer kamadak already read (zero-copy Bytes) and videos open once with a
  kind() dispatch; per-image opens 2-3 -> 1 (1.44x warm geomean, 1.6-3.2x
  cold cache; extraction outputs gated identical incl. the MIME-mislabel
  track fallback).
- Chunked-upload session ops: owner gate folded into the operation's own
  DashMap lookup + stack-encoded uuid compare (5 -> 3 lookups, -2 allocs,
  1.28x per chunk).

Blob cache (bench_blob_cache_index + round-3 regression guard):
- CachedBlobBackend index: tokio::sync::Mutex<LruCache> -> moka::sync::Cache
  with byte weigher. The mutex serialized every cached chunk read and scaled
  NEGATIVELY (2.08 -> 1.07 Mops/s from 1 -> 2 readers); moka probes are
  lock-free (2.17x at K=2). Byte budget now enforced by moka (manual
  current_size + collect_evictions machinery deleted); eviction listener
  unlinks size-evicted files only (Replaced entries keep their file —
  gated). Single-flight miss gate unchanged (16 concurrent misses -> 1
  fetch re-verified via the round-3 harness).
- put_blob now populates the cache BEFORE the inner backend consumes the
  source file (the old order failed 100% of the time — local renames,
  S3/Azure delete the source — so the first read after a whole-file put
  re-downloaded from the remote); inner-put failure invalidates the entry.

Frontend (vitest gates):
- List-view thumbnails request the 150px icon rendition instead of 400px
  preview into a 40px slot (~7.1x fewer pixels, ~4-5x fewer bytes per
  thumbnail across list views); grid keeps preview.

Rejected by its own bench (kept as evidence in bench_round12_micro §2):
- Single-pass compression predicate: the monomorphized And-chain already
  costs ~4.6 ns / 0 allocs total; the fused node measured within noise.

New migration: 20260719000000_users_search_trgm.sql (trgm indexes).
Deferred with prepared design: grouped file/grid view virtualization
(single-VirtualRows flatten, the photos pattern) — next round's headline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
2026-07-19 01:32:00 +00:00
Claude f621c96c63 Merge origin/main (ResourceList item-based refactor #612, IdP auto-redirect #593) into round 11
Conflict resolution re-applies the round-11 SPA optimizations onto the
refactored item-based ResourceList (which independently converged on the
favoriteIds-prop star — S2 is now upstream's shape):

- ResourceList.selectedItems: upstream reintroduced the full O(N)
  items.filter per selection toggle; replaced with the round-11
  id-index projection — O(k·log k), item order preserved
  (benches/ROUND11.md §S1)
- favorites/recent pages: upstream's rewrite kept the host-side
  selectedItems shadow + unused selectedIds mirror and ignored the
  batchToolbar snippet param; re-applied the param-consuming shape and
  deleted the shadows

Validation on the merged tree: clippy --all-features --all-targets
-D warnings clean; cargo test --workspace 524 passed; frontend npm run
check 0 errors; vitest 310 passed (upstream's new tests + round-11 gates)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
2026-07-18 23:15:10 +00:00
Claude b01633791b perf: round 11 finale — benchmark verdicts, geo min(uuid) rollback, ROUND11.md
- benches/ROUND11.md: final BEFORE/AFTER numbers for all 21 micro
  sections, the 5 query sections, the 4 log-writer arms, the SPA gates,
  and the two cross-round regression guards (bench_row_path 2.52x,
  bench_dto_map — both byte-identical)
- ROLLBACK (gate-caught): min(fm.file_id)::text geo-cluster cast —
  PostgreSQL has no min(uuid) aggregate; the bench section now reproduces
  the rejection and the per-row-cast original stays
- REJECTED (bench-measured): tracing-appender non-blocking writer —
  slower than sync on fast sinks (1.41M vs 0.99M ev/s, worse tail) and
  the slow-sink drain gate showed shutdown tail-loss risk for the audit
  channel; dep moved to dev-dependencies (harness only)
- RateLimiter final form: lock-free get + insert (8.0 → 6.0 allocs, wall
  neutral; and_upsert_with variant rejected at 2 365 ns)
- fix: TrashedItemParts insertion had stolen TrashedItem's derive line
  (caught by clippy --all-targets)
- grant_role enum values corrected in the queries bench

Validation: cargo fmt + clippy --all-features --all-targets -D warnings
clean; cargo test --workspace 524 passed / 0 failed; frontend npm run
check 0 errors + vitest 306 passed (1 pre-existing round-7 wall-clock
gate flaked only under concurrent Rust-build CPU contention; passes in
isolation)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
2026-07-18 22:46:20 +00:00
Claude 221c1f31b0 perf: round 11 — StoragePath joined-only, classifier fusion, memoized bodies, query-shape pack, SPA fine-grained stars
Backend (each change benchmark-gated with BEFORE replicas + equivalence
gates; see examples/bench_round11_micro.rs, bench_round11_queries.rs,
bench_log_writer.rs and benches/ROUND11.md — final numbers land in the
follow-up doc commit):

- StoragePath re-representation: single canonical joined String, segments
  derived on demand; File/Folder drop the duplicated path_string field
  (4000→1000 allocs per 500-row listing page)
- Display classifier fusion: classify_display shares one stack-lowered
  extension across the three decision trees; call sites in FileDto,
  folder/favorites/recent handlers, trash, path-resolver (+ interning
  where Arc::from was still used)
- /status.php and /openapi.json memoized into OnceLock<Bytes> (openapi
  rebuilt a 171 KiB spec per request: 2.8 ms → 18 ns)
- NC upload-session PROPFIND: write! + pre-sized body + stack RFC2822
  dates (2.3-2.6x, 2582→772 allocs at 256 chunks)
- REST download: dead FileDto clone removed (capture mime/size + move)
- CalendarEventDto/TrashedItem into_parts moves (11 KiB ical_data memcpy
  gone per CalDAV row); CardDAV getlastmodified stack render
- 4xx path: borrowed ErrorResponse serialize, ErrorKind::as_str,
  not_found/already_exists clone kill
- vCard emit via write!; search page moved out with into_iter skip/take;
  content-hit UUIDs parsed once; group last-user check via HashSet
- RateLimiter: lock-free get + insert (and_upsert_with variant REJECTED
  by benchmark); CSRF token borrow-compare + borrowed cookie extraction
- Thumbnail/preview ETags built from as_str (Debug-identical bytes)
- Encrypted backend: encrypt_in_place_detached single-buffer write path,
  chunk-sized reserve in collect_stream; retry labels made lazy
- PG: deferred upload registration 3→1 round-trips (persist_file CTE
  template); direct_grant_cache for Calendar/AddressBook/Playlist authz
  (single-flight + set_role/clear_role invalidation); expand_user
  tokio::join!; geo clusters min(uuid)::text; recluster face assignment
  batched into one UNNEST update
- People recluster cosine: norms precomputed once (bit-identical gate)
- NC capabilities poll logs demoted to debug; tracing-appender dep added
  for the log-writer benchmark

Frontend:
- ResourceList.selectedEntries O(N)-per-toggle → id-index projection
  O(k log k); favorites/recent consume the batchToolbar snippet param and
  drop their duplicate filter + dead selectedIds mirror
- Recent: star state via new favoriteIds prop — a star click no longer
  rebuilds all N entries
- admin timeAgo >30d fallback uses the cached Intl.DateTimeFormat
- vitest gates in src/lib/components/round11.bench.test.ts

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
2026-07-18 22:02:00 +00:00
Dionisio Pozo ffabacbb5b Merge pull request #593 from swissiety/idp-auto-redirect
feat: if the only auth method configured is SSO login: auto redirect to SSO Provider
2026-07-18 22:49:25 +02:00
Claude c51af68432 perf: round 10 — auth alloc purge, parent-herd batching, query-shape pack, NC 304s
Benchmark-gated (benches/ROUND10.md; every change carries a BEFORE/AFTER
harness with equivalence/safety gates — two designs were rejected or
rewritten by their own benches before adoption):

- Auth hot path: TokenClaims/CurrentUser display fields to Arc<str>, role
  to inline SmolStr end-to-end (Bearer, cookie, Basic-auth cache) — 4→1
  allocs per authenticated request, 3→0 per warm DAV request; JWT
  Encoding/Decoding/Validation built once.
- Cold shared-album herd: leader-inline parent batching in PgAclEngine
  (+ cascade try_get_with single-flight) — 100→2 parent queries per
  100-thumb cold herd, herd wall 1.9x, sequential + warm paths unchanged,
  all ROUND8/9 safety gates plus new herd-equivalence gates.
- Query-shape pack: share download double-fetch 2→1 (2.18x), contact-group
  COUNT(*) 14.9x, save_faces UNNEST 3.9x, playlist reorder UNNEST 63.7x
  (now atomic), search files∥folders join! 1.45x, move drive-lookup join!
  2.14x, trash partial (drive_id, trashed_at) indexes, CalDAV event-gate
  narrow read, favorites/recents binary-decode port, dead count_files
  removed.
- NC surface: preview + avatar honour If-None-Match (e2e: 5 KB and 197 KB
  → 0 bytes per revalidation), avatar WebP→PNG transcode memoised,
  PROPFIND/trashbin integer+date emits on stack formatters, folder-header
  enrichment join!, chunk-PUT retry stat folded into create_new open.
- common::fmt integer rendering rewritten on the std 2-digit LUT after the
  round's own bench caught the div-loop losing to to_string (16.1 ns vs
  22.5; speeds every prior-round call site).
- Micro-pack: WebDAV scope probe borrow-only, ShareService base_url
  snapshot, cookie_secure OnceLock, Arc'd AES-GCM cipher, stack request-id,
  tantivy analyzer clone dropped.
- SPA: search stale-guard + AbortController (10→1 completed round-trips,
  stale-clobber gone), getFolder in-flight dedup, gridColumns matchMedia
  hoist (10k→0 style reads).

Backend: cargo fmt + clippy -D warnings clean, 524 tests green.
Frontend: npm run check clean, 301 vitest green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DdM7V7M3QPW7HEHg3gLov
2026-07-18 20:33:50 +00:00
Markus Schmidt 7f04236c4b Merge branch 'main' into idp-auto-redirect 2026-07-18 20:16:54 +02:00
Claude fdf445d2b0 perf: round 9 — decorator PUT reactivation, session/search/dedup alloc purges, PROPFIND join!, folder-level cascade
Benchmark-gated round (benches/ROUND9.md): every change carries a
BEFORE/AFTER bench with equivalence/safety gates; verdicts below are from
the committed harnesses on 4 cores / local PG 16.

Backend:
- Blob decorators (Retry/Cached) now forward put_blob_from_bytes_unsynced
  + sync_blobs — the trait default had silently reinstated HEAD-before-PUT
  per chunk on decorated remote stacks, undoing ROUND3 §8. Full production
  stack: 500 probes -> 0, 1.9x wall at 10 ms RTT (bench_s3_put §3).
- NC PROPFIND per-page enrichment triple (favorites / oc:fileid / dead
  props) overlapped with tokio::join!: 2.07x local, 2.86x at 5 ms RTT
  (bench_nc_enrich_join, injected-latency decide-by-bench).
- Search enrichment consumes its DTOs and carries the interned Arc<str>
  display fields end-to-end (SearchFileResultDto type change, OpenAPI
  shape preserved): enrich_file 2.0x, 11.6 -> 2.2 allocs/row; the NC
  REPORT conversion stops re-running all three classifiers per row
  (bench_search_enrich).
- NC session Arc end-to-end: SharedNcSession extractor (8 -> 0 allocs),
  Arc<FolderDto> chroot cache (4 -> 0/hit), single shared Arc<CurrentUser>
  + lazy span render (11 -> 6/build) (bench_nc_session).
- Storage micro-pack: atomic create_new chunk writes (2.1x fresh),
  stream_chunks over the manifest Arc (4097 -> 0 allocs/read incl. the
  Range path), manifest single-flight (herd 64 -> 1 loads), hex_lower for
  chunk Content-MD5 (18 -> 1 allocs) (bench_storage_micro).
- OCS capabilities memoized into OnceLock<[Bytes;2]>: 237x, 102 -> 0
  allocs/poll, byte-identical (bench_capabilities_static).
- Drive::is_empty COUNT(*) sum -> EXISTS: 34.4x on a 100k-file drive
  (bench_drive_is_empty).
- favorites/recents row-map ROUND7 port: path/name/blob_hash moved,
  -2.75 allocs/row (bench_resource_row_map §2).
- Folder rows decode binary UUIDs (ROUND6 §10 port): 1.03-1.07x page
  fetch, honest verdict incl. one noise-band wash documented
  (bench_folder_uuid_decode).
- Authz: file cascade decision decomposed into memoized folder-level
  decision + direct-grant lookup (ROUND8 deferred item). Cold shared-album
  first view 592 -> 418 µs/thumb; warm path unchanged; safety gates incl.
  new direct-grant sibling isolation, revoke-flush re-verified, full
  integration authz suite green (bench_thumbnail_cascade_cache).

Frontend (vitest gates committed beside the code):
- resolveLabel/resolveRecipient O(directory) scan -> id-keyed Map: 13.9x
  (recipients.bench.test.ts).
- ResourceList selection-prune effect skips when nothing is selected
  (100 -> 0 Set builds per drain) and the photos timeline reads a
  listener-fed mobile flag instead of matchMedia per recompute
  (listDerives.bench.test.ts).

Verification: cargo fmt + clippy --all-features --all-targets -D warnings
clean; 524 unit + 554 integration (--cfg integration_tests) tests pass;
frontend npm run check clean with 293 vitest tests green.

Deferred with rationale in ROUND9.md: CalDAV authz-before-fetch reorder
(maintainer sign-off), per-page batched parent resolution, JWT-claims
Arc<str>, batch_operations signature widening.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XDc9VtXvskJ6dnMRraSndn
2026-07-18 16:12:04 +00:00
Dionisio Pozo 2317d594e3 Merge pull request #610 from EdouardVanbelle/security/grants2 2026-07-18 16:07:17 +02:00
Claude 79b94126be perf(authz): round 8 — cache the File/Folder grant-cascade decision for shared-album thumbnails
get_thumbnail_impl runs require_permission(Read) on every request. For a drive
member that's a drive_role_cache hit, but a shared-album recipient — granted a
folder (the album), not drive membership — fails the drive-role precheck and
falls through to file_cascade_grant_exists (a role_grants ⋈ folders lpath
ancestor query), once per file. Browsers revalidate immutable thumbnails
constantly, so the same (recipient, file, Read) decision was recomputed on
every thumbnail of every view — ~100 grant queries per 100-photo album per
navigate-away-and-back.

New cascade_grant_cache ((Subject, Resource, Permission) → bool, 30 s TTL)
memoises that decision. The check is NEVER skipped — the ordering is unchanged,
authz still runs on every request; only the result is cached, and only after
the drive-role precheck fails (so a later drive grant can't be shadowed by a
stale entry). Invalidation mirrors drive_role_cache's convention: explicit
invalidate_all on every File/Folder set_role/clear_role (immediate revoke on
the direct share path), 30 s TTL for the indirect paths (group membership,
moves, expiry) "rather than a deep invalidation tree".

Bench (bench_thumbnail_cascade_cache) with hard safety gates — recipient
allowed, outsider denied, and a clear_role revoke denies the very next check
(proving the grant-write flush): 100-photo album revalidation 2576 → 2.70
µs/thumb (~950x), 257.6 → 0.27 ms/view. Validated against the full
--cfg integration_tests authz suite (554 tests) + 524 workspace tests, clippy
-D warnings clean.

Deliberately not done: moving authz after the 304/cache short-circuit (a
security-posture change — a revoked user could serve cached thumbnails). With
the decision cached, the authz on the 304 path is now a memory hit, so the
"zero DB work on a 304" intent is restored without weakening the check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
2026-07-18 14:03:43 +00:00