75a123ae6c7cb314195f60a0ce98ae831dc63eee
630 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dionisio Pozo
|
86268049ff |
Merge pull request #641 from EdouardVanbelle/refactor/front-resource-list
feat(fileDto, folderDto): add is_favorite + is_shared |
||
|
DioCrafts
|
d66956824c | Optimize storage, GC, and upload hot paths | ||
|
Edouard Vanbelle
|
d9987782c4 |
feat(fileDto, folderDto): add is_favorite + is_shared
- provide is_shared and is_favorite information in DTO, information propagated as badge/buttons per items
regarding performances I try to be minimalis on SQL to prevent any perf regression
- remove old set of sharedids and favoriteids (was not functionnal anymore)
- fix date picker (no past selection) in grant
- fix contextmenu close on /files section
|
||
|
Bradley Nelson
|
9f4bc54113 |
Merge upstream/main into feat/external-file-mounts
Resolve conflicts between the external-file-mounts feature and upstream's D5/D7 refactor (per-file provenance, keyset pagination, cross-drive move gates, resource-access hook, folder-cascade lifecycle hook). Key resolutions: - FolderService::new now takes (repo, authz, file_lifecycle, mount_router); all callers + DI updated. - FileRetrievalService / FileManagementService keep both the mount_router and the new resource_access_hook / drive_repo / storage_usage wiring. - list_files_batch_with_perms: adapt the mount branch from offset- to keyset (after_name) pagination, mirroring paginate_mount_entries. - download_file_impl: keep upstream's &HeaderMap + `impl IntoResponse + use<>` signature, retain the mount-download branch. - Mount DTOs: the retired `owner_id` field maps onto created_by/updated_by (the mount owner) — the fields the frontend now uses for owner display. - admin/+page.svelte: keep upstream's user-delete modal + the 'mounts' tab. - Bump memmap2 0.9.10 -> 0.9.11 (RUSTSEC critical advisory fix) and regenerate Cargo.lock against the merged Cargo.toml. |
||
|
Dionisio Pozo
|
67fe944c2a | Merge pull request #596 from swissiety/rfc-5789-http-patch | ||
|
Claude
|
8e55caa8a9 |
perf(round29): cache-serve borrow-probe, NC REPORT href buffer, auth per-req allocs, DB over-fetch
Seven behaviour-preserving allocation / copy / bandwidth cuts, each behind a counting-allocator BEFORE/AFTER gate that exit(1)s unless AFTER allocates strictly fewer than BEFORE (benches/ROUND29.md, examples/bench_round29_micro.rs). - [B] Content-cache serve fast path (optimized_inner Tier 1 + get_file_range_preloaded — the video-scrub hot path): probe the cache with a borrow first and build the owned get_or_load args (quoted-etag / key / id Strings) only on a miss, instead of allocating them before every probe and discarding them on a hit. 6 -> 0 allocs per cache hit. Splits get_or_load into get + load_and_cache so the miss path is not re-probed and the hit/miss stat counters stay byte-identical. Also drops the unconditional content_hash/name clones that ran for the >=10 MB streaming tier that used neither. - [A] NextCloud REPORT emit loops: per-row href String (and format! per folder row) -> one reused href_buf via nc_href_into / nc_collection_href_into with the URL-encoded user computed once per page. 1497 fewer allocs on a 500-row page. - [C] read_full: a single-frame blob is returned zero-copy instead of a second whole-payload memcpy into a fresh BytesMut; multi-frame path unchanged. - [D] login-lockout key: to_lowercase()+format! -> one pre-sized ASCII buffer (non-ASCII keeps str::to_lowercase). 3 -> 1 alloc/req, byte-identical key. - [E] NC composite-username parse: owned clone/to_string -> &str borrow of the already-owned raw_username. 1 -> 0 alloc on the common no-marker path. - [F] get_contacts_in_group: stop SELECTing the discarded multi-KB vcard column (the live method ROUND25 §Q2 missed; ContactDto has no vcard field). - [G] count_admin_users: add count_users_by_role -> scalar COUNT(*) instead of hydrating every admin's full row (incl. up-to-512 KiB avatar + ui_preferences JSONB) only to .len() it, on a bootstrap-polled status endpoint. All seven gates pass; cargo fmt --check and cargo clippy --all-features --all-targets -D warnings clean. §F/§G additionally validated against a live PostgreSQL 16 with the full migration set (query validity, result equivalence, 600000 -> 8 byte wire delta on the admin count). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LhpDZxSQTAGnAqCHUdtG5N |
||
|
Claude
|
8c936de50d |
perf(round27): NextCloud PROPFIND oc:id per-row buffer, contact JSONB write direct-serialize
Two behaviour-preserving allocation cuts (benches/ROUND27.md), each with a
counting-allocator BEFORE/AFTER gate that rolls back if AFTER does not allocate
fewer than BEFORE:
- H1 NextCloud PROPFIND: the streaming page loops built oc:id as a fresh String
per child (format_oc_id -> format!("{:08}{}", id, instance)). Add
format_oc_id_into(&mut buf, id, svc) and compute into one oc_buf reused across
the page (next to the existing href buffer) — 1 String/row -> 0. 998->0
per-row allocs on a 500-row page, 2.16x wall. The write fns still take
Option<&str>, so no signature change; oc:id bytes identical. Scoped to the two
PROPFIND page loops (the hot directory-listing path); REPORT/trashbin deferred.
- P2 contact create/update: bind sqlx::types::Json(&dtos) (Encode runs to_writer
straight into the JSONB buffer) instead of serde_json::to_value(&dtos) + bind,
which built a throwaway Value DOM per JSONB column. Write-side twin of ROUND23
J1. 21->2 allocs, 4.68x wall for a 3-entry column. Behaviour-preserving:
to_value sorts keys and direct serialize keeps struct order, but Postgres
normalizes JSONB key order so the stored value is identical (verified via psql:
'{...alpha...}'::jsonb = '{...struct...}'::jsonb -> t), and reads decode by
field name; the etag comes from the domain entity, not the stored JSONB.
Adds bench_round27_micro. Verified: cargo fmt clean, cargo clippy --features
bench -D warnings clean (real exit), cargo test --lib --features bench = 529
passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
|
||
|
Claude
|
5b2bb8f883 |
perf(round26): drive-policy JSONB decode, CachedBlobBackend shard-dir pre-create, delta-upload foldhash
Three benchmark-gated optimizations from the ROUND25 backlog (benches/ROUND26.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- P1 drive_pg_repository policy reads: decode d.policies through
sqlx::types::Json<DrivePolicies> (one from_slice over the raw JSONB bytes)
instead of a throwaway serde_json::Value DOM + DrivePolicies::from_value —
6 -> 0 allocs/read, 2.77x wall. A shared policies_from_row helper preserves
the lenient unwrap_or_default fallback (malformed bag -> all-false).
- D1 CachedBlobBackend: pre-create the 256 {00..ff} shard dirs at initialize()
(mirroring LocalBlobBackend, reusing HEX_PREFIXES) and drop the redundant
per-write create_dir_all on already-existing shards — ~45us + a blocking-pool
dispatch removed per cache write on cached-remote deployments.
- G1 delta-upload have/need hash sets (distinct_hashes, authorize_chunk_download):
SipHash -> foldhash::quality::RandomState — a fast hasher that stays
DoS-resistant via a per-instance random seed, the required property for the
attacker-controlled 64-hex client hashes — 2.37x wall on a 40k-hash
negotiation. foldhash was already in the lockfile transitively (hashbrown).
Tested and REVERTED (kept as-is): moving the moka eviction unlink off the reactor
via spawn_blocking. The benchmark refuted it — on the local cache dir the
spawn_blocking dispatch (~20us) costs more than the inline unlink (~7us) it would
replace. See ROUND26.md §D2.
Adds bench_round26_{micro,diskio,hasher} (counting allocator / async wall / wall).
Verified: cargo fmt clean, cargo clippy --features bench -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
|
||
|
Claude
|
e8e4ef4b15 |
perf(round25): in-place encrypted decrypt, dedup hash move, dead folder-Query, playlist N+1 fold, contact vcard over-fetch
Five benchmark-gated optimizations from a fresh six-way audit (benches/ROUND25.md), each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE: - M1 EncryptedBlobBackend::decrypt_bytes: replace split_off (a fresh Vec + full ciphertext memcpy on every decrypted chunk, contradicting its own "in place" doc) with in-place detached decrypt + a zero-copy Bytes::slice past the nonce. Peak RAM per read drops from ~2x to ~1x the payload (-262KB/op at 256KiB; scales with blob size). Plaintext byte-identical; tamper/wrong-key tests pass. - M2 delta commit: move-unzip the owned chunk list instead of a third per-occurrence hash clone (-4000 allocs on a 4000-chunk commit). - M3 folder ZIP download: drop the dead Query<HashMap> extractor it never read (byte-identical response; 5->0 allocs/request). - Q1 public-playlist listing: fold the per-playlist COUNT(*) N+1 into one LEFT JOIN ... GROUP BY via a new inherent repo method (101 -> 1 round-trips, 36x wall on a 100-playlist page). - Q2 contact REST listings (paginated/search/by-group): stop over-fetching the multi-KB vcard TEXT the ContactDto discards, via a shared lite row mapper and narrowed SELECTs (6.4x wall on 1000 contacts with 8KiB vcards). The whole-book vCard export and CardDAV sync paths keep the column. Adds bench_round25_micro (counting allocator tracking count+bytes) and bench_round25_queries (live Postgres), both with equivalence gates and a rollback exit(1). Verified: cargo fmt clean, cargo clippy -D warnings clean, cargo test --lib --features bench = 529 passed / 0 failed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT |
||
|
Dionisio Pozo
|
0d82632f92 |
Merge pull request #629 from EdouardVanbelle/refactor/front-resource-list
Refactor(front) restore legacy frontend features |
||
|
Edouard Vanbelle
|
931e27d09c | fix(resources): wire missing created_by and updated_by | ||
|
Claude
|
1ec7030cc7 |
perf: round 23 — Postgres query-shape pass: typed JSONB decode, drive-policy borrow-deserialize, user-profile join!, subject-group CTE reuse, dedup unzip
Benchmark-gated, same rule as ROUND2-22: BEFORE/AFTER with a value-equivalence gate and rollback-on-regression. Two harnesses — bench_round23_micro (no Postgres; deterministic allocation gate) and bench_round23_queries (live Postgres; p50 latency + strict equivalence gate against seeded fixtures). See benches/ROUND23.md. - J1: contact_pg_repository::row_to_contact (+ the inlined contact_group sibling) decode the 3 JSONB columns via sqlx::types::Json<T> (one from_slice pass) instead of row.get::<serde_json::Value> + from_value (a throwaway Value DOM per column, walked a second time). Per contact row of every list / multiget / CardDAV sync. Micro 84 -> 33 allocs/op (2.15x); PG 3794 -> 2360 ns/contact (1.61x) on 500 real rows. - J2: DrivePolicies::from_value deserializes straight from the borrow (T::deserialize(&Value)) instead of from_value(value.clone()) — dropping the full-DOM clone on every drive-policy read (move/copy, share, grant); one-line body change, all 7 callers unchanged. Micro 5 -> 0 allocs/op (11.51x). - P1: get_user_profile overlaps the two independent caller+target reads with tokio::join! (self-case still a single fetch; caller-error precedence preserved via caller_res? first) instead of two serial round-trips. PG 577 -> 312 us/call (1.85x). - G1: subject_group remove_member computes the child's transitive-user recursive CTE once and reuses it for both the would-empty pre-check and the cache invalidation, instead of running the identical CTE twice (the edge delete is above the child, so its descendants can't change). PG 829 -> 412 us/removal (2.01x). - U1: dedup_service (store_loose_chunks final registration + the ingest run_rollback) reshapes the owned, dead-after Vec<(String,i64)> via into_iter().unzip() instead of cloning every 64-byte hash for the sync_blobs(&[String]) + UNNEST bind. Micro 256 -> 0 hash clones. Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0 failed. The PG benches run against a local PostgreSQL 16 (schema applied from migrations/); every equivalence gate passes. The download_zip per-item N+1 (the audit's highest raw-latency candidate) is deferred to a dedicated pass: its fix moves the sole authorization inside the stream call, so it needs an AuthZ-ordering + anti-enumeration proof, not a perf banner. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo |
||
|
Claude
|
77f13ac643 |
perf: round 21 — CalDAV/CardDAV row-mapper pre-size, dedup hash-bind & digest-key dedup, CardDAV etag/BDAY emit, NC trashbin content-type
Round 21 of the benchmark-gated perf sweep. Six behaviour-preserving, allocation-reducing changes, each with a BEFORE/AFTER counting-allocator section in examples/bench_round21_micro.rs and a byte/-value equivalence gate; all six pass their deterministic alloc gate (a non-winning AFTER exits 1 = rollback). - R1: pre-size the 16 CalDAV/CardDAV row-mapper Vecs (+1 HashMap) with Vec::with_capacity(rows.len()) — the ROUND20 §I1 file-side pattern extended to the calendar/contact repos it deferred. 7 → 1 allocs/op. - R2: settle_batch binds a borrowed Vec<&str> instead of cloning every chunk hash into a Vec<String> (sqlx encodes &[&str] as text[] identically; favorites_pg_repository.rs:271 precedent). 33 → 1 allocs/op, 39x wall. - R3: store_loose_chunks keys its intra-request dedup set on the raw [u8;32] BLAKE3 digest and moves the hex on a duplicate (the ROUND17 §D2 pattern applied to the delta-upload sibling). 401 → 209 allocs/op. - R4: CardDAV getetag emits borrowed pre-escaped " quotes via a shared write_quoted_etag helper (ROUND20 §C1 pattern, all 4 CardDAV etag sites). 3 → 0 allocs/op. - R5: BDAY stamped via the new fmt::compact_date stack renderer instead of chrono's strftime interpreter (chrono fallback out of the 4-digit-year range; byte-identical, unit-tested vs chrono). 2 → 0 allocs/op, 10.5x wall. - R6: NC trashbin folder content-type via Cow::Borrowed instead of .to_string() on the constant (ROUND16 §M1 pattern). 1 → 0 allocs/op. See benches/ROUND21.md for the full write-up and the deferred-items list (HeaderMap-clone hot handlers, Query→typed-struct, WebDAV dead-props HashSet, and others surfaced by the audit that want their own validated pass). Validated: cargo fmt, cargo clippy --features bench --all-targets -D warnings, cargo test --lib --features test_utils (529 passed). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015gHVq5Wy2TzdWeSqtEmK6m |
||
|
Claude
|
867e1fe259 |
perf: round 20 — iCal/vCard parse allocs, owned-DTO moves, Result-collect pre-size, NC etag/favorites emit
Benchmark-gated (benches/ROUND20.md), same rule as rounds 2-19: every change ships with a BEFORE/AFTER counting-allocator micro-benchmark and a byte-value equivalence gate; a non-winning AFTER is rolled back (never applied). The rollback rule is encoded in the harness (GATE FAIL exit). All 8 sections pass. Reproduce: cargo run --release --features bench --example bench_round20_micro - A1 CalendarEvent iCal parse: replace the throwaway per-property HashMap<String,Vec<String>> (DTSTART/DTEND/RECURRENCE-ID) with a direct VALUE=DATE scan; prop_with_params kept #[cfg(test)] (6->2 allocs/event, 4.2x) - A2 UserDto::from: add User::into_parts and MOVE image (<=512 KiB data URI) + ui_preferences JSON instead of cloning on every /api/auth/me (27->14 allocs) - A3 parse_vcard: drop the per-line to_ascii_uppercase copy + the lines Vec; promote ascii_ci_contains to common::text and share it (8->1 allocs/contact) - A4 Calendar/AddressBook DTO: into_parts move incl. custom_properties map (18->10) - I1 file-listing repos: collect::<Result<Vec>>() size-hints to 0 and grows from capacity 0; pre-size with Vec::with_capacity (8->1 container reallocs, 4 sites) - I4 plaintext_stream: lazy emit iterator instead of eager Vec collect (43x wall) - C1 NC write_etag_element: borrowed pre-escaped quote events, no owned quoted String/escape re-alloc; byte-identical output (3->0 allocs/PROPFIND row) - C3 NC favorites REPORT: map.remove() move instead of get().clone() (~7 allocs/fav) Deferred (documented in ROUND20.md): NC oc:id/trashbin buffer reuse, I1 sibling CardDAV/CalDAV listing paths, Contact JSONB Json<Vec<_>> decode, dedup settle_batch &str bind, and a fast DoS-resistant hasher for hot trusted-key maps (needs a dependency decision). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JsJjcVX9RoN96DMa35Wqzd |
||
|
Claude
|
c207e66f4b |
perf: round 17 — dedup ingest/verify hash-clone purge, CardDAV vCard TYPE tokens
Targets the content-addressable dedup write path (the ROUND15-deferred "dedup_service hash-String re-allocations") from both ends — the streaming ingest loop and the delta-commit verification read — plus a CardDAV vCard micro-cut. Every change is benchmark-gated with a hard rollback rule; no PostgreSQL needed for any arm (benches/ROUND17.md). Backend (counting-allocator, examples/bench_round17_micro.rs): - D2 chunk-ingest (store_from_stream, the hottest write path — every chunk of every upload): the 64-char hex hash String was allocated 3x per chunk (to_hex + chunk_hashes clone + session_seen insert-clone, the last dropped on a duplicate). The intra-upload dedup set now keys on the raw 32-byte BLAKE3 digest ([u8;32], Copy, no heap) and the manifest push is branch-split so a duplicate moves the hex in: 3 -> 2 allocs/new chunk, 3 -> 1/duplicate. Measured 214 -> 149 allocs/op (1.14x wall) on a 64-chunk 1-in-2-dup batch; smaller/faster set too (32B inline keys vs 64B heap Strings). - D1 hash_chunk_sequence (delta-commit verification): took chunks by &[(String,u64)] and fed the backend stream with iter().cloned(), re-cloning every chunk hash a second time on top of the owned Vec the caller already built. Take the Vec by value + into_iter(): 65 -> 0 internal allocs/op, ~2.5us of clone work removed per verify. - V1 vCard TYPE tokens (contact_to_vcard + generate_vcard, 5 sites): each EMAIL/TEL/ADR TYPE= param used ty.to_uppercase() — a throw-away String per token per contact. New shared fmt::push_upper writes the upper-cased chars straight into the buffer (byte-identical to str::to_uppercase, unit-tested): 13 -> 5 allocs/op, 1.19x wall. Gates: each section asserts byte/-value equivalence (D2 the ordered manifest + sizes + write-set; D1 the removed clone is a pure copy; V1 the full vCard) and exits non-zero if an AFTER arm fails to reduce allocations. push_upper is unit-tested byte-equal to str::to_uppercase (fmt::tests). Verified end-to-end: cargo fmt clean, clippy --release --all-targets --features bench -D warnings clean, and the harness prints GATE PASS against the built release lib. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XMmt7vNETYUbEG3Hc17LDx |
||
|
Dionisio Pozo
|
a09569b5c2 |
Merge pull request #625 from AtalayaLabs/claude/performance-optimization-analysis-raoezl
Round 16: incremental lanes/contextMap builders & alloc cuts |
||
|
Claude
|
955f4a7b9f |
perf: round 16 — shares-lane & contextMap incremental builders, folder/href/disposition/preview alloc cuts
Finishes the route-level half of the O(N²/page) grouped-listing class ROUND15 fixed inside ResourceList, plus a backend CPU/alloc micro-pack. Every change is benchmark-gated with a hard rollback rule; no PostgreSQL needed for any arm (benches/ROUND16.md). Frontend (vitest): - F1 "My shares" lanes: the `lanes` $derived.by re-bucketed the whole accumulated grant list on every page and every grant edit. SharedLanesBuilder re-emits only the fresh page (fan-out + first-appearance header), reusing untouched lanes' array refs. 25.5x fewer emit calls, 8.8x wall, O(N²/page)→O(N). - F2 contextMap (trash/recent/favorites/shared-with-me): each rebuilt a fresh N-key Map, re-hashing every accumulated id, per page. primeContextPage holds a persistent SvelteMap primed per page (the shipped favoriteIds shape). 25.5x fewer entry calls, 7.2x wall. - Extracted the shared O(1) append test (isAppendExtension); F1's gate re-covers it. Backend (counting-allocator): - M1 folder display constants Arc::from -> intern_display (3 sites): 3 -> 0 allocs/row. - M2 build_content_disposition (every download + Range seek): 3 -> 1 alloc, 6x, 2.67x wall. - M3 nc_href (every NC PROPFIND/REPORT href): Vec+join+format -> one pre-sized buffer, keeping urlencoding::encode (byte-identical). 38 -> 27 allocs/op. - M4 NC preview fileId: collect-then-parse -> borrow-slice parse. 4 -> 0 allocs. Gates: sharedLanes/listContext.bench.test.ts, examples/bench_round16_micro.rs (GATE PASS all sections). Frontend: vitest 331 pass, svelte-check clean. Backend: clippy -D warnings clean, 524 lib tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0193NjactJVqfU32gxeJDj8m |
||
|
Edouard Vanbelle
|
05dfda9aa3 |
feat(drive): add quota update handler
per today: admin only can update quota
shared drive can have quota updated (personal drive's quota belong to user's quota)
|
||
|
Claude
|
3be85fa9f0 |
perf: round 15 — grouped-listing O(N²) rebucket, exif/reseed allocs, tantivy zero-hit snippet skip
Benchmark-gated, same rule as rounds 2–14: every change ships with a BEFORE/AFTER benchmark and an equivalence/safety gate; an AFTER that doesn't beat its BEFORE is rolled back. The rule is encoded per harness (GATE FAIL non-zero exit in the Rust examples, threshold expect() in vitest). F1 — Grouped listings (trash / recent / favorites / shared-with-me) re-bucketed the WHOLE accumulated list on every infinite-scroll page. ResourceSectionsBuilder (new, off the reactive graph) re-buckets only the fresh page and hands VirtualList the same rows array reference for untouched buckets. 50×50 (2 500-item) drain: 63 750 → 2 500 bucketOf calls (25.5×), 12.5 → 1.3 ms wall (9.9×); O(N²/page) → O(N). Deep-equal to the full-rebuild reference at every page for both a contiguous (date) and a non-contiguous (trash-by-drive) group-by; reference-stability + fallback gated. B1 — exif Make/Model: the display String was thrown away to allocate the trimmed copy; display_value_trimmed trims in place (drain + truncate), 2 → 1 alloc per field (8 → 4 allocs/op, 1.26×). B2 — content-index worker: text_extractor::supports (lowercases MIME + extension) was called twice per file per drain batch; classify once into a Vec<bool> and thread it through both uses. 256-file batch: 704 → 353 allocs, 34.5 → 16.7 µs (2.07×). B3 — tantivy: skip SnippetGenerator::create on a zero-hit content search (return Ok(vec![]) once top_docs.is_empty()); the per-hit loop was empty. 400-doc index: 1 575.6 → 1 237.2 ns (1.27×), widens with index size. Harnesses: examples/bench_round15_micro.rs, examples/bench_round15_tantivy.rs, frontend resourceSections.bench.test.ts; writeup in benches/ROUND15.md. Also normalizes two round14 bench examples that were committed unformatted (cargo fmt --all). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012o47jSrtL7xuNGTHXmtiYL |
||
|
Claude
|
c930f865b0 |
perf: round 14 — faces narrow projection, auth per-request allocs, CalDAV emit buffers, frontend set churn
Benchmark-gated (benches/ROUND14.md); every change ships a BEFORE/AFTER
benchmark with an equivalence gate and is rolled back on regression (the
rule is encoded as a GATE FAIL exit / threshold expect).
Backend
- Q1 faces_for_file → narrow face_boxes_for_file(id, person_id, bbox) with the
caller filter pushed into SQL: drops the 2 KiB embedding BYTEA + 6 unused
columns per face. 15-face lightbox open 0.312→0.219 ms, 32 KB→840 B/req.
- A1 cookie auth uses the borrow-only extract_cookie_str (already backs CSRF)
instead of extract_cookie_value's owned String: -1 alloc/cookie request.
- A2 compute_relevance ASCII case-fold fast path vs name.to_lowercase() per
result row (Unicode fallback preserved): 1.40x, 12→3 allocs/page.
- A3 sub pre-parsed to Uuid at decode time (TokenClaims.sub_id) vs re-parsing
the 36-char claim on every request incl. cache hits: 22.7→0.7 ns.
- A4 auth + NextCloud middlewares borrow request.headers() instead of taking
axum's HeaderMap extractor (a full map clone): 2→0 allocs/authed request.
- A5 CalDAV getlastmodified via the stack rfc2822_utc (byte-identical to
chrono) vs a per-event to_rfc2822() heap String: 5→0 allocs.
- A6 CalDAV per-event href + quoted etag written into reused page buffers vs a
fresh format! pair per event: 3.48x, 240→6 allocs/40-event page.
Frontend
- F1 t() shares one frozen EMPTY_PARAMS for the no-interpolation call forms vs
a throwaway {} per call: -1 alloc/call.
- F2 favorites favoriteIds is a persistent SvelteSet with per-page add (clear
on reset) vs a brand-new set over the whole accumulated list each page:
22.3x over a 40-page drain (O(N^2)→O(N)).
Verified: cargo check --all-targets, cargo clippy -D warnings, both bench
packs (GATE PASS), frontend npm run check + vitest (4/4). ROUND14.md also
records the investigated-but-deferred backlog (music N+1, contact vcard
over-fetch, CachedBlobBackend syscalls, ResourceList.sections builder, etc.).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PymgCdK78NzUF3oRAQCJfN
|
||
|
M.Schmidt
|
af74c94028 |
fix(webdav): make PATCH's concurrency guard a real compare-and-swap
The app-level ETag re-check before the write still left a gap between the check and the actual UPDATE for a concurrent writer to land in. Push the check into the write path itself: swap_blob_hash now takes an expected_hash and only applies the SET under the same FOR UPDATE row lock it already held, closing the race instead of just narrowing it. Adds ErrorKind::PreconditionFailed (412) for the CAS-miss path; PUT/WOPI/chunked-upload keep blind-overwrite semantics by passing None |
||
|
Claude
|
f58d72a780 |
perf: round 13 — grouped-view virtualization, notification/login query narrowing, HTTP dedup, locale precompute
Benchmark-gated (BEFORE/AFTER + equivalence/safety gate per change), same discipline as rounds 2-12. Full write-up in benches/ROUND13.md. Shipped: - V1 Grouped views windowed (files route + ResourceList). The grid arm was the last unwindowed path (trash is grouped-by-default in grid): each swimlane now feeds its own VirtualList, outer container a flex stack. vitest gate: 800-item grouped grid mounts <120 .file-item (was 800). - Q1 get_users_by_ids drops the <=512 KiB avatar image + ui_preferences JSONB (notification path never reads them). 30-member fan-out 8.60 -> 0.25 ms (34.3x), ~7.7 MB off the wire. - Q2 Login provisioning is_empty() -> SELECT EXISTS for calendar + address book (every login). 0.193 -> 0.170 ms, widens with owned-row count. - Q3 Recent-access prunes only when the upsert inserted (RETURNING xmax=0) — a re-access can't grow the set. 0.567 -> 0.324 ms (1.75x). - L1 Locale supported-codes precomputed once vs rebuilt per anonymous request. 616 -> 17.3 ns (35.7x), 18 -> 1 allocs. - H1 Duplicate /api TraceLayer removed (global stack already wraps it). 1.86 -> 1.42 us/request, -6 allocs. - H2 client_ip span field: borrow-only ClientIpDisplay vs owned String. 187 -> 173 ns, -1 alloc. Not shipped (discipline): the "media hooks read the blob 3x" lead was a correctness bug, not a perf dup — the raw-path metadata/faces readers resolve only for local+unencrypted+single-chunk blobs and silently produce nothing otherwise. Flagged for maintainers; routing through read_blob_bytes is a correctness fix (perf-neutral-to-negative), not a benchmark-gated perf change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB |
||
|
Claude
|
50eca0627f |
perf: round 12 — auth write-path narrowing, fused quota gate, moka blob-cache index, media single-read, sized listing JSON
Benchmark-gated round (benches/ROUND12.md; every change ships with a BEFORE/AFTER harness + equivalence gates, one candidate rejected by its own bench): DB / query shapes (bench_round12_queries): - NC sharee search: username-only projection instead of the 21-column row (incl. the <=512 KiB avatar) per match, + gin_trgm_ops indexes on auth.users for the leading-wildcard ILIKE (4.98x; 54.7x with index). - Password login: delete the redundant full-row update_user — create_session already stamps last_login_at in its own txn (4.45x per login). - Email-verified stamp: narrow conditional UPDATE (8.9x); OIDC repeat login now compares profile state in memory and issues ZERO queries when nothing changed (was: full 17-column rewrite per login). - Refresh rotation: revoke+insert+stamp fused into one transaction via new rotate_session port method (1.18x). - WOPI CheckFileInfo / authorize_wopi_access: require(Read) + get_file + check(Update) overlapped with tokio::join!, original result precedence (cold 1.34x). - Upload quota gate: user-envelope + drive-cap checks fused into ONE round-trip (check_upload_quotas) — the NC chunked PUT pays this per chunk (1.81x, 2 -> 1 queries/chunk); shared verdict evaluators keep error shapes byte-identical. CPU / allocs (bench_round12_micro): - sized_json: pre-sized listing serialization replacing axum Json's 128 B seed + doubling-realloc chain on files/folder-resources/photos/search responses (1.40x, 13 -> 2 allocs per 500-row page; byte-identical). - Security headers: 4 SetResponseHeaderLayer folded into the CSP middleware pass (5 layers -> 1; 1.43x per request, -26 allocs; header set gated byte-identical incl. 304s). - Media capture-metadata: single-read extraction — nom-exif now parses the buffer kamadak already read (zero-copy Bytes) and videos open once with a kind() dispatch; per-image opens 2-3 -> 1 (1.44x warm geomean, 1.6-3.2x cold cache; extraction outputs gated identical incl. the MIME-mislabel track fallback). - Chunked-upload session ops: owner gate folded into the operation's own DashMap lookup + stack-encoded uuid compare (5 -> 3 lookups, -2 allocs, 1.28x per chunk). Blob cache (bench_blob_cache_index + round-3 regression guard): - CachedBlobBackend index: tokio::sync::Mutex<LruCache> -> moka::sync::Cache with byte weigher. The mutex serialized every cached chunk read and scaled NEGATIVELY (2.08 -> 1.07 Mops/s from 1 -> 2 readers); moka probes are lock-free (2.17x at K=2). Byte budget now enforced by moka (manual current_size + collect_evictions machinery deleted); eviction listener unlinks size-evicted files only (Replaced entries keep their file — gated). Single-flight miss gate unchanged (16 concurrent misses -> 1 fetch re-verified via the round-3 harness). - put_blob now populates the cache BEFORE the inner backend consumes the source file (the old order failed 100% of the time — local renames, S3/Azure delete the source — so the first read after a whole-file put re-downloaded from the remote); inner-put failure invalidates the entry. Frontend (vitest gates): - List-view thumbnails request the 150px icon rendition instead of 400px preview into a 40px slot (~7.1x fewer pixels, ~4-5x fewer bytes per thumbnail across list views); grid keeps preview. Rejected by its own bench (kept as evidence in bench_round12_micro §2): - Single-pass compression predicate: the monomorphized And-chain already costs ~4.6 ns / 0 allocs total; the fused node measured within noise. New migration: 20260719000000_users_search_trgm.sql (trgm indexes). Deferred with prepared design: grouped file/grid view virtualization (single-VirtualRows flatten, the photos pattern) — next round's headline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB |
||
|
Claude
|
b01633791b |
perf: round 11 finale — benchmark verdicts, geo min(uuid) rollback, ROUND11.md
- benches/ROUND11.md: final BEFORE/AFTER numbers for all 21 micro sections, the 5 query sections, the 4 log-writer arms, the SPA gates, and the two cross-round regression guards (bench_row_path 2.52x, bench_dto_map — both byte-identical) - ROLLBACK (gate-caught): min(fm.file_id)::text geo-cluster cast — PostgreSQL has no min(uuid) aggregate; the bench section now reproduces the rejection and the per-row-cast original stays - REJECTED (bench-measured): tracing-appender non-blocking writer — slower than sync on fast sinks (1.41M vs 0.99M ev/s, worse tail) and the slow-sink drain gate showed shutdown tail-loss risk for the audit channel; dep moved to dev-dependencies (harness only) - RateLimiter final form: lock-free get + insert (8.0 → 6.0 allocs, wall neutral; and_upsert_with variant rejected at 2 365 ns) - fix: TrashedItemParts insertion had stolen TrashedItem's derive line (caught by clippy --all-targets) - grant_role enum values corrected in the queries bench Validation: cargo fmt + clippy --all-features --all-targets -D warnings clean; cargo test --workspace 524 passed / 0 failed; frontend npm run check 0 errors + vitest 306 passed (1 pre-existing round-7 wall-clock gate flaked only under concurrent Rust-build CPU contention; passes in isolation) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7 |
||
|
Claude
|
221c1f31b0 |
perf: round 11 — StoragePath joined-only, classifier fusion, memoized bodies, query-shape pack, SPA fine-grained stars
Backend (each change benchmark-gated with BEFORE replicas + equivalence gates; see examples/bench_round11_micro.rs, bench_round11_queries.rs, bench_log_writer.rs and benches/ROUND11.md — final numbers land in the follow-up doc commit): - StoragePath re-representation: single canonical joined String, segments derived on demand; File/Folder drop the duplicated path_string field (4000→1000 allocs per 500-row listing page) - Display classifier fusion: classify_display shares one stack-lowered extension across the three decision trees; call sites in FileDto, folder/favorites/recent handlers, trash, path-resolver (+ interning where Arc::from was still used) - /status.php and /openapi.json memoized into OnceLock<Bytes> (openapi rebuilt a 171 KiB spec per request: 2.8 ms → 18 ns) - NC upload-session PROPFIND: write! + pre-sized body + stack RFC2822 dates (2.3-2.6x, 2582→772 allocs at 256 chunks) - REST download: dead FileDto clone removed (capture mime/size + move) - CalendarEventDto/TrashedItem into_parts moves (11 KiB ical_data memcpy gone per CalDAV row); CardDAV getlastmodified stack render - 4xx path: borrowed ErrorResponse serialize, ErrorKind::as_str, not_found/already_exists clone kill - vCard emit via write!; search page moved out with into_iter skip/take; content-hit UUIDs parsed once; group last-user check via HashSet - RateLimiter: lock-free get + insert (and_upsert_with variant REJECTED by benchmark); CSRF token borrow-compare + borrowed cookie extraction - Thumbnail/preview ETags built from as_str (Debug-identical bytes) - Encrypted backend: encrypt_in_place_detached single-buffer write path, chunk-sized reserve in collect_stream; retry labels made lazy - PG: deferred upload registration 3→1 round-trips (persist_file CTE template); direct_grant_cache for Calendar/AddressBook/Playlist authz (single-flight + set_role/clear_role invalidation); expand_user tokio::join!; geo clusters min(uuid)::text; recluster face assignment batched into one UNNEST update - People recluster cosine: norms precomputed once (bit-identical gate) - NC capabilities poll logs demoted to debug; tracing-appender dep added for the log-writer benchmark Frontend: - ResourceList.selectedEntries O(N)-per-toggle → id-index projection O(k log k); favorites/recent consume the batchToolbar snippet param and drop their duplicate filter + dead selectedIds mirror - Recent: star state via new favoriteIds prop — a star click no longer rebuilds all N entries - admin timeAgo >30d fallback uses the cached Intl.DateTimeFormat - vitest gates in src/lib/components/round11.bench.test.ts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7 |
||
|
Claude
|
c51af68432 |
perf: round 10 — auth alloc purge, parent-herd batching, query-shape pack, NC 304s
Benchmark-gated (benches/ROUND10.md; every change carries a BEFORE/AFTER harness with equivalence/safety gates — two designs were rejected or rewritten by their own benches before adoption): - Auth hot path: TokenClaims/CurrentUser display fields to Arc<str>, role to inline SmolStr end-to-end (Bearer, cookie, Basic-auth cache) — 4→1 allocs per authenticated request, 3→0 per warm DAV request; JWT Encoding/Decoding/Validation built once. - Cold shared-album herd: leader-inline parent batching in PgAclEngine (+ cascade try_get_with single-flight) — 100→2 parent queries per 100-thumb cold herd, herd wall 1.9x, sequential + warm paths unchanged, all ROUND8/9 safety gates plus new herd-equivalence gates. - Query-shape pack: share download double-fetch 2→1 (2.18x), contact-group COUNT(*) 14.9x, save_faces UNNEST 3.9x, playlist reorder UNNEST 63.7x (now atomic), search files∥folders join! 1.45x, move drive-lookup join! 2.14x, trash partial (drive_id, trashed_at) indexes, CalDAV event-gate narrow read, favorites/recents binary-decode port, dead count_files removed. - NC surface: preview + avatar honour If-None-Match (e2e: 5 KB and 197 KB → 0 bytes per revalidation), avatar WebP→PNG transcode memoised, PROPFIND/trashbin integer+date emits on stack formatters, folder-header enrichment join!, chunk-PUT retry stat folded into create_new open. - common::fmt integer rendering rewritten on the std 2-digit LUT after the round's own bench caught the div-loop losing to to_string (16.1 ns vs 22.5; speeds every prior-round call site). - Micro-pack: WebDAV scope probe borrow-only, ShareService base_url snapshot, cookie_secure OnceLock, Arc'd AES-GCM cipher, stack request-id, tantivy analyzer clone dropped. - SPA: search stale-guard + AbortController (10→1 completed round-trips, stale-clobber gone), getFolder in-flight dedup, gridColumns matchMedia hoist (10k→0 style reads). Backend: cargo fmt + clippy -D warnings clean, 524 tests green. Frontend: npm run check clean, 301 vitest green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018DdM7V7M3QPW7HEHg3gLov |
||
|
Claude
|
fdf445d2b0 |
perf: round 9 — decorator PUT reactivation, session/search/dedup alloc purges, PROPFIND join!, folder-level cascade
Benchmark-gated round (benches/ROUND9.md): every change carries a BEFORE/AFTER bench with equivalence/safety gates; verdicts below are from the committed harnesses on 4 cores / local PG 16. Backend: - Blob decorators (Retry/Cached) now forward put_blob_from_bytes_unsynced + sync_blobs — the trait default had silently reinstated HEAD-before-PUT per chunk on decorated remote stacks, undoing ROUND3 §8. Full production stack: 500 probes -> 0, 1.9x wall at 10 ms RTT (bench_s3_put §3). - NC PROPFIND per-page enrichment triple (favorites / oc:fileid / dead props) overlapped with tokio::join!: 2.07x local, 2.86x at 5 ms RTT (bench_nc_enrich_join, injected-latency decide-by-bench). - Search enrichment consumes its DTOs and carries the interned Arc<str> display fields end-to-end (SearchFileResultDto type change, OpenAPI shape preserved): enrich_file 2.0x, 11.6 -> 2.2 allocs/row; the NC REPORT conversion stops re-running all three classifiers per row (bench_search_enrich). - NC session Arc end-to-end: SharedNcSession extractor (8 -> 0 allocs), Arc<FolderDto> chroot cache (4 -> 0/hit), single shared Arc<CurrentUser> + lazy span render (11 -> 6/build) (bench_nc_session). - Storage micro-pack: atomic create_new chunk writes (2.1x fresh), stream_chunks over the manifest Arc (4097 -> 0 allocs/read incl. the Range path), manifest single-flight (herd 64 -> 1 loads), hex_lower for chunk Content-MD5 (18 -> 1 allocs) (bench_storage_micro). - OCS capabilities memoized into OnceLock<[Bytes;2]>: 237x, 102 -> 0 allocs/poll, byte-identical (bench_capabilities_static). - Drive::is_empty COUNT(*) sum -> EXISTS: 34.4x on a 100k-file drive (bench_drive_is_empty). - favorites/recents row-map ROUND7 port: path/name/blob_hash moved, -2.75 allocs/row (bench_resource_row_map §2). - Folder rows decode binary UUIDs (ROUND6 §10 port): 1.03-1.07x page fetch, honest verdict incl. one noise-band wash documented (bench_folder_uuid_decode). - Authz: file cascade decision decomposed into memoized folder-level decision + direct-grant lookup (ROUND8 deferred item). Cold shared-album first view 592 -> 418 µs/thumb; warm path unchanged; safety gates incl. new direct-grant sibling isolation, revoke-flush re-verified, full integration authz suite green (bench_thumbnail_cascade_cache). Frontend (vitest gates committed beside the code): - resolveLabel/resolveRecipient O(directory) scan -> id-keyed Map: 13.9x (recipients.bench.test.ts). - ResourceList selection-prune effect skips when nothing is selected (100 -> 0 Set builds per drain) and the photos timeline reads a listener-fed mobile flag instead of matchMedia per recompute (listDerives.bench.test.ts). Verification: cargo fmt + clippy --all-features --all-targets -D warnings clean; 524 unit + 554 integration (--cfg integration_tests) tests pass; frontend npm run check clean with 293 vitest tests green. Deferred with rationale in ROUND9.md: CalDAV authz-before-fetch reorder (maintainer sign-off), per-page batched parent resolution, JWT-claims Arc<str>, batch_operations signature widening. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDc9VtXvskJ6dnMRraSndn |
||
|
Dionisio Pozo
|
2317d594e3 | Merge pull request #610 from EdouardVanbelle/security/grants2 | ||
|
Claude
|
79b94126be |
perf(authz): round 8 — cache the File/Folder grant-cascade decision for shared-album thumbnails
get_thumbnail_impl runs require_permission(Read) on every request. For a drive member that's a drive_role_cache hit, but a shared-album recipient — granted a folder (the album), not drive membership — fails the drive-role precheck and falls through to file_cascade_grant_exists (a role_grants ⋈ folders lpath ancestor query), once per file. Browsers revalidate immutable thumbnails constantly, so the same (recipient, file, Read) decision was recomputed on every thumbnail of every view — ~100 grant queries per 100-photo album per navigate-away-and-back. New cascade_grant_cache ((Subject, Resource, Permission) → bool, 30 s TTL) memoises that decision. The check is NEVER skipped — the ordering is unchanged, authz still runs on every request; only the result is cached, and only after the drive-role precheck fails (so a later drive grant can't be shadowed by a stale entry). Invalidation mirrors drive_role_cache's convention: explicit invalidate_all on every File/Folder set_role/clear_role (immediate revoke on the direct share path), 30 s TTL for the indirect paths (group membership, moves, expiry) "rather than a deep invalidation tree". Bench (bench_thumbnail_cascade_cache) with hard safety gates — recipient allowed, outsider denied, and a clear_role revoke denies the very next check (proving the grant-write flush): 100-photo album revalidation 2576 → 2.70 µs/thumb (~950x), 257.6 → 0.27 ms/view. Validated against the full --cfg integration_tests authz suite (554 tests) + 524 workspace tests, clippy -D warnings clean. Deliberately not done: moving authz after the 304/cache short-circuit (a security-posture change — a revoked user could serve cached thumbnails). With the decision cached, the authz on the 304 path is now a memory hit, so the "zero DB work on a 304" intent is restored without weakening the check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA |
||
|
Claude
|
f53fba42a1 |
Merge branch 'main' into claude/performance-optimization-round-6
Resolves the one conflict in file_blob_read_repository.rs's suggest_files_by_name: main added the CALLER_CAN_READ_DRIVE authz scope (caller_id param + drive-membership filter, AuthZ audit finding #1 — the suggest query previously leaked names/paths across tenants), round 6 switched the same query's id/folder_id columns to binary UUID decode. Kept both: main's authz structure (format! + CALLER_CAN_READ_DRIVE + caller_id bind) with round 6's binary decode (fi.id / fi.folder_id, no ::text) so the query matches the FileRow = (Uuid, …) tuple. The deliberately-text sites (min(fm.file_id::text), folder path lookup) stay text. Verified: build + clippy -D warnings clean, 524 unit + 554 integration tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA |
||
|
Claude
|
9729f033b2 |
perf: round 6 backend — CardDAV cursor streaming, borrowed NC id chain, binary UUID decode, one-alloc hex
Benchmark-gated (equivalence + BEFORE/AFTER in examples/bench_*, results and reproduce commands in benches/ROUND6.md): - CardDAV whole-book REPORT + depth-1 PROPFIND stream through a PG cursor (stream_contacts_by_book, 500-contact pages) instead of materialising every vCard twice: 8 000 contacts TTFB 37.4 → 7.6 ms (4.9x), peak heap 19.0 → 7.0 MiB (2.7x), wall -23%; REPORT and PROPFIND byte-identical to the buffered writers. - NC numeric-id chain fully borrowed: get_or_create_file_ids/folder_ids take &[&str] and return HashMap<Uuid, i64>; batch_resolve_ids callers (PROPFIND pages, REPORT, trashbin, OCS search) pass id slices and look up via nc_id_of. 2.006 → 0.006 allocs/child (334x), 1.53x wall per 500-child page. batch_check_favorites binds &[&str] as text[]. - file_blob_read_repository listing SELECTs drop id::text/folder_id::text server casts: rows decode binary Uuid (16 vs 36 bytes on the wire) and render once in row_to_file. A/B on 500-row pages: 1.225 → 1.044 ms mean (1.17x), p95 1.686 → 1.345 (bench_uuid_text_cast; single-row, param and min() sites left as-is deliberately). - IncrementalHasher::finalize_hex renders through common::fmt::hex_lower instead of one format! per digest byte: 18 → 1 (md5) / 35 → 1 (sha256) allocs per chunk finalize, 14-15x wall. - Share landing overlaps the access-count UPDATE with the unlock fetch via tokio::join! (one round-trip off every public link hit). - REJECTED by benchmark and reverted: try_join_all fan-out of the batch-favorites authz pre-check — 42.6 → 56.4 ms cold, 0.15 → 0.23 ms warm against local-socket PG (bench_favorites_authz kept as evidence). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA |
||
|
Dionisio Pozo
|
4f51cb7aab |
Merge pull request #613 from AtalayaLabs/claude/performance-optimization-round-5
perf: round 5 — CalDAV cursor streaming, SPA interning gaps, NC href prefix, per-request micro-allocs |
||
|
Edouard Vanbelle
|
bd7b0710a8 |
fix(cache): invalidate root folder cache on rename
this fix https://github.com/AtalayaLabs/OxiCloud/issues/607
which was introduced by commit
|
||
|
Claude
|
63cf6646d0 |
perf: round 5 — CalDAV cursor streaming, SPA interning gaps, NC href prefix, per-request micro-allocs
Seven benchmark-gated changes (benches/ROUND5.md; BEFORE/AFTER bench + equivalence gate each, rollback rule as ROUND2-4 — two intermediate CalDAV shapes measured worse and were themselves rolled back before shipping): - CalDAV whole-calendar responses (REPORT no-range/sync-collection, depth-1 collection PROPFIND, .ics GET): buffered double-residency → ONE window-ordered scan (MIN(start_time) OVER (PARTITION BY ical_uid)) streamed through a PG cursor, pages cut at UID boundaries. TTFB 23.3→11.0 ms (2.1x), peak heap 14.2→8.0 MiB at 4k events / 45→24 MiB at 12k, wall +9-15% (documented trade, ZIP-streaming class); both multistatus and ICS byte-identical to the buffered output. Rejected shapes kept in the doc: per-page GROUP-BY keyset (3-4x wall) and per-uid ANY hydration (~20 µs/index descent). - SPA listing interning gaps: folder/recent/favorites resources handlers (and the WebDAV pseudo-root) called raw Arc::from per row for the closed display set ROUND3 interned — now intern_display/intern_mime, 4→0 allocs/row, byte-identical Arc contents. - NC PROPFIND child hrefs: username + parent path encoded once per request instead of per child (543→165 ns/row, 13→4 allocs); native WebDAV href drops its intermediate encode String. - suggest enrichment: entity clone + field re-clones per keystroke row → consume + move (166.5→126.8 µs/200 rows, 20→7 allocs/row). - list_readable_by returns the cache's Arc (246→128 ns warm hit, 4→0 allocs) — deep Vec clone per DAV-selector request removed. - CardDAV REPORT: borrowed props, reused href buffer, exact-size etag quoting (3.04→2.34 ms per 5k-contact getetag poll). - Auth span records: user_id.to_string() per request ×3 → tracing::field::display. Checks: cargo fmt, clippy --all-features --all-targets -D warnings, cargo test --workspace (523 passed). Follow-ups (CardDAV streaming, &[&str] id batches, ::text UUID casts A/B, share-landing join) recorded in benches/ROUND5.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA |
||
|
Dionisio Pozo
|
8aa013d3ee |
Merge pull request #605 from AtalayaLabs/claude/performance-optimization-analysis-sisndc
ROUND4: Row-path allocs, drive cache, CalDAV parse, PROPFIND emit, N+1 hydration, Azure streaming, faces bound |
||
|
Claude
|
12dc648cff |
perf: round 4 — one-pass row paths, drive-selector cache, CalDAV single-parse, streamed Azure, batched hydration
Nine benchmark-gated changes (benches/ROUND4.md; every one ships with a BEFORE/AFTER bench + equivalence gate, rollback rule as ROUND2/3): - Row→entity path build: one-pass StoragePath::from_folder_and_name / from_joined + normalize_storage_name_owned + alloc-free Display — 743→417 ns/file-row (1.78x), −5 allocs/row on every listing surface. - WebDAV drive-selector: per-user readable_cache (single-flight, 30 s TTL, explicit invalidation incl. membership + group changes) replaces the grants join per request — 441 µs → 0.8 µs (~550x), 0 queries warm. - CalDAV from_ical/update_ical_data: 8 full IcalParser runs per VEVENT → 1 (7.1x per PUT, 4.4x on 50-event imports); alloc-free split_vevents, chunk scan without the whole-body uppercase copy (1.4x), borrowed-key UID grouping (1.3x), REPORT props no longer cloned. - PROPFIND emit: partition Vecs dropped (single-pass 404 list) + stack rendered RFC 3339/2822 dates, sizes, quoted etags (common::fmt, chrono-byte-identical, sweep-tested) on both DAV surfaces — 1.22x per page, 17.9→12.0 allocs/row. - Grant-listing hydration: calendars/address books/playlists batch hydrate via = ANY($1) — 15 serial queries → 1 (~13x per sync poll). - user-flags cache: get→insert → try_get_with single-flight (32→1 queries per cold herd). - Azure downloads: whole-blob Vec buffering → streamed SDK pages — TTFB 349→4 ms (87x), peak heap 480→1.9 MiB (254x) on 256 MiB blobs; new OXICLOUD_AZURE_ENDPOINT_URL override (Azurite/bench hook). - Face indexing: unbounded per-image tokio::spawn → core-count semaphore, permit before blob read — peak heap 1175→176 MiB (6.7x). Checks: cargo fmt, clippy --all-features --all-targets -D warnings, cargo test --workspace (523 passed) + --features test_utils. hurl API suite and dockerized integration DB not runnable in this environment — left to CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA |
||
|
Dionisio Pozo
|
807536b945 |
Merge pull request #601 from EdouardVanbelle/security/grants
security/grants |
||
|
Claude
|
cd4c62042a |
perf: keyset/LATERAL SQL shapes, auth+blob-cache single-flight, spool buffers, DTO interning
Round 3 of benchmark-gated optimizations (benches/ROUND3.md; every change gated by a before/after benchmark — an AFTER that did not beat its BEFORE was to be rolled back; none needed it. Equivalence gates assert identical row sequences / byte-identical output on every behavior-preserving rewrite): DB hot paths (local PG16, EXPLAIN-verified): - Web-UI listing (list_resources_paged): cursor pushed INSIDE the folders/files UNION-ALL branches as sargable row-value comparisons with per-branch ORDER/LIMIT + two partial expression indexes (folder_id, LOWER(name), id). 20k-entry folder: 26.6 -> 1.3 ms/page (19.5x); other sort modes at parity or better. New migration 20260918000000. [benches/LISTING-KEYSET.md section in ROUND3] - Photos timeline (list_media_files): per-drive CROSS JOIN LATERAL top-N on the timeline index, joins moved above the top-N. 50k-photo library: 97.4 -> 1.6 ms/page (55.7x). The old "LIMIT stops the scan early" comment was refuted by EXPLAIN. - PROPFIND sub-folders (both DAV surfaces): keyset list_folders_batch off idx_folders_unique_name replaces COUNT(*) OVER() + LIMIT/OFFSET (5k dirs: 79.7 -> 17.9 ms full walk, 4.5x). Concurrency: - Basic-auth cache single-flight (moka try_get_with): 8 concurrent DAV connections at TTL expiry paid 8 Argon2id runs (2.6 s CPU + 8x64 MiB); now 1 (300 ms). Failed verifications remain uncached. - CachedBlobBackend per-hash single-flight + unique tmp names: 16 concurrent cold readers = 16 full remote downloads racing truncating writes on ONE deterministic .tmp (corruptible cache); now 1 download (16x less egress, 2.8x wall on a shared link) and torn files can never be renamed into the cache. I/O and allocations: - Chunk-assembly reads 64K -> 512K buffers (2.3x, 8x fewer syscalls); chunk-spool writes via BufWriter 512K (5.6x, 32x fewer syscalls). - S3/Azure put_blob_from_bytes_unsynced overrides: dedup settle no longer pays a HEAD probe per new chunk (2 RTT -> 1, 1.8x); Azure stops copying every chunk (Bytes -> Body, -0.44 ms - 4 MiB alloc per 4 MiB chunk). - Entity->DTO mapping: Arc<str> interning of closed-set display fields + common MIMEs, 1-alloc etag/size formatting, FolderDto moves instead of clones. File row: 11 -> 4 allocs; folder row: 11.8 -> 1 (2.1x faster). - CardDAV REPORT: deleted dead per-contact vCard pre-generation and the O(N^2) uid scan whose result was discarded (5k contacts: 55.7 -> 5.7 ms, 9.8x); byte-identical XML asserted. - Search-results cache: byte weigher + 32 MiB budget (OXICLOUD_SEARCH_CACHE_MAX_BYTES) replaces the 1000-ENTRY cap that let ~300 MiB of enriched rows sit in RSS; read latency parity. - Dropped aws-config + aws-smithy-types (zero references; -82 dep-graph nodes, three SDK stacks gone from every build). tokio "process" is now an explicit feature (was enabled transitively by aws-config). Frontend: - Cached Intl.DateTimeFormat keyed by (locale, options) in formatDate and 4 sibling callsites: 20k dates 2612 -> 51 ms (51.6x); vitest gate asserts output identity across locales and a 3x floor. Validation: cargo fmt + clippy --all-features --all-targets -D warnings clean; 518 unit + 548 integration-cfg tests green; new-shape endpoints smoke-tested end-to-end over HTTP (all 5 listing sort modes with cursor walks, WebDAV PROPFIND Depth-1, photos timeline, Basic-auth DAV login); frontend npm run check clean, new vitest gates green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsU2qEzny3A8WQUEuMNCr |
||
|
Edouard Vanbelle
|
c1924c825b | security(search): ensure that search suggenstion returns answer the user has access to | ||
|
Claude
|
82ee7da0d2 |
perf: serve ranges from RAM cache, stream ZIPs, overlap ingest settle, O(1) chunk gate
Round 2 of benchmark-gated optimizations (benches/ROUND2.md; every change gated by a before/after in examples/bench_round2.rs — an AFTER that did not beat its BEFORE was to be rolled back; none needed it): - Range requests (REST/DAV/shares) answered from the moka content cache for sub-10MB files: PG resolve + open/seek/read -> Bytes::slice. 256KiB seeks: 1,730/s -> 3.7M/s (p50 552us -> 0.15us). - Streaming folder/share ZIPs via tokio duplex: TTFB no longer scales with archive size (326ms -> 0.4ms on 192MiB corpus; total also faster). Content-Length dropped (size unknown up front). - NC chunked-upload per-PUT gate: O(k) directory scan+stat -> in-RAM per-session counter (lazy rebuild on cold start). 1,000-chunk upload gate cost: 33.1s -> 0.09s cumulative. - Delta download + commit-verify now use the CDC path's buffered(read_prefetch) read-ahead: 64-chunk drain at 5ms open latency 440ms -> 51ms; order preserved. - CDC ingest settles batches on a spawned task (depth-1 pipeline) so the source stream keeps flowing during PG pin + backend writes; rollback ledger shared + lock-serialized so compensation stays exact on cancellation. 512MiB paced ingest: 60-69 -> 74-75 MB/s. OXICLOUD_INGEST_OVERLAP=0 restores inline settling (ops/bench hatch). - Frontend: instant-upload BLAKE3 hashing moved off the main thread to a bounded Web Worker pool (File handles by reference); vitest gate asserts the pool beats sequential (first gate draft posting buffers was 2.6x slower and was rewritten — copies dominated). Validation: cargo fmt + clippy -D warnings clean; 514 unit + 544 integration tests green; 270 frontend tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w |
||
|
Claude
|
aba89c4f5d |
perf: eliminate N+1 hot-path queries, cache immutable lookups, stop re-compressing compressed bytes
Every change is benchmark-verified (harness + before/after numbers in benches/, measured on this branch; reproduction commands in each doc): DAV / sync-client hot paths - PROPFIND dead-properties: one = ANY($1) query per 500-child page instead of one sequential query per child, and indexable `=` predicates instead of IS NOT DISTINCT FROM (seq scans). 2,000-child folder: 1.07-4.54 s of DB chatter -> 4-6 ms (258-773x). Applied to native + NC PROPFIND and both NC REPORT handlers. [benches/DEAD-PROPS.md] - Folder paging: keyset cursor (name > $last) + new partial index (folder_id, name) replaces LIMIT/OFFSET full-folder rescan per page. Full 20k-file walk: 1266 ms -> 77 ms (16.5x). New migration 20260917000000. [benches/PROPFIND-PAGING.md] - NC chroot / default-drive resolution: moka caches (30 s TTL, explicit invalidation on drive mutations) for find_default_for_user and the markerless chroot FolderDto. 2 uncached queries + 2 pool checkouts per NC/WebDAV/WOPI request -> sub-us moka hit (p50 0.7-3.6 ms -> ~1 us). [benches/CHROOT-CACHE.md] - Quota: PROPFINDs whose prop list never names a quota prop skip the 2-query resolution entirely (wants_quota()); the remaining lookups read 2 columns instead of the full auth.users row with its <=512 KiB avatar (11-16x, p50 3.4 ms -> 0.29 ms). Same narrow read now gates every upload quota check. [benches/QUOTA-PATH.md] CPU on the request path - ZIP exports (folder download, share ZIP, batch download): entries whose MIME says already-compressed (JPEG/MP4/zip/pdf/...) are Stored instead of Deflate - deflate ran inline on the tokio writer task at ~41 MB/s for ~0% size gain. Mixed media corpus: 4.31x wall and CPU, archive size unchanged. Shared predicate in common::mime_detect. [benches/ZIP-MEDIA.md] - Compression layers: tower-http's default maps to Brotli QUALITY 11 (verified in brotli-8.0.2 source and empirically: 90 ms per 64 KiB JSON response, 1.3 s per 700 KiB bundle). Both layers pinned to Precise(4): 99x less CPU for ~15% more bytes. SPA assets are now precompressed at build time (scripts/precompress.mjs, 77% smaller) and served via ServeDir::precompressed_br/gzip: 2016x less per-request work, and clients get the better q11 bytes. [benches/STATIC-PRECOMPRESSED.md] Batched / cached backend paths [benches/NPLUS1-AND-CACHES.md] - Content-search ReBAC re-verification: new AuthorizationEngine::check_files_read_batch (default = old loop; PgAclEngine override batches drive resolution + reuses role cache). 200 sequential point SELECTs per search -> 1-2 queries. - Batch-ZIP subtree downloads: drop per-file re-authz + per-file Recent recording (2 writes/file) for subtree entries already authorized at the root - mirrors the native folder-download path. ~6,000 statements removed from a 2,000-file archive. - CDC chunk manifests: immutable by content address, now moka-cached (weight-bounded 32 MiB, 60 s TTL, positive-only, invalidated on delete) - removes one manifest query (p50 0.44-4.4 ms) from every stream, range and full blob read. - People tab: grouped COUNT + batched cover lookup instead of dragging every face row with its 2 KiB embedding (10k faces: 30.4 ms & 21 MB -> 3.8 ms & 1.3 KB, 8.1x); merge() is one set-based UPDATE. [benches/PEOPLE-LIST.md] - Photos timeline cursor: raw timestamptz comparison instead of EXTRACT(EPOCH ...) wrapper + IS NULL OR disjunction - cursor is an index boundary again, deep scroll stops re-scanning skipped rows. - Public share landing: one atomic UPDATE ... access_count + 1 (was SELECT + full-row write-back: racy, lost updates, clobbered concurrent owner edits) - 3 round-trips -> 2 per visit. - move_to_trash: dead full-entity SELECT feeding a documented no-op removed from both branches; dead fields dropped from TrashService. - NFC normalization: is_nfc_quick fast path skips the decompose/recompose state machine for the ~100% already-NFC case (every row loaded from PG). Frontend - Large folders paint after page one (~200 items) via fetchFolderListing's new onPage hook instead of waiting for every sequential page. - Tested-and-reverted (kept for the record): cached Intl.Collator for name sorts - vitest showed it 2x SLOWER than V8's argument-less localeCompare fast path (5.6 ms vs 12.1 ms / 5k names). Sort order untouched. New bench harnesses under examples/ (bench feature): zip_media, dead_props, chroot_cache, quota_path, people_list, propfind_paging, static_precompress. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w |
||
|
Edouard Vanbelle
|
a6427fc028 |
feat(drive): add readonly policy
permmit admin to freeze a drive, trash janitor background job is also disabled for this drive |
||
|
Dionisio Pozo
|
fc5a3a581f |
Merge pull request #592 from EdouardVanbelle/fix/carddav-parser-tel-adr
fix(carddav): fix tel uri |
||
|
Edouard Vanbelle
|
99d0287fec |
fix(test): NextcloudChunkedUploadService
fix timing issues during tests that raises: ``` thread 'infrastructure::services::nextcloud_chunked_upload_service::tests::test_chunk_paths_sorted_regardless_of_upload_order' (6962) panicked at src/infrastructure/services/nextcloud_chunked_upload_service.rs:306:9: assertion `left == right` failed left: [66, 67] right: [65, 66, 67] note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ``` |
||
|
Edouard Vanbelle
|
7966c7178a | fix(528): pass3: PUT with RECURRENCE-ID | ||
|
Edouard Vanbelle
|
02f67f7a5c | fix(528): pass2 add recurrence_id field | ||
|
Edouard Vanbelle
|
f331dbf0ee | feat(account): upgrade external to internal | ||
|
Dionisio Pozo
|
5ae551a93d |
Merge pull request #577 from EdouardVanbelle/feat/users-perfs-and-filter-dotfiles
feat: users prefs server side + filter dotfiles + filter shares by resource type |
||
|
Edouard Vanbelle
|
e94063d96a |
test(login/register): via password or magic-link
Password login ┌─────┬────────────────────────────────────────────────────┬────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────┐ │ # │ Case │ Where │ Assertion │ ├─────┼────────────────────────────────────────────────────┼────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────┤ │ L1 │ Login by username │ auth_login.hurl Case 1 │ 200 + access_token, user.email match │ ├─────┼────────────────────────────────────────────────────┼────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────┤ │ L2 │ Login by email (dispatch on @) │ auth_login.hurl Case 2 │ 200, same session shape as L1 │ ├─────┼────────────────────────────────────────────────────┼────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────┤ │ L3 │ Bad password on username path │ auth_login.hurl Case 3 │ 403 anti-enum │ ├─────┼────────────────────────────────────────────────────┼────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────┤ │ L4 │ Bad password on email path │ auth_login.hurl Case 4 │ 403 anti-enum (same shape as L3) │ ├─────┼────────────────────────────────────────────────────┼────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────┤ │ L5 │ Unknown username │ auth_login.hurl Case 5 │ 403 anti-enum (same shape as L3) │ ├─────┼────────────────────────────────────────────────────┼────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────┤ │ L6 │ Unknown email │ auth_login.hurl Case 6 │ 403 anti-enum (same shape as L3) │ ├─────┼────────────────────────────────────────────────────┼────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────┤ │ L7 │ /api/auth/oidc/providers reports methods correctly │ auth_login.hurl Case 7 │ password_login_enabled: true, magic_link_login_enabled: true, require_verified_email: false │ └─────┴────────────────────────────────────────────────────┴────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────┘ Password registration ┌─────┬───────────────────────────────────────────────────┬──────────────────────────────┬─────────────────────────────────────────────────────────┐ │ # │ Case │ Where │ Assertion │ ├─────┼───────────────────────────────────────────────────┼──────────────────────────────┼─────────────────────────────────────────────────────────┤ │ R1 │ Classic username + email + password → uniform 200 │ registration.hurl Step 2 │ anti-enum message contains "request received" │ ├─────┼───────────────────────────────────────────────────┼──────────────────────────────┼─────────────────────────────────────────────────────────┤ │ R2 │ Login after register works │ registration.hurl Step 2b │ 200 + session for the new user │ ├─────┼───────────────────────────────────────────────────┼──────────────────────────────┼─────────────────────────────────────────────────────────┤ │ R3 │ Email collision → uniform 200 (no rewrite) │ registration.hurl Steps 8-10 │ attacker password doesn't work; original account intact │ ├─────┼───────────────────────────────────────────────────┼──────────────────────────────┼────────────────────────────┤ │ R4 │ Username collision → uniform 200 │ registration.hurl Step 11 │ same anti-enum shape │ ├─────┼───────────────────────────────────────────────────┼──────────────────────────────┼────────────────────────────┤ │ R5 │ Off-domain rejection │ registration.hurl Step 12 │ 403 RegistrationDomainNotAllowed │ ├─────┼───────────────────────────────────────────────────┼──────────────────────────────┼────────────────────────────┤ │ R6 │ Case-insensitive domain match │ registration.hurl Step 12b │ uniform 200 on charlie@EXAMPLE.COM │ └─────┴───────────────────────────────────────────────────┴──────────────────────────────┴────────────────────────────┘ Magic-link registration (email-only signup) ┌─────┬──────────────────────────────────────────────────────────────────────────────────────────────────┬───────────────────────────────────────────────────┐ │ # │ Case │ Where │ Assertion │ ├─────┼──────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────┤ │ MR1 │ Email-only signup → welcome mail queued │ registration.hurl Step 3 │ uniform 200 + browser-binding cookie set │ ├─────┼──────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────┤ │ MR2 │ Welcome mail contains magic-link URL │ registration.hurl Step 4 │ captured from mock SMTP │ ├─────┼──────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────┤ │ MR3 │ PR 22 cross-browser confirmation page │ registration.hurl Step 5a │ 200 HTML "different browser" │ ├─────┼──────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────┤ │ MR4 │ Cookie-bound redemption lands on SPA │ registration.hurl Step 5b │ 302 → /files (SvelteKit route, post-migration) │ ├─────┼──────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────┤ │ MR5 │ email_verified_at stamped after redemption │ registration.hurl Step 6 │ field present on /api/auth/me │ ├─────┼──────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────┤ │ MR6 │ Second magic-link post-signup │ registration.hurl Step 7 │ uniform 200 │ ├─────┼──────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────┤ │ MR7 │ Profile PATCH — no-op, name set, empty-string rejected, username-taken 409, claim-once 409, etc. │ registration.hurl Steps 6a–6i │ full profile lifecycle │ └─────┴──────────────────────────────────────────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────┘ Magic-link login (existing account) ┌─────┬──────────────────────────────────────────────────────────┬──────────────────────────────────────┬───────────────────────────────────────┐ │ # │ Case │ Where │ Assertion │ ├─────┼──────────────────────────────────────────────────────────┼──────────────────────────────────────┼───────────────────────────────────────┤ │ ML1 │ Baseline password login still works │ auth_magic_link_login.hurl Steps 1-2 │ 200 │ ├─────┼──────────────────────────────────────────────────────────┼──────────────────────────────────────┼───────────────────────────────────────┤ │ ML2 │ magic-link/send with email identifier │ auth_magic_link_login.hurl Step 3 │ uniform 200 + cookie │ ├─────┼──────────────────────────────────────────────────────────┼──────────────────────────────────────┼───────────────────────────────────────┤ │ ML3 │ magic-link/send with username identifier (dispatch on @) │ auth_magic_link_login.hurl Step 4 │ uniform 200 │ ├─────┼──────────────────────────────────────────────────────────┼──────────────────────────────────────┼───────────────────────────────────────┤ │ ML4 │ Password-user policy: mail actually sent │ auth_magic_link_login.hurl Step 5 │ SMTP capture proves permit_magic_link_for_password_users in effect │ ├─────┼──────────────────────────────────────────────────────────┼──────────────────────────────────────┼───────────────────────────────────────┤ │ ML5 │ Redemption creates a session │ auth_magic_link_login.hurl Steps 6-7 │ 302 → /files, /api/auth/me returns the same user │ ├─────┼──────────────────────────────────────────────────────────┼──────────────────────────────────────┼───────────────────────────────────────┤ │ ML6 │ Anti-enum on unknown identifier │ auth_magic_link_login.hurl Step 8 │ same uniform 200 shape as ML3 │ └─────┴──────────────────────────────────────────────────────────┴──────────────────────────────────────┴───────────────────────────────────────┘ OIDC ┌─────┬────────────────────────────────────────────────────────────────────────┬───────────────────┬────────────────────────────────────────────────────────────────────────────────────────────┐ │ # │ Case │ Where │ Assertion │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O1 │ Setup local admin (bootstrap) │ oidc.hurl Step 1 │ 201 │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O2 │ Providers endpoint — OIDC visible │ oidc.hurl Step 2 │ enabled: true, provider_name: MockSSO, password_login_enabled: true, magic_link_login_enabled: false (OIDC-master rule) │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O2b │ Magic-link/send refused (endpoint layer) │ oidc.hurl Step 2b │ 403 MagicLinkLoginDisabled — proves the policy gate fires, not a 503 SMTP-unwired │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O3 │ Authorize redirect includes PKCE + state │ oidc.hurl Step 3 │ 307 to fake IdP │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O4 │ IdP round-trip + JIT provisioning │ oidc.hurl Step 4 │ Callback lands on /login?oidc_code=… │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O5 │ Code exchange → session cookies │ oidc.hurl Step 5 │ 200 + all three cookies │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O6 │ JIT profile mapping (name, given/family, picture, groups → admin role) │ oidc.hurl Step 6 │ every claim reflected on /api/auth/me │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O7 │ Refresh rotation on OIDC session │ oidc.hurl Step 7 │ new access/refresh/CSRF cookies │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O8 │ Refreshed cookies authenticate │ oidc.hurl Step 8 │ 200 on /api/auth/me │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O9 │ Repeat login = same local user (no dup) │ oidc.hurl Step 9 │ user_id stable │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O10 │ Anti-takeover: unverified email → refused │ oidc.hurl Step 10 │ 401/403 │ ├─────┼────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤ │ O11 │ One-time code replay refused │ oidc.hurl Step 11 │ second /exchange → 401 │ └─────┴────────────────────────────────────────────────────────────────────────┴───────────────────┴────────────────────────────────────────────────────────────────────────────────────────────┘ test |
||
|
Edouard Vanbelle
|
5aaf49859e |
feat(user-perf): add ui user-perf + dotfile filter
- add resource kind filter (file, folder, drive) in shared section (localStorage stored) - add user preferences serverside store - add client side dotfile filter (show/hide dotfiles) (user perf stored, default: dotfiles are shown) for security trashed dotfile are always displayed protection added: if a folder has only hidden items, a notification invite user to display it if a user rename or create a hidden item, a notification tells it to user |