8d1fde2747f677d702ee6833f5e305249f810a34
59 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Edouard Vanbelle
|
cc3be1ec38 | refactor: apply clippy recos for rustc 1.98.0 | ||
|
Edouard Vanbelle
|
543a1a88eb | feat(admin > users): UI: correct oidc badge | ||
|
Edouard Vanbelle
|
d0712817c5 | refactor(dpop): apply clippy | ||
|
Edouard Vanbelle
|
e9495a63ad |
feat(oidc): permit auto/manual oidc account link/unlink
link are checking that email matches, +email alias are normalize into email if email is already used on another account, link is not possible not usurpation risk as the IDP is choosen by the admin |
||
|
Edouard Vanbelle
|
d8b3f2e026 |
refactor(oidc): migrate provider into issuer
this make OIDC compliant with the invariant binding (issuer and subject) admin can now rename their provider without breaking clarifing federation_kind: report the kind of federation wired not the allowed login method hybryd login method are still allowed |
||
|
Edouard Vanbelle
|
9485ee5540 | feat(storage key rot): add blob header engine | ||
|
Edouard Vanbelle
|
2de71b6d9a | feat(storage): add readonly during storage migration | ||
|
Edouard Vanbelle
|
99e6de6fce | feat(search): invalidate user's cache on share or favorite change | ||
|
Edouard Vanbelle
|
c22741bc7f | refactor(search): normalize answer to /resources format | ||
|
Dionisio Pozo
|
86268049ff |
Merge pull request #641 from EdouardVanbelle/refactor/front-resource-list
feat(fileDto, folderDto): add is_favorite + is_shared |
||
|
Edouard Vanbelle
|
d9987782c4 |
feat(fileDto, folderDto): add is_favorite + is_shared
- provide is_shared and is_favorite information in DTO, information propagated as badge/buttons per items
regarding performances I try to be minimalis on SQL to prevent any perf regression
- remove old set of sharedids and favoriteids (was not functionnal anymore)
- fix date picker (no past selection) in grant
- fix contextmenu close on /files section
|
||
|
Claude
|
8e55caa8a9 |
perf(round29): cache-serve borrow-probe, NC REPORT href buffer, auth per-req allocs, DB over-fetch
Seven behaviour-preserving allocation / copy / bandwidth cuts, each behind a counting-allocator BEFORE/AFTER gate that exit(1)s unless AFTER allocates strictly fewer than BEFORE (benches/ROUND29.md, examples/bench_round29_micro.rs). - [B] Content-cache serve fast path (optimized_inner Tier 1 + get_file_range_preloaded — the video-scrub hot path): probe the cache with a borrow first and build the owned get_or_load args (quoted-etag / key / id Strings) only on a miss, instead of allocating them before every probe and discarding them on a hit. 6 -> 0 allocs per cache hit. Splits get_or_load into get + load_and_cache so the miss path is not re-probed and the hit/miss stat counters stay byte-identical. Also drops the unconditional content_hash/name clones that ran for the >=10 MB streaming tier that used neither. - [A] NextCloud REPORT emit loops: per-row href String (and format! per folder row) -> one reused href_buf via nc_href_into / nc_collection_href_into with the URL-encoded user computed once per page. 1497 fewer allocs on a 500-row page. - [C] read_full: a single-frame blob is returned zero-copy instead of a second whole-payload memcpy into a fresh BytesMut; multi-frame path unchanged. - [D] login-lockout key: to_lowercase()+format! -> one pre-sized ASCII buffer (non-ASCII keeps str::to_lowercase). 3 -> 1 alloc/req, byte-identical key. - [E] NC composite-username parse: owned clone/to_string -> &str borrow of the already-owned raw_username. 1 -> 0 alloc on the common no-marker path. - [F] get_contacts_in_group: stop SELECTing the discarded multi-KB vcard column (the live method ROUND25 §Q2 missed; ContactDto has no vcard field). - [G] count_admin_users: add count_users_by_role -> scalar COUNT(*) instead of hydrating every admin's full row (incl. up-to-512 KiB avatar + ui_preferences JSONB) only to .len() it, on a bootstrap-polled status endpoint. All seven gates pass; cargo fmt --check and cargo clippy --all-features --all-targets -D warnings clean. §F/§G additionally validated against a live PostgreSQL 16 with the full migration set (query validity, result equivalence, 600000 -> 8 byte wire delta on the admin count). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LhpDZxSQTAGnAqCHUdtG5N |
||
|
Claude
|
8c936de50d |
perf(round27): NextCloud PROPFIND oc:id per-row buffer, contact JSONB write direct-serialize
Two behaviour-preserving allocation cuts (benches/ROUND27.md), each with a
counting-allocator BEFORE/AFTER gate that rolls back if AFTER does not allocate
fewer than BEFORE:
- H1 NextCloud PROPFIND: the streaming page loops built oc:id as a fresh String
per child (format_oc_id -> format!("{:08}{}", id, instance)). Add
format_oc_id_into(&mut buf, id, svc) and compute into one oc_buf reused across
the page (next to the existing href buffer) — 1 String/row -> 0. 998->0
per-row allocs on a 500-row page, 2.16x wall. The write fns still take
Option<&str>, so no signature change; oc:id bytes identical. Scoped to the two
PROPFIND page loops (the hot directory-listing path); REPORT/trashbin deferred.
- P2 contact create/update: bind sqlx::types::Json(&dtos) (Encode runs to_writer
straight into the JSONB buffer) instead of serde_json::to_value(&dtos) + bind,
which built a throwaway Value DOM per JSONB column. Write-side twin of ROUND23
J1. 21->2 allocs, 4.68x wall for a 3-entry column. Behaviour-preserving:
to_value sorts keys and direct serialize keeps struct order, but Postgres
normalizes JSONB key order so the stored value is identical (verified via psql:
'{...alpha...}'::jsonb = '{...struct...}'::jsonb -> t), and reads decode by
field name; the etag comes from the domain entity, not the stored JSONB.
Adds bench_round27_micro. Verified: cargo fmt clean, cargo clippy --features
bench -D warnings clean (real exit), cargo test --lib --features bench = 529
passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
|
||
|
Claude
|
eeb41c9c28 |
fix(bench): silence clippy neg_cmp_op_on_partial_ord in round25/26 gate helpers
The BEFORE/AFTER rollback gates wrote `if !(after < before)`, which clippy::neg_cmp_op_on_partial_ord flags on f64 (a negated comparison on a partially-ordered type). Under CI's `cargo clippy --all-targets -- -D warnings` this fails. Replace with the equivalent `if after >= before` (fail the gate when AFTER does not strictly beat BEFORE). No behavioral change to any benchmark. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT |
||
|
Claude
|
5b2bb8f883 |
perf(round26): drive-policy JSONB decode, CachedBlobBackend shard-dir pre-create, delta-upload foldhash
Three benchmark-gated optimizations from the ROUND25 backlog (benches/ROUND26.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- P1 drive_pg_repository policy reads: decode d.policies through
sqlx::types::Json<DrivePolicies> (one from_slice over the raw JSONB bytes)
instead of a throwaway serde_json::Value DOM + DrivePolicies::from_value —
6 -> 0 allocs/read, 2.77x wall. A shared policies_from_row helper preserves
the lenient unwrap_or_default fallback (malformed bag -> all-false).
- D1 CachedBlobBackend: pre-create the 256 {00..ff} shard dirs at initialize()
(mirroring LocalBlobBackend, reusing HEX_PREFIXES) and drop the redundant
per-write create_dir_all on already-existing shards — ~45us + a blocking-pool
dispatch removed per cache write on cached-remote deployments.
- G1 delta-upload have/need hash sets (distinct_hashes, authorize_chunk_download):
SipHash -> foldhash::quality::RandomState — a fast hasher that stays
DoS-resistant via a per-instance random seed, the required property for the
attacker-controlled 64-hex client hashes — 2.37x wall on a 40k-hash
negotiation. foldhash was already in the lockfile transitively (hashbrown).
Tested and REVERTED (kept as-is): moving the moka eviction unlink off the reactor
via spawn_blocking. The benchmark refuted it — on the local cache dir the
spawn_blocking dispatch (~20us) costs more than the inline unlink (~7us) it would
replace. See ROUND26.md §D2.
Adds bench_round26_{micro,diskio,hasher} (counting allocator / async wall / wall).
Verified: cargo fmt clean, cargo clippy --features bench -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
|
||
|
Claude
|
e8e4ef4b15 |
perf(round25): in-place encrypted decrypt, dedup hash move, dead folder-Query, playlist N+1 fold, contact vcard over-fetch
Five benchmark-gated optimizations from a fresh six-way audit (benches/ROUND25.md), each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE: - M1 EncryptedBlobBackend::decrypt_bytes: replace split_off (a fresh Vec + full ciphertext memcpy on every decrypted chunk, contradicting its own "in place" doc) with in-place detached decrypt + a zero-copy Bytes::slice past the nonce. Peak RAM per read drops from ~2x to ~1x the payload (-262KB/op at 256KiB; scales with blob size). Plaintext byte-identical; tamper/wrong-key tests pass. - M2 delta commit: move-unzip the owned chunk list instead of a third per-occurrence hash clone (-4000 allocs on a 4000-chunk commit). - M3 folder ZIP download: drop the dead Query<HashMap> extractor it never read (byte-identical response; 5->0 allocs/request). - Q1 public-playlist listing: fold the per-playlist COUNT(*) N+1 into one LEFT JOIN ... GROUP BY via a new inherent repo method (101 -> 1 round-trips, 36x wall on a 100-playlist page). - Q2 contact REST listings (paginated/search/by-group): stop over-fetching the multi-KB vcard TEXT the ContactDto discards, via a shared lite row mapper and narrowed SELECTs (6.4x wall on 1000 contacts with 8KiB vcards). The whole-book vCard export and CardDAV sync paths keep the column. Adds bench_round25_micro (counting allocator tracking count+bytes) and bench_round25_queries (live Postgres), both with equivalence gates and a rollback exit(1). Verified: cargo fmt clean, cargo clippy -D warnings clean, cargo test --lib --features bench = 529 passed / 0 failed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT |
||
|
Dionisio Pozo
|
0d82632f92 |
Merge pull request #629 from EdouardVanbelle/refactor/front-resource-list
Refactor(front) restore legacy frontend features |
||
|
Edouard Vanbelle
|
931e27d09c | fix(resources): wire missing created_by and updated_by | ||
|
Claude
|
ffb536e0ae |
perf: round 24 — download_zip per-item authz+metadata N+1 → batch (validated authorization pass)
The ROUND23-deferred download_zip N+1, given its own validated pass. The individually-selected files were authorized + fetched one at a time via get_file_with_perms (require + get = 2 serial round-trips/file) before any streaming — a 200-file selection was 400 serial round-trips. AFTER routes the whole multi-select through the new FileRetrievalService::get_files_by_ids_with_perms: one check_files_read_batch (the PgAclEngine resolves every file's drive in ONE query and primes the resource->drive cache) + one get_files_by_ids. 2N round-trips -> 2. Authorization is unchanged and still enforced BEFORE any ZIP entry is written: - add_file_entry_streamed writes the entry header (the filename) before it opens the authorized stream, so the pre-filter is load-bearing — a denied file must never reach it or its name leaks into the archive. AFTER a denied/missing id is absent from the authorized map and is skipped in the same input order, exactly as the old loop skipped a denied get_file_with_perms; it never reaches the entry write. The authz moved from a per-file require to one batch check EARLIER in the same function, not into or after the stream. - The stream open keeps its own per-file Read check (now a primed-cache hit) + Recents recording; check_files_read_batch is documented + gated as identical to looping require. Because the change is authorization-sensitive, the gate is the security property itself. bench_round24_zip_authz drives the real PgAclEngine over a seeded, interleaved mix of owned (granted drive) + denied (other drive) + missing ids and asserts: the batch inclusion set AND input order are identical to the per-file require loop; the included set is exactly the caller's owned files; no denied or missing id is ever included (the authz-regression tripwire); and the batch fetch returns exactly the owned files. Latency (cold, 600-item 1/3-owned selection): 559 -> 267 ms (2.10x; the realistic all-owned selection is O(1) -> a larger win). See benches/ROUND24.md. The folder selections are left as-is (root counts are small and there is no check_folders_read_batch primitive to batch through). Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0 failed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo |
||
|
Claude
|
1ec7030cc7 |
perf: round 23 — Postgres query-shape pass: typed JSONB decode, drive-policy borrow-deserialize, user-profile join!, subject-group CTE reuse, dedup unzip
Benchmark-gated, same rule as ROUND2-22: BEFORE/AFTER with a value-equivalence gate and rollback-on-regression. Two harnesses — bench_round23_micro (no Postgres; deterministic allocation gate) and bench_round23_queries (live Postgres; p50 latency + strict equivalence gate against seeded fixtures). See benches/ROUND23.md. - J1: contact_pg_repository::row_to_contact (+ the inlined contact_group sibling) decode the 3 JSONB columns via sqlx::types::Json<T> (one from_slice pass) instead of row.get::<serde_json::Value> + from_value (a throwaway Value DOM per column, walked a second time). Per contact row of every list / multiget / CardDAV sync. Micro 84 -> 33 allocs/op (2.15x); PG 3794 -> 2360 ns/contact (1.61x) on 500 real rows. - J2: DrivePolicies::from_value deserializes straight from the borrow (T::deserialize(&Value)) instead of from_value(value.clone()) — dropping the full-DOM clone on every drive-policy read (move/copy, share, grant); one-line body change, all 7 callers unchanged. Micro 5 -> 0 allocs/op (11.51x). - P1: get_user_profile overlaps the two independent caller+target reads with tokio::join! (self-case still a single fetch; caller-error precedence preserved via caller_res? first) instead of two serial round-trips. PG 577 -> 312 us/call (1.85x). - G1: subject_group remove_member computes the child's transitive-user recursive CTE once and reuses it for both the would-empty pre-check and the cache invalidation, instead of running the identical CTE twice (the edge delete is above the child, so its descendants can't change). PG 829 -> 412 us/removal (2.01x). - U1: dedup_service (store_loose_chunks final registration + the ingest run_rollback) reshapes the owned, dead-after Vec<(String,i64)> via into_iter().unzip() instead of cloning every 64-byte hash for the sync_blobs(&[String]) + UNNEST bind. Micro 256 -> 0 hash clones. Verified: cargo clippy --features bench --all-targets -D warnings clean, cargo fmt --all --check clean, cargo test --lib --features bench = 529 passed / 0 failed. The PG benches run against a local PostgreSQL 16 (schema applied from migrations/); every equivalence gate passes. The download_zip per-item N+1 (the audit's highest raw-latency candidate) is deferred to a dedicated pass: its fix moves the sole authorization inside the stream call, so it needs an AuthZ-ordering + anti-enumeration proof, not a perf banner. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo |
||
|
Claude
|
992bdae898 |
perf: round 22 — hot-GET HeaderMap borrow, native-WebDAV/CalDAV etag borrowed quotes, FileDto content_hash move, CalendarEvent stamp, ShareItemType case-fold
Benchmark-gated, same rule as ROUND2-21: every change ships with a
BEFORE/AFTER counting-allocator benchmark and a byte/-value equivalence
gate; an AFTER that fails to reduce allocations exits non-zero (rollback).
See benches/ROUND22.md and examples/bench_round22_micro.rs. All arms
no-Postgres.
- H1: the hot GET handlers (get_thumbnail, download_file, list_files_query,
list_photos, NextCloud preview, public-share download/access) take
`req: Request` last and read `req.headers()` by borrow instead of axum's
HeaderMap extractor, whose FromRequestParts impl clones the whole request
header table just to read 1-3 headers (the ROUND14 §A4 middleware pattern,
finally propagated to the handlers). 2 -> 0 allocs/req · 9.95x wall.
- W1: native WebDAV write_etag_quoted — the etag emitter for every /webdav/
PROPFIND row (per file AND per folder, up to 500/page) — emits the quotes
as borrowed pre-escaped " text events instead of escaping a "{etag}"
String (the ROUND20 §C1 / ROUND21 §R4 pattern). 3 -> 0 allocs/row.
- C1: CalDAV getetag routed through a shared write_quoted_etag helper across
all 5 sites (3 per-event + 2 per-calendar); the now-dead etag: &mut String
buffer threaded through write_event_response/standard/requested props + the
two per-page buffers removed. 2 -> 0 allocs/row.
- D1: FileDto::from reuses the moved parts.blob_hash instead of cloning it
via the content_hash() getter (the ROUND19/20 move-not-clone sweep missed
it — hash/etag are read before into_parts()). Per file row of every
listing. 1 -> 0 allocs/row.
- E1: CalendarEvent::update_time_range/update_all_day stamp timed
DTSTART/DTEND via fmt::compact_ical_utc stack render (chrono fallback out
of range) instead of the %Y%m%dT%H%M%SZ strftime interpreter. 4 -> 0.
- S1: ShareItemType::try_from uses eq_ignore_ascii_case instead of a
throwaway to_lowercase() String. 1 -> 0 allocs/parse.
Verified: cargo clippy --features bench --all-targets -D warnings clean,
cargo fmt --all --check clean, cargo test --lib --features bench = 529
passed / 0 failed (incl. the OpenAPI-spec-validity test guarding the H1
utoipa-handler signature change).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
|
||
|
Claude
|
77f13ac643 |
perf: round 21 — CalDAV/CardDAV row-mapper pre-size, dedup hash-bind & digest-key dedup, CardDAV etag/BDAY emit, NC trashbin content-type
Round 21 of the benchmark-gated perf sweep. Six behaviour-preserving, allocation-reducing changes, each with a BEFORE/AFTER counting-allocator section in examples/bench_round21_micro.rs and a byte/-value equivalence gate; all six pass their deterministic alloc gate (a non-winning AFTER exits 1 = rollback). - R1: pre-size the 16 CalDAV/CardDAV row-mapper Vecs (+1 HashMap) with Vec::with_capacity(rows.len()) — the ROUND20 §I1 file-side pattern extended to the calendar/contact repos it deferred. 7 → 1 allocs/op. - R2: settle_batch binds a borrowed Vec<&str> instead of cloning every chunk hash into a Vec<String> (sqlx encodes &[&str] as text[] identically; favorites_pg_repository.rs:271 precedent). 33 → 1 allocs/op, 39x wall. - R3: store_loose_chunks keys its intra-request dedup set on the raw [u8;32] BLAKE3 digest and moves the hex on a duplicate (the ROUND17 §D2 pattern applied to the delta-upload sibling). 401 → 209 allocs/op. - R4: CardDAV getetag emits borrowed pre-escaped " quotes via a shared write_quoted_etag helper (ROUND20 §C1 pattern, all 4 CardDAV etag sites). 3 → 0 allocs/op. - R5: BDAY stamped via the new fmt::compact_date stack renderer instead of chrono's strftime interpreter (chrono fallback out of the 4-digit-year range; byte-identical, unit-tested vs chrono). 2 → 0 allocs/op, 10.5x wall. - R6: NC trashbin folder content-type via Cow::Borrowed instead of .to_string() on the constant (ROUND16 §M1 pattern). 1 → 0 allocs/op. See benches/ROUND21.md for the full write-up and the deferred-items list (HeaderMap-clone hot handlers, Query→typed-struct, WebDAV dead-props HashSet, and others surfaced by the audit that want their own validated pass). Validated: cargo fmt, cargo clippy --features bench --all-targets -D warnings, cargo test --lib --features test_utils (529 passed). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015gHVq5Wy2TzdWeSqtEmK6m |
||
|
Claude
|
867e1fe259 |
perf: round 20 — iCal/vCard parse allocs, owned-DTO moves, Result-collect pre-size, NC etag/favorites emit
Benchmark-gated (benches/ROUND20.md), same rule as rounds 2-19: every change ships with a BEFORE/AFTER counting-allocator micro-benchmark and a byte-value equivalence gate; a non-winning AFTER is rolled back (never applied). The rollback rule is encoded in the harness (GATE FAIL exit). All 8 sections pass. Reproduce: cargo run --release --features bench --example bench_round20_micro - A1 CalendarEvent iCal parse: replace the throwaway per-property HashMap<String,Vec<String>> (DTSTART/DTEND/RECURRENCE-ID) with a direct VALUE=DATE scan; prop_with_params kept #[cfg(test)] (6->2 allocs/event, 4.2x) - A2 UserDto::from: add User::into_parts and MOVE image (<=512 KiB data URI) + ui_preferences JSON instead of cloning on every /api/auth/me (27->14 allocs) - A3 parse_vcard: drop the per-line to_ascii_uppercase copy + the lines Vec; promote ascii_ci_contains to common::text and share it (8->1 allocs/contact) - A4 Calendar/AddressBook DTO: into_parts move incl. custom_properties map (18->10) - I1 file-listing repos: collect::<Result<Vec>>() size-hints to 0 and grows from capacity 0; pre-size with Vec::with_capacity (8->1 container reallocs, 4 sites) - I4 plaintext_stream: lazy emit iterator instead of eager Vec collect (43x wall) - C1 NC write_etag_element: borrowed pre-escaped quote events, no owned quoted String/escape re-alloc; byte-identical output (3->0 allocs/PROPFIND row) - C3 NC favorites REPORT: map.remove() move instead of get().clone() (~7 allocs/fav) Deferred (documented in ROUND20.md): NC oc:id/trashbin buffer reuse, I1 sibling CardDAV/CalDAV listing paths, Contact JSONB Json<Vec<_>> decode, dedup settle_batch &str bind, and a fast DoS-resistant hasher for hot trusted-key maps (needs a dependency decision). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JsJjcVX9RoN96DMa35Wqzd |
||
|
Claude
|
9754aecfa9 |
perf: round 19 — auth/WOPI/vCard/PROPFIND per-request & per-row alloc cuts
Benchmark-gated (examples/bench_round19_micro.rs, benches/ROUND19.md): every
section ships a BEFORE/AFTER counting-allocator arm with a byte/-value
equivalence gate and a GATE-FAIL-rollback exit. All eight pass. No Postgres.
- M1 verify_basic_auth cache key: blake3::hash(format!("{u}:{p}")) → incremental
Hasher (byte-identical key, 2→0 allocs on every Basic-auth DAV request)
- M2 WopiTokenService: prebuild Validation/DecodingKey/EncodingKey in new()
instead of per-call (mirrors JwtTokenService; 16→12 allocs/validate)
- V1/V2 vCard emit (contact_to_vcard/generate_vcard): FN fallback drops the
throwaway to_string, NOTE skips the escape copy for newline-free notes, REV
uses new common::fmt::compact_ical_utc stack renderer (11.5× vs chrono
strftime, 3→0 allocs); per-contact 9→4 allocs
- M4 trash_service::row_to_item_dto: move name/path/blob_hash out of the owned
row instead of cloning (3 clones/file row gone)
- M5 search cache key: Uuid::hyphenated().encode_lower stack buffer instead of
to_string (identical u64 key, 1→0 allocs/request)
- M6 streaming PROPFIND: reuse one href buffer across the page instead of a
format! per child (native + NC handlers; 192→3 allocs on a 64-child page)
- M7 nextcloud extract_url_user: return Cow instead of forcing into_owned
(zero-alloc on the common ASCII-username path)
common::fmt::compact_ical_utc added with chrono-parity unit tests (CASES +
60-year sweep). cargo fmt + clippy --all-targets clean; 526 lib unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ront9bk7YMoffVQkGG47gh
|
||
|
Claude
|
6bc09cb3b3 |
perf: round 18 — calendar-event in-place iCal edit, ResourceList incremental id-index
Two items from the ROUND17 deferred list, each benchmark-gated with a
BEFORE/AFTER equivalence gate and a rollback-on-regression check (ROUND2–17
discipline). See benches/ROUND18.md.
[C1] backend — CalendarEvent::update_ical_property / remove_ical_property
rewrote the ENTIRE ical_data body with format!("{}{}{}") on every call and
allocated two search needles per call. calendar_storage_adapter::update_event
fans a multi-field edit out into one call per changed field, so a full REST
edit paid one full-body (up to ~11 KB) allocation per property. The body is
now mutated in place (replace_range for an existing property, four insert/
insert_str for a new one, byte-identical spans) and the single "\nNAME:"
needle is built on the stack (the "\r\nNAME:" needle was redundant — the LF
form is its suffix). bench_round18_micro [C1]: 70 -> 2 allocs/op (68 fewer),
2.46x wall, emitted body byte-identical.
[F1] frontend — ResourceList itemIndexById rebuilt a fresh Map over the whole
accumulated list every infinite-scroll page (O(N)/page, O(N^2) drain) and,
being a new instance each page, re-fired the reap-stale effect (another O(N)
id Set/page). New ItemIndexBuilder extends a persistent Map with the fresh
page only and reuses the reference across appends; the reap-stale effect now
tests membership against it. round18.bench.test.ts [F1]: 40x50 drain 74.1 ->
6.4 ms (11.5x), deep-equal to the reference at every page, reference-contract
gate (same-ref append / new-ref rebuild).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoxFtikahM1N4PE5s3ZVH3
|
||
|
Claude
|
c207e66f4b |
perf: round 17 — dedup ingest/verify hash-clone purge, CardDAV vCard TYPE tokens
Targets the content-addressable dedup write path (the ROUND15-deferred "dedup_service hash-String re-allocations") from both ends — the streaming ingest loop and the delta-commit verification read — plus a CardDAV vCard micro-cut. Every change is benchmark-gated with a hard rollback rule; no PostgreSQL needed for any arm (benches/ROUND17.md). Backend (counting-allocator, examples/bench_round17_micro.rs): - D2 chunk-ingest (store_from_stream, the hottest write path — every chunk of every upload): the 64-char hex hash String was allocated 3x per chunk (to_hex + chunk_hashes clone + session_seen insert-clone, the last dropped on a duplicate). The intra-upload dedup set now keys on the raw 32-byte BLAKE3 digest ([u8;32], Copy, no heap) and the manifest push is branch-split so a duplicate moves the hex in: 3 -> 2 allocs/new chunk, 3 -> 1/duplicate. Measured 214 -> 149 allocs/op (1.14x wall) on a 64-chunk 1-in-2-dup batch; smaller/faster set too (32B inline keys vs 64B heap Strings). - D1 hash_chunk_sequence (delta-commit verification): took chunks by &[(String,u64)] and fed the backend stream with iter().cloned(), re-cloning every chunk hash a second time on top of the owned Vec the caller already built. Take the Vec by value + into_iter(): 65 -> 0 internal allocs/op, ~2.5us of clone work removed per verify. - V1 vCard TYPE tokens (contact_to_vcard + generate_vcard, 5 sites): each EMAIL/TEL/ADR TYPE= param used ty.to_uppercase() — a throw-away String per token per contact. New shared fmt::push_upper writes the upper-cased chars straight into the buffer (byte-identical to str::to_uppercase, unit-tested): 13 -> 5 allocs/op, 1.19x wall. Gates: each section asserts byte/-value equivalence (D2 the ordered manifest + sizes + write-set; D1 the removed clone is a pure copy; V1 the full vCard) and exits non-zero if an AFTER arm fails to reduce allocations. push_upper is unit-tested byte-equal to str::to_uppercase (fmt::tests). Verified end-to-end: cargo fmt clean, clippy --release --all-targets --features bench -D warnings clean, and the harness prints GATE PASS against the built release lib. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XMmt7vNETYUbEG3Hc17LDx |
||
|
Dionisio Pozo
|
a09569b5c2 |
Merge pull request #625 from AtalayaLabs/claude/performance-optimization-analysis-raoezl
Round 16: incremental lanes/contextMap builders & alloc cuts |
||
|
Claude
|
955f4a7b9f |
perf: round 16 — shares-lane & contextMap incremental builders, folder/href/disposition/preview alloc cuts
Finishes the route-level half of the O(N²/page) grouped-listing class ROUND15 fixed inside ResourceList, plus a backend CPU/alloc micro-pack. Every change is benchmark-gated with a hard rollback rule; no PostgreSQL needed for any arm (benches/ROUND16.md). Frontend (vitest): - F1 "My shares" lanes: the `lanes` $derived.by re-bucketed the whole accumulated grant list on every page and every grant edit. SharedLanesBuilder re-emits only the fresh page (fan-out + first-appearance header), reusing untouched lanes' array refs. 25.5x fewer emit calls, 8.8x wall, O(N²/page)→O(N). - F2 contextMap (trash/recent/favorites/shared-with-me): each rebuilt a fresh N-key Map, re-hashing every accumulated id, per page. primeContextPage holds a persistent SvelteMap primed per page (the shipped favoriteIds shape). 25.5x fewer entry calls, 7.2x wall. - Extracted the shared O(1) append test (isAppendExtension); F1's gate re-covers it. Backend (counting-allocator): - M1 folder display constants Arc::from -> intern_display (3 sites): 3 -> 0 allocs/row. - M2 build_content_disposition (every download + Range seek): 3 -> 1 alloc, 6x, 2.67x wall. - M3 nc_href (every NC PROPFIND/REPORT href): Vec+join+format -> one pre-sized buffer, keeping urlencoding::encode (byte-identical). 38 -> 27 allocs/op. - M4 NC preview fileId: collect-then-parse -> borrow-slice parse. 4 -> 0 allocs. Gates: sharedLanes/listContext.bench.test.ts, examples/bench_round16_micro.rs (GATE PASS all sections). Frontend: vitest 331 pass, svelte-check clean. Backend: clippy -D warnings clean, 524 lib tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0193NjactJVqfU32gxeJDj8m |
||
|
Edouard Vanbelle
|
d4ae23ef13 | chore: apply clippy recos | ||
|
Claude
|
3be85fa9f0 |
perf: round 15 — grouped-listing O(N²) rebucket, exif/reseed allocs, tantivy zero-hit snippet skip
Benchmark-gated, same rule as rounds 2–14: every change ships with a BEFORE/AFTER benchmark and an equivalence/safety gate; an AFTER that doesn't beat its BEFORE is rolled back. The rule is encoded per harness (GATE FAIL non-zero exit in the Rust examples, threshold expect() in vitest). F1 — Grouped listings (trash / recent / favorites / shared-with-me) re-bucketed the WHOLE accumulated list on every infinite-scroll page. ResourceSectionsBuilder (new, off the reactive graph) re-buckets only the fresh page and hands VirtualList the same rows array reference for untouched buckets. 50×50 (2 500-item) drain: 63 750 → 2 500 bucketOf calls (25.5×), 12.5 → 1.3 ms wall (9.9×); O(N²/page) → O(N). Deep-equal to the full-rebuild reference at every page for both a contiguous (date) and a non-contiguous (trash-by-drive) group-by; reference-stability + fallback gated. B1 — exif Make/Model: the display String was thrown away to allocate the trimmed copy; display_value_trimmed trims in place (drain + truncate), 2 → 1 alloc per field (8 → 4 allocs/op, 1.26×). B2 — content-index worker: text_extractor::supports (lowercases MIME + extension) was called twice per file per drain batch; classify once into a Vec<bool> and thread it through both uses. 256-file batch: 704 → 353 allocs, 34.5 → 16.7 µs (2.07×). B3 — tantivy: skip SnippetGenerator::create on a zero-hit content search (return Ok(vec![]) once top_docs.is_empty()); the per-hit loop was empty. 400-doc index: 1 575.6 → 1 237.2 ns (1.27×), widens with index size. Harnesses: examples/bench_round15_micro.rs, examples/bench_round15_tantivy.rs, frontend resourceSections.bench.test.ts; writeup in benches/ROUND15.md. Also normalizes two round14 bench examples that were committed unformatted (cargo fmt --all). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012o47jSrtL7xuNGTHXmtiYL |
||
|
Claude
|
c930f865b0 |
perf: round 14 — faces narrow projection, auth per-request allocs, CalDAV emit buffers, frontend set churn
Benchmark-gated (benches/ROUND14.md); every change ships a BEFORE/AFTER
benchmark with an equivalence gate and is rolled back on regression (the
rule is encoded as a GATE FAIL exit / threshold expect).
Backend
- Q1 faces_for_file → narrow face_boxes_for_file(id, person_id, bbox) with the
caller filter pushed into SQL: drops the 2 KiB embedding BYTEA + 6 unused
columns per face. 15-face lightbox open 0.312→0.219 ms, 32 KB→840 B/req.
- A1 cookie auth uses the borrow-only extract_cookie_str (already backs CSRF)
instead of extract_cookie_value's owned String: -1 alloc/cookie request.
- A2 compute_relevance ASCII case-fold fast path vs name.to_lowercase() per
result row (Unicode fallback preserved): 1.40x, 12→3 allocs/page.
- A3 sub pre-parsed to Uuid at decode time (TokenClaims.sub_id) vs re-parsing
the 36-char claim on every request incl. cache hits: 22.7→0.7 ns.
- A4 auth + NextCloud middlewares borrow request.headers() instead of taking
axum's HeaderMap extractor (a full map clone): 2→0 allocs/authed request.
- A5 CalDAV getlastmodified via the stack rfc2822_utc (byte-identical to
chrono) vs a per-event to_rfc2822() heap String: 5→0 allocs.
- A6 CalDAV per-event href + quoted etag written into reused page buffers vs a
fresh format! pair per event: 3.48x, 240→6 allocs/40-event page.
Frontend
- F1 t() shares one frozen EMPTY_PARAMS for the no-interpolation call forms vs
a throwaway {} per call: -1 alloc/call.
- F2 favorites favoriteIds is a persistent SvelteSet with per-page add (clear
on reset) vs a brand-new set over the whole accumulated list each page:
22.3x over a 40-page drain (O(N^2)→O(N)).
Verified: cargo check --all-targets, cargo clippy -D warnings, both bench
packs (GATE PASS), frontend npm run check + vitest (4/4). ROUND14.md also
records the investigated-but-deferred backlog (music N+1, contact vcard
over-fetch, CachedBlobBackend syscalls, ResourceList.sections builder, etc.).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PymgCdK78NzUF3oRAQCJfN
|
||
|
Claude
|
f58d72a780 |
perf: round 13 — grouped-view virtualization, notification/login query narrowing, HTTP dedup, locale precompute
Benchmark-gated (BEFORE/AFTER + equivalence/safety gate per change), same discipline as rounds 2-12. Full write-up in benches/ROUND13.md. Shipped: - V1 Grouped views windowed (files route + ResourceList). The grid arm was the last unwindowed path (trash is grouped-by-default in grid): each swimlane now feeds its own VirtualList, outer container a flex stack. vitest gate: 800-item grouped grid mounts <120 .file-item (was 800). - Q1 get_users_by_ids drops the <=512 KiB avatar image + ui_preferences JSONB (notification path never reads them). 30-member fan-out 8.60 -> 0.25 ms (34.3x), ~7.7 MB off the wire. - Q2 Login provisioning is_empty() -> SELECT EXISTS for calendar + address book (every login). 0.193 -> 0.170 ms, widens with owned-row count. - Q3 Recent-access prunes only when the upsert inserted (RETURNING xmax=0) — a re-access can't grow the set. 0.567 -> 0.324 ms (1.75x). - L1 Locale supported-codes precomputed once vs rebuilt per anonymous request. 616 -> 17.3 ns (35.7x), 18 -> 1 allocs. - H1 Duplicate /api TraceLayer removed (global stack already wraps it). 1.86 -> 1.42 us/request, -6 allocs. - H2 client_ip span field: borrow-only ClientIpDisplay vs owned String. 187 -> 173 ns, -1 alloc. Not shipped (discipline): the "media hooks read the blob 3x" lead was a correctness bug, not a perf dup — the raw-path metadata/faces readers resolve only for local+unencrypted+single-chunk blobs and silently produce nothing otherwise. Flagged for maintainers; routing through read_blob_bytes is a correctness fix (perf-neutral-to-negative), not a benchmark-gated perf change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB |
||
|
Claude
|
50eca0627f |
perf: round 12 — auth write-path narrowing, fused quota gate, moka blob-cache index, media single-read, sized listing JSON
Benchmark-gated round (benches/ROUND12.md; every change ships with a BEFORE/AFTER harness + equivalence gates, one candidate rejected by its own bench): DB / query shapes (bench_round12_queries): - NC sharee search: username-only projection instead of the 21-column row (incl. the <=512 KiB avatar) per match, + gin_trgm_ops indexes on auth.users for the leading-wildcard ILIKE (4.98x; 54.7x with index). - Password login: delete the redundant full-row update_user — create_session already stamps last_login_at in its own txn (4.45x per login). - Email-verified stamp: narrow conditional UPDATE (8.9x); OIDC repeat login now compares profile state in memory and issues ZERO queries when nothing changed (was: full 17-column rewrite per login). - Refresh rotation: revoke+insert+stamp fused into one transaction via new rotate_session port method (1.18x). - WOPI CheckFileInfo / authorize_wopi_access: require(Read) + get_file + check(Update) overlapped with tokio::join!, original result precedence (cold 1.34x). - Upload quota gate: user-envelope + drive-cap checks fused into ONE round-trip (check_upload_quotas) — the NC chunked PUT pays this per chunk (1.81x, 2 -> 1 queries/chunk); shared verdict evaluators keep error shapes byte-identical. CPU / allocs (bench_round12_micro): - sized_json: pre-sized listing serialization replacing axum Json's 128 B seed + doubling-realloc chain on files/folder-resources/photos/search responses (1.40x, 13 -> 2 allocs per 500-row page; byte-identical). - Security headers: 4 SetResponseHeaderLayer folded into the CSP middleware pass (5 layers -> 1; 1.43x per request, -26 allocs; header set gated byte-identical incl. 304s). - Media capture-metadata: single-read extraction — nom-exif now parses the buffer kamadak already read (zero-copy Bytes) and videos open once with a kind() dispatch; per-image opens 2-3 -> 1 (1.44x warm geomean, 1.6-3.2x cold cache; extraction outputs gated identical incl. the MIME-mislabel track fallback). - Chunked-upload session ops: owner gate folded into the operation's own DashMap lookup + stack-encoded uuid compare (5 -> 3 lookups, -2 allocs, 1.28x per chunk). Blob cache (bench_blob_cache_index + round-3 regression guard): - CachedBlobBackend index: tokio::sync::Mutex<LruCache> -> moka::sync::Cache with byte weigher. The mutex serialized every cached chunk read and scaled NEGATIVELY (2.08 -> 1.07 Mops/s from 1 -> 2 readers); moka probes are lock-free (2.17x at K=2). Byte budget now enforced by moka (manual current_size + collect_evictions machinery deleted); eviction listener unlinks size-evicted files only (Replaced entries keep their file — gated). Single-flight miss gate unchanged (16 concurrent misses -> 1 fetch re-verified via the round-3 harness). - put_blob now populates the cache BEFORE the inner backend consumes the source file (the old order failed 100% of the time — local renames, S3/Azure delete the source — so the first read after a whole-file put re-downloaded from the remote); inner-put failure invalidates the entry. Frontend (vitest gates): - List-view thumbnails request the 150px icon rendition instead of 400px preview into a 40px slot (~7.1x fewer pixels, ~4-5x fewer bytes per thumbnail across list views); grid keeps preview. Rejected by its own bench (kept as evidence in bench_round12_micro §2): - Single-pass compression predicate: the monomorphized And-chain already costs ~4.6 ns / 0 allocs total; the fused node measured within noise. New migration: 20260719000000_users_search_trgm.sql (trgm indexes). Deferred with prepared design: grouped file/grid view virtualization (single-VirtualRows flatten, the photos pattern) — next round's headline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB |
||
|
Claude
|
b01633791b |
perf: round 11 finale — benchmark verdicts, geo min(uuid) rollback, ROUND11.md
- benches/ROUND11.md: final BEFORE/AFTER numbers for all 21 micro sections, the 5 query sections, the 4 log-writer arms, the SPA gates, and the two cross-round regression guards (bench_row_path 2.52x, bench_dto_map — both byte-identical) - ROLLBACK (gate-caught): min(fm.file_id)::text geo-cluster cast — PostgreSQL has no min(uuid) aggregate; the bench section now reproduces the rejection and the per-row-cast original stays - REJECTED (bench-measured): tracing-appender non-blocking writer — slower than sync on fast sinks (1.41M vs 0.99M ev/s, worse tail) and the slow-sink drain gate showed shutdown tail-loss risk for the audit channel; dep moved to dev-dependencies (harness only) - RateLimiter final form: lock-free get + insert (8.0 → 6.0 allocs, wall neutral; and_upsert_with variant rejected at 2 365 ns) - fix: TrashedItemParts insertion had stolen TrashedItem's derive line (caught by clippy --all-targets) - grant_role enum values corrected in the queries bench Validation: cargo fmt + clippy --all-features --all-targets -D warnings clean; cargo test --workspace 524 passed / 0 failed; frontend npm run check 0 errors + vitest 306 passed (1 pre-existing round-7 wall-clock gate flaked only under concurrent Rust-build CPU contention; passes in isolation) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7 |
||
|
Claude
|
221c1f31b0 |
perf: round 11 — StoragePath joined-only, classifier fusion, memoized bodies, query-shape pack, SPA fine-grained stars
Backend (each change benchmark-gated with BEFORE replicas + equivalence gates; see examples/bench_round11_micro.rs, bench_round11_queries.rs, bench_log_writer.rs and benches/ROUND11.md — final numbers land in the follow-up doc commit): - StoragePath re-representation: single canonical joined String, segments derived on demand; File/Folder drop the duplicated path_string field (4000→1000 allocs per 500-row listing page) - Display classifier fusion: classify_display shares one stack-lowered extension across the three decision trees; call sites in FileDto, folder/favorites/recent handlers, trash, path-resolver (+ interning where Arc::from was still used) - /status.php and /openapi.json memoized into OnceLock<Bytes> (openapi rebuilt a 171 KiB spec per request: 2.8 ms → 18 ns) - NC upload-session PROPFIND: write! + pre-sized body + stack RFC2822 dates (2.3-2.6x, 2582→772 allocs at 256 chunks) - REST download: dead FileDto clone removed (capture mime/size + move) - CalendarEventDto/TrashedItem into_parts moves (11 KiB ical_data memcpy gone per CalDAV row); CardDAV getlastmodified stack render - 4xx path: borrowed ErrorResponse serialize, ErrorKind::as_str, not_found/already_exists clone kill - vCard emit via write!; search page moved out with into_iter skip/take; content-hit UUIDs parsed once; group last-user check via HashSet - RateLimiter: lock-free get + insert (and_upsert_with variant REJECTED by benchmark); CSRF token borrow-compare + borrowed cookie extraction - Thumbnail/preview ETags built from as_str (Debug-identical bytes) - Encrypted backend: encrypt_in_place_detached single-buffer write path, chunk-sized reserve in collect_stream; retry labels made lazy - PG: deferred upload registration 3→1 round-trips (persist_file CTE template); direct_grant_cache for Calendar/AddressBook/Playlist authz (single-flight + set_role/clear_role invalidation); expand_user tokio::join!; geo clusters min(uuid)::text; recluster face assignment batched into one UNNEST update - People recluster cosine: norms precomputed once (bit-identical gate) - NC capabilities poll logs demoted to debug; tracing-appender dep added for the log-writer benchmark Frontend: - ResourceList.selectedEntries O(N)-per-toggle → id-index projection O(k log k); favorites/recent consume the batchToolbar snippet param and drop their duplicate filter + dead selectedIds mirror - Recent: star state via new favoriteIds prop — a star click no longer rebuilds all N entries - admin timeAgo >30d fallback uses the cached Intl.DateTimeFormat - vitest gates in src/lib/components/round11.bench.test.ts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7 |
||
|
Claude
|
c51af68432 |
perf: round 10 — auth alloc purge, parent-herd batching, query-shape pack, NC 304s
Benchmark-gated (benches/ROUND10.md; every change carries a BEFORE/AFTER harness with equivalence/safety gates — two designs were rejected or rewritten by their own benches before adoption): - Auth hot path: TokenClaims/CurrentUser display fields to Arc<str>, role to inline SmolStr end-to-end (Bearer, cookie, Basic-auth cache) — 4→1 allocs per authenticated request, 3→0 per warm DAV request; JWT Encoding/Decoding/Validation built once. - Cold shared-album herd: leader-inline parent batching in PgAclEngine (+ cascade try_get_with single-flight) — 100→2 parent queries per 100-thumb cold herd, herd wall 1.9x, sequential + warm paths unchanged, all ROUND8/9 safety gates plus new herd-equivalence gates. - Query-shape pack: share download double-fetch 2→1 (2.18x), contact-group COUNT(*) 14.9x, save_faces UNNEST 3.9x, playlist reorder UNNEST 63.7x (now atomic), search files∥folders join! 1.45x, move drive-lookup join! 2.14x, trash partial (drive_id, trashed_at) indexes, CalDAV event-gate narrow read, favorites/recents binary-decode port, dead count_files removed. - NC surface: preview + avatar honour If-None-Match (e2e: 5 KB and 197 KB → 0 bytes per revalidation), avatar WebP→PNG transcode memoised, PROPFIND/trashbin integer+date emits on stack formatters, folder-header enrichment join!, chunk-PUT retry stat folded into create_new open. - common::fmt integer rendering rewritten on the std 2-digit LUT after the round's own bench caught the div-loop losing to to_string (16.1 ns vs 22.5; speeds every prior-round call site). - Micro-pack: WebDAV scope probe borrow-only, ShareService base_url snapshot, cookie_secure OnceLock, Arc'd AES-GCM cipher, stack request-id, tantivy analyzer clone dropped. - SPA: search stale-guard + AbortController (10→1 completed round-trips, stale-clobber gone), getFolder in-flight dedup, gridColumns matchMedia hoist (10k→0 style reads). Backend: cargo fmt + clippy -D warnings clean, 524 tests green. Frontend: npm run check clean, 301 vitest green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018DdM7V7M3QPW7HEHg3gLov |
||
|
Claude
|
fdf445d2b0 |
perf: round 9 — decorator PUT reactivation, session/search/dedup alloc purges, PROPFIND join!, folder-level cascade
Benchmark-gated round (benches/ROUND9.md): every change carries a BEFORE/AFTER bench with equivalence/safety gates; verdicts below are from the committed harnesses on 4 cores / local PG 16. Backend: - Blob decorators (Retry/Cached) now forward put_blob_from_bytes_unsynced + sync_blobs — the trait default had silently reinstated HEAD-before-PUT per chunk on decorated remote stacks, undoing ROUND3 §8. Full production stack: 500 probes -> 0, 1.9x wall at 10 ms RTT (bench_s3_put §3). - NC PROPFIND per-page enrichment triple (favorites / oc:fileid / dead props) overlapped with tokio::join!: 2.07x local, 2.86x at 5 ms RTT (bench_nc_enrich_join, injected-latency decide-by-bench). - Search enrichment consumes its DTOs and carries the interned Arc<str> display fields end-to-end (SearchFileResultDto type change, OpenAPI shape preserved): enrich_file 2.0x, 11.6 -> 2.2 allocs/row; the NC REPORT conversion stops re-running all three classifiers per row (bench_search_enrich). - NC session Arc end-to-end: SharedNcSession extractor (8 -> 0 allocs), Arc<FolderDto> chroot cache (4 -> 0/hit), single shared Arc<CurrentUser> + lazy span render (11 -> 6/build) (bench_nc_session). - Storage micro-pack: atomic create_new chunk writes (2.1x fresh), stream_chunks over the manifest Arc (4097 -> 0 allocs/read incl. the Range path), manifest single-flight (herd 64 -> 1 loads), hex_lower for chunk Content-MD5 (18 -> 1 allocs) (bench_storage_micro). - OCS capabilities memoized into OnceLock<[Bytes;2]>: 237x, 102 -> 0 allocs/poll, byte-identical (bench_capabilities_static). - Drive::is_empty COUNT(*) sum -> EXISTS: 34.4x on a 100k-file drive (bench_drive_is_empty). - favorites/recents row-map ROUND7 port: path/name/blob_hash moved, -2.75 allocs/row (bench_resource_row_map §2). - Folder rows decode binary UUIDs (ROUND6 §10 port): 1.03-1.07x page fetch, honest verdict incl. one noise-band wash documented (bench_folder_uuid_decode). - Authz: file cascade decision decomposed into memoized folder-level decision + direct-grant lookup (ROUND8 deferred item). Cold shared-album first view 592 -> 418 µs/thumb; warm path unchanged; safety gates incl. new direct-grant sibling isolation, revoke-flush re-verified, full integration authz suite green (bench_thumbnail_cascade_cache). Frontend (vitest gates committed beside the code): - resolveLabel/resolveRecipient O(directory) scan -> id-keyed Map: 13.9x (recipients.bench.test.ts). - ResourceList selection-prune effect skips when nothing is selected (100 -> 0 Set builds per drain) and the photos timeline reads a listener-fed mobile flag instead of matchMedia per recompute (listDerives.bench.test.ts). Verification: cargo fmt + clippy --all-features --all-targets -D warnings clean; 524 unit + 554 integration (--cfg integration_tests) tests pass; frontend npm run check clean with 293 vitest tests green. Deferred with rationale in ROUND9.md: CalDAV authz-before-fetch reorder (maintainer sign-off), per-page batched parent resolution, JWT-claims Arc<str>, batch_operations signature widening. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDc9VtXvskJ6dnMRraSndn |
||
|
Claude
|
79b94126be |
perf(authz): round 8 — cache the File/Folder grant-cascade decision for shared-album thumbnails
get_thumbnail_impl runs require_permission(Read) on every request. For a drive member that's a drive_role_cache hit, but a shared-album recipient — granted a folder (the album), not drive membership — fails the drive-role precheck and falls through to file_cascade_grant_exists (a role_grants ⋈ folders lpath ancestor query), once per file. Browsers revalidate immutable thumbnails constantly, so the same (recipient, file, Read) decision was recomputed on every thumbnail of every view — ~100 grant queries per 100-photo album per navigate-away-and-back. New cascade_grant_cache ((Subject, Resource, Permission) → bool, 30 s TTL) memoises that decision. The check is NEVER skipped — the ordering is unchanged, authz still runs on every request; only the result is cached, and only after the drive-role precheck fails (so a later drive grant can't be shadowed by a stale entry). Invalidation mirrors drive_role_cache's convention: explicit invalidate_all on every File/Folder set_role/clear_role (immediate revoke on the direct share path), 30 s TTL for the indirect paths (group membership, moves, expiry) "rather than a deep invalidation tree". Bench (bench_thumbnail_cascade_cache) with hard safety gates — recipient allowed, outsider denied, and a clear_role revoke denies the very next check (proving the grant-write flush): 100-photo album revalidation 2576 → 2.70 µs/thumb (~950x), 257.6 → 0.27 ms/view. Validated against the full --cfg integration_tests authz suite (554 tests) + 524 workspace tests, clippy -D warnings clean. Deliberately not done: moving authz after the 304/cache short-circuit (a security-posture change — a revoked user could serve cached thumbnails). With the decision cached, the authz on the 304 path is now a memory hit, so the "zero DB work on a 304" intent is restored without weakening the check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA |
||
|
Claude
|
7626dc95c1 |
perf: round 7 — photos timeline O(N²)→incremental, range-seek authz duplication, resources row-map clone
Benchmark-gated (equivalence + BEFORE/AFTER; results + reproduce commands in
benches/ROUND7.md):
- Photos timeline re-grouped + re-laid-out the whole accumulated library on
every 60-item page (both `groups` and `photoRows` were $derived over the
full list), Σ ≈ O(N²/60) main-thread work during a scroll. Pages arrive
newest-first so grouping is append-only: the new PhotoTimeline
(lib/utils/photoTimeline.ts) re-buckets only the fresh page and re-lays-out
only changed groups, reusing untouched groups' cached rows, falling back to
a full rebuild on any config/deletion/non-append change. The pure
buildPhotoRows is the verbatim reference the gate holds it equal to at every
page. 50×60 drain: 76 500 → 3 000 grouping ops (25.5x), 23.0 → 2.2 ms
(10.6x).
- Range downloads paid authz + access-notify twice: download_file_impl
resolves the file via get_file_with_perms, then the Range branch re-ran
require_file + notify_file_accessed per request. Media/PDF viewers fetch
exclusively via Range (one request per seek), so every seek in a scrub
re-authorized an already-cleared file. Now routed through the non-perms
get_file_range_preloaded (matching the share-landing + WebDAV range paths);
the unused _with_perms range method is removed. The request-level gate still
denies before the branch runs (bench asserts member granted, outsider
denied). Per seek removed: WARM 0.67 µs, COLD 1362.66 µs — a grant-cascade
drive-resolve query per seek for a shared-drive recipient on a cold cache.
- /api/folders/{id}/resources row→DTO mapping cloned row.name into the DTO
though the row is owned; folders move it (fixed icons), files compute the
name-derived icon/category classes first then move it. 500-row page:
10.004 → 9.004 allocs/row (500 clones removed), output identical.
Deferred with rationale in ROUND7.md: thumbnail ACL-before-304 (security
posture — needs a security review, not a perf tweak), batch_operations
Arc<str>→String widening, list-view O(N²) on smaller lists, and the serial→
join! pairs (decide-by-bench with injected latency, per the round-6 rejection).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
|
||
|
Claude
|
9729f033b2 |
perf: round 6 backend — CardDAV cursor streaming, borrowed NC id chain, binary UUID decode, one-alloc hex
Benchmark-gated (equivalence + BEFORE/AFTER in examples/bench_*, results and reproduce commands in benches/ROUND6.md): - CardDAV whole-book REPORT + depth-1 PROPFIND stream through a PG cursor (stream_contacts_by_book, 500-contact pages) instead of materialising every vCard twice: 8 000 contacts TTFB 37.4 → 7.6 ms (4.9x), peak heap 19.0 → 7.0 MiB (2.7x), wall -23%; REPORT and PROPFIND byte-identical to the buffered writers. - NC numeric-id chain fully borrowed: get_or_create_file_ids/folder_ids take &[&str] and return HashMap<Uuid, i64>; batch_resolve_ids callers (PROPFIND pages, REPORT, trashbin, OCS search) pass id slices and look up via nc_id_of. 2.006 → 0.006 allocs/child (334x), 1.53x wall per 500-child page. batch_check_favorites binds &[&str] as text[]. - file_blob_read_repository listing SELECTs drop id::text/folder_id::text server casts: rows decode binary Uuid (16 vs 36 bytes on the wire) and render once in row_to_file. A/B on 500-row pages: 1.225 → 1.044 ms mean (1.17x), p95 1.686 → 1.345 (bench_uuid_text_cast; single-row, param and min() sites left as-is deliberately). - IncrementalHasher::finalize_hex renders through common::fmt::hex_lower instead of one format! per digest byte: 18 → 1 (md5) / 35 → 1 (sha256) allocs per chunk finalize, 14-15x wall. - Share landing overlaps the access-count UPDATE with the unlock fetch via tokio::join! (one round-trip off every public link hit). - REJECTED by benchmark and reverted: try_join_all fan-out of the batch-favorites authz pre-check — 42.6 → 56.4 ms cold, 0.15 → 0.23 ms warm against local-socket PG (bench_favorites_authz kept as evidence). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA |
||
|
Claude
|
63cf6646d0 |
perf: round 5 — CalDAV cursor streaming, SPA interning gaps, NC href prefix, per-request micro-allocs
Seven benchmark-gated changes (benches/ROUND5.md; BEFORE/AFTER bench + equivalence gate each, rollback rule as ROUND2-4 — two intermediate CalDAV shapes measured worse and were themselves rolled back before shipping): - CalDAV whole-calendar responses (REPORT no-range/sync-collection, depth-1 collection PROPFIND, .ics GET): buffered double-residency → ONE window-ordered scan (MIN(start_time) OVER (PARTITION BY ical_uid)) streamed through a PG cursor, pages cut at UID boundaries. TTFB 23.3→11.0 ms (2.1x), peak heap 14.2→8.0 MiB at 4k events / 45→24 MiB at 12k, wall +9-15% (documented trade, ZIP-streaming class); both multistatus and ICS byte-identical to the buffered output. Rejected shapes kept in the doc: per-page GROUP-BY keyset (3-4x wall) and per-uid ANY hydration (~20 µs/index descent). - SPA listing interning gaps: folder/recent/favorites resources handlers (and the WebDAV pseudo-root) called raw Arc::from per row for the closed display set ROUND3 interned — now intern_display/intern_mime, 4→0 allocs/row, byte-identical Arc contents. - NC PROPFIND child hrefs: username + parent path encoded once per request instead of per child (543→165 ns/row, 13→4 allocs); native WebDAV href drops its intermediate encode String. - suggest enrichment: entity clone + field re-clones per keystroke row → consume + move (166.5→126.8 µs/200 rows, 20→7 allocs/row). - list_readable_by returns the cache's Arc (246→128 ns warm hit, 4→0 allocs) — deep Vec clone per DAV-selector request removed. - CardDAV REPORT: borrowed props, reused href buffer, exact-size etag quoting (3.04→2.34 ms per 5k-contact getetag poll). - Auth span records: user_id.to_string() per request ×3 → tracing::field::display. Checks: cargo fmt, clippy --all-features --all-targets -D warnings, cargo test --workspace (523 passed). Follow-ups (CardDAV streaming, &[&str] id batches, ::text UUID casts A/B, share-landing join) recorded in benches/ROUND5.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA |
||
|
Claude
|
12dc648cff |
perf: round 4 — one-pass row paths, drive-selector cache, CalDAV single-parse, streamed Azure, batched hydration
Nine benchmark-gated changes (benches/ROUND4.md; every one ships with a BEFORE/AFTER bench + equivalence gate, rollback rule as ROUND2/3): - Row→entity path build: one-pass StoragePath::from_folder_and_name / from_joined + normalize_storage_name_owned + alloc-free Display — 743→417 ns/file-row (1.78x), −5 allocs/row on every listing surface. - WebDAV drive-selector: per-user readable_cache (single-flight, 30 s TTL, explicit invalidation incl. membership + group changes) replaces the grants join per request — 441 µs → 0.8 µs (~550x), 0 queries warm. - CalDAV from_ical/update_ical_data: 8 full IcalParser runs per VEVENT → 1 (7.1x per PUT, 4.4x on 50-event imports); alloc-free split_vevents, chunk scan without the whole-body uppercase copy (1.4x), borrowed-key UID grouping (1.3x), REPORT props no longer cloned. - PROPFIND emit: partition Vecs dropped (single-pass 404 list) + stack rendered RFC 3339/2822 dates, sizes, quoted etags (common::fmt, chrono-byte-identical, sweep-tested) on both DAV surfaces — 1.22x per page, 17.9→12.0 allocs/row. - Grant-listing hydration: calendars/address books/playlists batch hydrate via = ANY($1) — 15 serial queries → 1 (~13x per sync poll). - user-flags cache: get→insert → try_get_with single-flight (32→1 queries per cold herd). - Azure downloads: whole-blob Vec buffering → streamed SDK pages — TTFB 349→4 ms (87x), peak heap 480→1.9 MiB (254x) on 256 MiB blobs; new OXICLOUD_AZURE_ENDPOINT_URL override (Azurite/bench hook). - Face indexing: unbounded per-image tokio::spawn → core-count semaphore, permit before blob read — peak heap 1175→176 MiB (6.7x). Checks: cargo fmt, clippy --all-features --all-targets -D warnings, cargo test --workspace (523 passed) + --features test_utils. hurl API suite and dockerized integration DB not runnable in this environment — left to CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA |
||
|
Claude
|
cd4c62042a |
perf: keyset/LATERAL SQL shapes, auth+blob-cache single-flight, spool buffers, DTO interning
Round 3 of benchmark-gated optimizations (benches/ROUND3.md; every change gated by a before/after benchmark — an AFTER that did not beat its BEFORE was to be rolled back; none needed it. Equivalence gates assert identical row sequences / byte-identical output on every behavior-preserving rewrite): DB hot paths (local PG16, EXPLAIN-verified): - Web-UI listing (list_resources_paged): cursor pushed INSIDE the folders/files UNION-ALL branches as sargable row-value comparisons with per-branch ORDER/LIMIT + two partial expression indexes (folder_id, LOWER(name), id). 20k-entry folder: 26.6 -> 1.3 ms/page (19.5x); other sort modes at parity or better. New migration 20260918000000. [benches/LISTING-KEYSET.md section in ROUND3] - Photos timeline (list_media_files): per-drive CROSS JOIN LATERAL top-N on the timeline index, joins moved above the top-N. 50k-photo library: 97.4 -> 1.6 ms/page (55.7x). The old "LIMIT stops the scan early" comment was refuted by EXPLAIN. - PROPFIND sub-folders (both DAV surfaces): keyset list_folders_batch off idx_folders_unique_name replaces COUNT(*) OVER() + LIMIT/OFFSET (5k dirs: 79.7 -> 17.9 ms full walk, 4.5x). Concurrency: - Basic-auth cache single-flight (moka try_get_with): 8 concurrent DAV connections at TTL expiry paid 8 Argon2id runs (2.6 s CPU + 8x64 MiB); now 1 (300 ms). Failed verifications remain uncached. - CachedBlobBackend per-hash single-flight + unique tmp names: 16 concurrent cold readers = 16 full remote downloads racing truncating writes on ONE deterministic .tmp (corruptible cache); now 1 download (16x less egress, 2.8x wall on a shared link) and torn files can never be renamed into the cache. I/O and allocations: - Chunk-assembly reads 64K -> 512K buffers (2.3x, 8x fewer syscalls); chunk-spool writes via BufWriter 512K (5.6x, 32x fewer syscalls). - S3/Azure put_blob_from_bytes_unsynced overrides: dedup settle no longer pays a HEAD probe per new chunk (2 RTT -> 1, 1.8x); Azure stops copying every chunk (Bytes -> Body, -0.44 ms - 4 MiB alloc per 4 MiB chunk). - Entity->DTO mapping: Arc<str> interning of closed-set display fields + common MIMEs, 1-alloc etag/size formatting, FolderDto moves instead of clones. File row: 11 -> 4 allocs; folder row: 11.8 -> 1 (2.1x faster). - CardDAV REPORT: deleted dead per-contact vCard pre-generation and the O(N^2) uid scan whose result was discarded (5k contacts: 55.7 -> 5.7 ms, 9.8x); byte-identical XML asserted. - Search-results cache: byte weigher + 32 MiB budget (OXICLOUD_SEARCH_CACHE_MAX_BYTES) replaces the 1000-ENTRY cap that let ~300 MiB of enriched rows sit in RSS; read latency parity. - Dropped aws-config + aws-smithy-types (zero references; -82 dep-graph nodes, three SDK stacks gone from every build). tokio "process" is now an explicit feature (was enabled transitively by aws-config). Frontend: - Cached Intl.DateTimeFormat keyed by (locale, options) in formatDate and 4 sibling callsites: 20k dates 2612 -> 51 ms (51.6x); vitest gate asserts output identity across locales and a 3x floor. Validation: cargo fmt + clippy --all-features --all-targets -D warnings clean; 518 unit + 548 integration-cfg tests green; new-shape endpoints smoke-tested end-to-end over HTTP (all 5 listing sort modes with cursor walks, WebDAV PROPFIND Depth-1, photos timeline, Basic-auth DAV login); frontend npm run check clean, new vitest gates green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsU2qEzny3A8WQUEuMNCr |
||
|
Claude
|
82ee7da0d2 |
perf: serve ranges from RAM cache, stream ZIPs, overlap ingest settle, O(1) chunk gate
Round 2 of benchmark-gated optimizations (benches/ROUND2.md; every change gated by a before/after in examples/bench_round2.rs — an AFTER that did not beat its BEFORE was to be rolled back; none needed it): - Range requests (REST/DAV/shares) answered from the moka content cache for sub-10MB files: PG resolve + open/seek/read -> Bytes::slice. 256KiB seeks: 1,730/s -> 3.7M/s (p50 552us -> 0.15us). - Streaming folder/share ZIPs via tokio duplex: TTFB no longer scales with archive size (326ms -> 0.4ms on 192MiB corpus; total also faster). Content-Length dropped (size unknown up front). - NC chunked-upload per-PUT gate: O(k) directory scan+stat -> in-RAM per-session counter (lazy rebuild on cold start). 1,000-chunk upload gate cost: 33.1s -> 0.09s cumulative. - Delta download + commit-verify now use the CDC path's buffered(read_prefetch) read-ahead: 64-chunk drain at 5ms open latency 440ms -> 51ms; order preserved. - CDC ingest settles batches on a spawned task (depth-1 pipeline) so the source stream keeps flowing during PG pin + backend writes; rollback ledger shared + lock-serialized so compensation stays exact on cancellation. 512MiB paced ingest: 60-69 -> 74-75 MB/s. OXICLOUD_INGEST_OVERLAP=0 restores inline settling (ops/bench hatch). - Frontend: instant-upload BLAKE3 hashing moved off the main thread to a bounded Web Worker pool (File handles by reference); vitest gate asserts the pool beats sequential (first gate draft posting buffers was 2.6x slower and was rewritten — copies dominated). Validation: cargo fmt + clippy -D warnings clean; 514 unit + 544 integration tests green; 270 frontend tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w |
||
|
Claude
|
aba89c4f5d |
perf: eliminate N+1 hot-path queries, cache immutable lookups, stop re-compressing compressed bytes
Every change is benchmark-verified (harness + before/after numbers in benches/, measured on this branch; reproduction commands in each doc): DAV / sync-client hot paths - PROPFIND dead-properties: one = ANY($1) query per 500-child page instead of one sequential query per child, and indexable `=` predicates instead of IS NOT DISTINCT FROM (seq scans). 2,000-child folder: 1.07-4.54 s of DB chatter -> 4-6 ms (258-773x). Applied to native + NC PROPFIND and both NC REPORT handlers. [benches/DEAD-PROPS.md] - Folder paging: keyset cursor (name > $last) + new partial index (folder_id, name) replaces LIMIT/OFFSET full-folder rescan per page. Full 20k-file walk: 1266 ms -> 77 ms (16.5x). New migration 20260917000000. [benches/PROPFIND-PAGING.md] - NC chroot / default-drive resolution: moka caches (30 s TTL, explicit invalidation on drive mutations) for find_default_for_user and the markerless chroot FolderDto. 2 uncached queries + 2 pool checkouts per NC/WebDAV/WOPI request -> sub-us moka hit (p50 0.7-3.6 ms -> ~1 us). [benches/CHROOT-CACHE.md] - Quota: PROPFINDs whose prop list never names a quota prop skip the 2-query resolution entirely (wants_quota()); the remaining lookups read 2 columns instead of the full auth.users row with its <=512 KiB avatar (11-16x, p50 3.4 ms -> 0.29 ms). Same narrow read now gates every upload quota check. [benches/QUOTA-PATH.md] CPU on the request path - ZIP exports (folder download, share ZIP, batch download): entries whose MIME says already-compressed (JPEG/MP4/zip/pdf/...) are Stored instead of Deflate - deflate ran inline on the tokio writer task at ~41 MB/s for ~0% size gain. Mixed media corpus: 4.31x wall and CPU, archive size unchanged. Shared predicate in common::mime_detect. [benches/ZIP-MEDIA.md] - Compression layers: tower-http's default maps to Brotli QUALITY 11 (verified in brotli-8.0.2 source and empirically: 90 ms per 64 KiB JSON response, 1.3 s per 700 KiB bundle). Both layers pinned to Precise(4): 99x less CPU for ~15% more bytes. SPA assets are now precompressed at build time (scripts/precompress.mjs, 77% smaller) and served via ServeDir::precompressed_br/gzip: 2016x less per-request work, and clients get the better q11 bytes. [benches/STATIC-PRECOMPRESSED.md] Batched / cached backend paths [benches/NPLUS1-AND-CACHES.md] - Content-search ReBAC re-verification: new AuthorizationEngine::check_files_read_batch (default = old loop; PgAclEngine override batches drive resolution + reuses role cache). 200 sequential point SELECTs per search -> 1-2 queries. - Batch-ZIP subtree downloads: drop per-file re-authz + per-file Recent recording (2 writes/file) for subtree entries already authorized at the root - mirrors the native folder-download path. ~6,000 statements removed from a 2,000-file archive. - CDC chunk manifests: immutable by content address, now moka-cached (weight-bounded 32 MiB, 60 s TTL, positive-only, invalidated on delete) - removes one manifest query (p50 0.44-4.4 ms) from every stream, range and full blob read. - People tab: grouped COUNT + batched cover lookup instead of dragging every face row with its 2 KiB embedding (10k faces: 30.4 ms & 21 MB -> 3.8 ms & 1.3 KB, 8.1x); merge() is one set-based UPDATE. [benches/PEOPLE-LIST.md] - Photos timeline cursor: raw timestamptz comparison instead of EXTRACT(EPOCH ...) wrapper + IS NULL OR disjunction - cursor is an index boundary again, deep scroll stops re-scanning skipped rows. - Public share landing: one atomic UPDATE ... access_count + 1 (was SELECT + full-row write-back: racy, lost updates, clobbered concurrent owner edits) - 3 round-trips -> 2 per visit. - move_to_trash: dead full-entity SELECT feeding a documented no-op removed from both branches; dead fields dropped from TrashService. - NFC normalization: is_nfc_quick fast path skips the decompose/recompose state machine for the ~100% already-NFC case (every row loaded from PG). Frontend - Large folders paint after page one (~200 items) via fetchFolderListing's new onPage hook instead of waiting for every sequential page. - Tested-and-reverted (kept for the record): cached Intl.Collator for name sorts - vitest showed it 2x SLOWER than V8's argument-less localeCompare fast path (5.6 ms vs 12.1 ms / 5k names). Sort order untouched. New bench harnesses under examples/ (bench feature): zip_media, dead_props, chroot_cache, quota_path, people_list, propfind_paging, static_precompress. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w |
||
|
Bradley Nelson
|
0a93123859 |
style(bench): rustfmt the blob/pool/runtime bench examples
These example targets landed unformatted on main and fail the Rustfmt CI check (`cargo fmt --all --check`); reformat them so this PR's checks pass. No logic changes. |
||
|
Claude
|
601fbdedf9 |
Revert "perf(blob): one syscall per new chunk in write_blob_bytes"
This reverts
|
||
|
Claude
|
ee51b32ba9 |
perf(blob): one syscall per new chunk in write_blob_bytes (create_new)
Replace the try_exists(stat) + File::create(O_CREAT|O_TRUNC) pair with a single OpenOptions::create_new (O_CREAT|O_EXCL), treating AlreadyExists as the existing idempotent skip. One metadata syscall per new chunk instead of two (each a spawn_blocking round-trip), and O_EXCL closes the check-then-create TOCTOU the old pair left open (a racing writer could be truncated). Honest measurement caveat (benches/BLOB-WRITE.md): the wall-clock throughput effect is BELOW the noise floor of the test environment — three 9-rep interleaved runs on the same ext4 device swing −12%..+21% at the 256 KiB CDC size, because a negative stat on a warm dentry cache is ~µs, dwarfed by the chunk's create+write+flush. So this is justified as a code-quality / correctness change (canonical idiom, strictly fewer syscalls, closes a TOCTOU, zero downside), NOT as a benchmarked perf win. The sibling idea — reusing the File handle for the fsync sweep — is deliberately NOT done: sync_blobs is a single end-of-stream sweep over all the upload's new hashes, so retaining handles would hold thousands of FDs open (>ulimit) on a large upload. The re-open sweep is a deliberate FD-frugal design. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5yYZ9s868mJwqT2Qz7ez |
||
|
Claude
|
013090a14e |
bench: measure peak RSS vs decode permits + record pool-concurrency results
Adds Part B (peak RSS for K concurrent decodes) to bench_pool_concurrency and records the findings in benches/POOL-CONCURRENCY.md. Honest result: under a 2-core quota the thumbnail decode pool shows flat throughput, p99, AND peak RSS (137 MiB) from K=1..16 — shrink-on-load already made each decode RAM-cheap, so over-permitting costs nothing measurable here. The effective_parallelism() migration is therefore a correctness/consistency change with no downside, mainly protecting the transcode + ffmpeg pools (and extreme host-core/quota ratios) this box can't reproduce — not a throughput win. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5yYZ9s868mJwqT2Qz7ez |
||
|
Claude
|
a504578303 |
bench: add CPU pool concurrency benchmark (thumbnail decode under a quota)
bench_pool_concurrency drives the real service path (Semaphore(K) gating spawn_blocking(bench_render_all)) with a gallery of concurrent callers and sweeps the decode-permit count K, reporting throughput + p50/p99. Run under taskset to model a CPU quota: it shows the effect of sizing the image pools to effective_parallelism (K=cores) vs the host count (over-subscribed). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5yYZ9s868mJwqT2Qz7ez |