Commit Graph

26 Commits

Author SHA1 Message Date
Claude 9754aecfa9 perf: round 19 — auth/WOPI/vCard/PROPFIND per-request & per-row alloc cuts
Benchmark-gated (examples/bench_round19_micro.rs, benches/ROUND19.md): every
section ships a BEFORE/AFTER counting-allocator arm with a byte/-value
equivalence gate and a GATE-FAIL-rollback exit. All eight pass. No Postgres.

- M1 verify_basic_auth cache key: blake3::hash(format!("{u}:{p}")) → incremental
  Hasher (byte-identical key, 2→0 allocs on every Basic-auth DAV request)
- M2 WopiTokenService: prebuild Validation/DecodingKey/EncodingKey in new()
  instead of per-call (mirrors JwtTokenService; 16→12 allocs/validate)
- V1/V2 vCard emit (contact_to_vcard/generate_vcard): FN fallback drops the
  throwaway to_string, NOTE skips the escape copy for newline-free notes, REV
  uses new common::fmt::compact_ical_utc stack renderer (11.5× vs chrono
  strftime, 3→0 allocs); per-contact 9→4 allocs
- M4 trash_service::row_to_item_dto: move name/path/blob_hash out of the owned
  row instead of cloning (3 clones/file row gone)
- M5 search cache key: Uuid::hyphenated().encode_lower stack buffer instead of
  to_string (identical u64 key, 1→0 allocs/request)
- M6 streaming PROPFIND: reuse one href buffer across the page instead of a
  format! per child (native + NC handlers; 192→3 allocs on a 64-child page)
- M7 nextcloud extract_url_user: return Cow instead of forcing into_owned
  (zero-alloc on the common ASCII-username path)

common::fmt::compact_ical_utc added with chrono-parity unit tests (CASES +
60-year sweep). cargo fmt + clippy --all-targets clean; 526 lib unit tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ront9bk7YMoffVQkGG47gh
2026-07-19 22:29:48 +00:00
Claude c207e66f4b perf: round 17 — dedup ingest/verify hash-clone purge, CardDAV vCard TYPE tokens
Targets the content-addressable dedup write path (the ROUND15-deferred
"dedup_service hash-String re-allocations") from both ends — the streaming
ingest loop and the delta-commit verification read — plus a CardDAV vCard
micro-cut. Every change is benchmark-gated with a hard rollback rule; no
PostgreSQL needed for any arm (benches/ROUND17.md).

Backend (counting-allocator, examples/bench_round17_micro.rs):
- D2 chunk-ingest (store_from_stream, the hottest write path — every chunk of
  every upload): the 64-char hex hash String was allocated 3x per chunk
  (to_hex + chunk_hashes clone + session_seen insert-clone, the last dropped on
  a duplicate). The intra-upload dedup set now keys on the raw 32-byte BLAKE3
  digest ([u8;32], Copy, no heap) and the manifest push is branch-split so a
  duplicate moves the hex in: 3 -> 2 allocs/new chunk, 3 -> 1/duplicate.
  Measured 214 -> 149 allocs/op (1.14x wall) on a 64-chunk 1-in-2-dup batch;
  smaller/faster set too (32B inline keys vs 64B heap Strings).
- D1 hash_chunk_sequence (delta-commit verification): took chunks by
  &[(String,u64)] and fed the backend stream with iter().cloned(), re-cloning
  every chunk hash a second time on top of the owned Vec the caller already
  built. Take the Vec by value + into_iter(): 65 -> 0 internal allocs/op, ~2.5us
  of clone work removed per verify.
- V1 vCard TYPE tokens (contact_to_vcard + generate_vcard, 5 sites): each
  EMAIL/TEL/ADR TYPE= param used ty.to_uppercase() — a throw-away String per
  token per contact. New shared fmt::push_upper writes the upper-cased chars
  straight into the buffer (byte-identical to str::to_uppercase, unit-tested):
  13 -> 5 allocs/op, 1.19x wall.

Gates: each section asserts byte/-value equivalence (D2 the ordered manifest +
sizes + write-set; D1 the removed clone is a pure copy; V1 the full vCard) and
exits non-zero if an AFTER arm fails to reduce allocations. push_upper is
unit-tested byte-equal to str::to_uppercase (fmt::tests). Verified end-to-end:
cargo fmt clean, clippy --release --all-targets --features bench -D warnings
clean, and the harness prints GATE PASS against the built release lib.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XMmt7vNETYUbEG3Hc17LDx
2026-07-19 19:33:41 +00:00
Claude f58d72a780 perf: round 13 — grouped-view virtualization, notification/login query narrowing, HTTP dedup, locale precompute
Benchmark-gated (BEFORE/AFTER + equivalence/safety gate per change), same
discipline as rounds 2-12. Full write-up in benches/ROUND13.md.

Shipped:
- V1 Grouped views windowed (files route + ResourceList). The grid arm was
  the last unwindowed path (trash is grouped-by-default in grid): each
  swimlane now feeds its own VirtualList, outer container a flex stack.
  vitest gate: 800-item grouped grid mounts <120 .file-item (was 800).
- Q1 get_users_by_ids drops the <=512 KiB avatar image + ui_preferences
  JSONB (notification path never reads them). 30-member fan-out 8.60 ->
  0.25 ms (34.3x), ~7.7 MB off the wire.
- Q2 Login provisioning is_empty() -> SELECT EXISTS for calendar + address
  book (every login). 0.193 -> 0.170 ms, widens with owned-row count.
- Q3 Recent-access prunes only when the upsert inserted (RETURNING xmax=0)
  — a re-access can't grow the set. 0.567 -> 0.324 ms (1.75x).
- L1 Locale supported-codes precomputed once vs rebuilt per anonymous
  request. 616 -> 17.3 ns (35.7x), 18 -> 1 allocs.
- H1 Duplicate /api TraceLayer removed (global stack already wraps it).
  1.86 -> 1.42 us/request, -6 allocs.
- H2 client_ip span field: borrow-only ClientIpDisplay vs owned String.
  187 -> 173 ns, -1 alloc.

Not shipped (discipline): the "media hooks read the blob 3x" lead was a
correctness bug, not a perf dup — the raw-path metadata/faces readers
resolve only for local+unencrypted+single-chunk blobs and silently produce
nothing otherwise. Flagged for maintainers; routing through read_blob_bytes
is a correctness fix (perf-neutral-to-negative), not a benchmark-gated
perf change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
2026-07-19 08:17:48 +00:00
Claude 221c1f31b0 perf: round 11 — StoragePath joined-only, classifier fusion, memoized bodies, query-shape pack, SPA fine-grained stars
Backend (each change benchmark-gated with BEFORE replicas + equivalence
gates; see examples/bench_round11_micro.rs, bench_round11_queries.rs,
bench_log_writer.rs and benches/ROUND11.md — final numbers land in the
follow-up doc commit):

- StoragePath re-representation: single canonical joined String, segments
  derived on demand; File/Folder drop the duplicated path_string field
  (4000→1000 allocs per 500-row listing page)
- Display classifier fusion: classify_display shares one stack-lowered
  extension across the three decision trees; call sites in FileDto,
  folder/favorites/recent handlers, trash, path-resolver (+ interning
  where Arc::from was still used)
- /status.php and /openapi.json memoized into OnceLock<Bytes> (openapi
  rebuilt a 171 KiB spec per request: 2.8 ms → 18 ns)
- NC upload-session PROPFIND: write! + pre-sized body + stack RFC2822
  dates (2.3-2.6x, 2582→772 allocs at 256 chunks)
- REST download: dead FileDto clone removed (capture mime/size + move)
- CalendarEventDto/TrashedItem into_parts moves (11 KiB ical_data memcpy
  gone per CalDAV row); CardDAV getlastmodified stack render
- 4xx path: borrowed ErrorResponse serialize, ErrorKind::as_str,
  not_found/already_exists clone kill
- vCard emit via write!; search page moved out with into_iter skip/take;
  content-hit UUIDs parsed once; group last-user check via HashSet
- RateLimiter: lock-free get + insert (and_upsert_with variant REJECTED
  by benchmark); CSRF token borrow-compare + borrowed cookie extraction
- Thumbnail/preview ETags built from as_str (Debug-identical bytes)
- Encrypted backend: encrypt_in_place_detached single-buffer write path,
  chunk-sized reserve in collect_stream; retry labels made lazy
- PG: deferred upload registration 3→1 round-trips (persist_file CTE
  template); direct_grant_cache for Calendar/AddressBook/Playlist authz
  (single-flight + set_role/clear_role invalidation); expand_user
  tokio::join!; geo clusters min(uuid)::text; recluster face assignment
  batched into one UNNEST update
- People recluster cosine: norms precomputed once (bit-identical gate)
- NC capabilities poll logs demoted to debug; tracing-appender dep added
  for the log-writer benchmark

Frontend:
- ResourceList.selectedEntries O(N)-per-toggle → id-index projection
  O(k log k); favorites/recent consume the batchToolbar snippet param and
  drop their duplicate filter + dead selectedIds mirror
- Recent: star state via new favoriteIds prop — a star click no longer
  rebuilds all N entries
- admin timeAgo >30d fallback uses the cached Intl.DateTimeFormat
- vitest gates in src/lib/components/round11.bench.test.ts

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
2026-07-18 22:02:00 +00:00
Claude c51af68432 perf: round 10 — auth alloc purge, parent-herd batching, query-shape pack, NC 304s
Benchmark-gated (benches/ROUND10.md; every change carries a BEFORE/AFTER
harness with equivalence/safety gates — two designs were rejected or
rewritten by their own benches before adoption):

- Auth hot path: TokenClaims/CurrentUser display fields to Arc<str>, role
  to inline SmolStr end-to-end (Bearer, cookie, Basic-auth cache) — 4→1
  allocs per authenticated request, 3→0 per warm DAV request; JWT
  Encoding/Decoding/Validation built once.
- Cold shared-album herd: leader-inline parent batching in PgAclEngine
  (+ cascade try_get_with single-flight) — 100→2 parent queries per
  100-thumb cold herd, herd wall 1.9x, sequential + warm paths unchanged,
  all ROUND8/9 safety gates plus new herd-equivalence gates.
- Query-shape pack: share download double-fetch 2→1 (2.18x), contact-group
  COUNT(*) 14.9x, save_faces UNNEST 3.9x, playlist reorder UNNEST 63.7x
  (now atomic), search files∥folders join! 1.45x, move drive-lookup join!
  2.14x, trash partial (drive_id, trashed_at) indexes, CalDAV event-gate
  narrow read, favorites/recents binary-decode port, dead count_files
  removed.
- NC surface: preview + avatar honour If-None-Match (e2e: 5 KB and 197 KB
  → 0 bytes per revalidation), avatar WebP→PNG transcode memoised,
  PROPFIND/trashbin integer+date emits on stack formatters, folder-header
  enrichment join!, chunk-PUT retry stat folded into create_new open.
- common::fmt integer rendering rewritten on the std 2-digit LUT after the
  round's own bench caught the div-loop losing to to_string (16.1 ns vs
  22.5; speeds every prior-round call site).
- Micro-pack: WebDAV scope probe borrow-only, ShareService base_url
  snapshot, cookie_secure OnceLock, Arc'd AES-GCM cipher, stack request-id,
  tantivy analyzer clone dropped.
- SPA: search stale-guard + AbortController (10→1 completed round-trips,
  stale-clobber gone), getFolder in-flight dedup, gridColumns matchMedia
  hoist (10k→0 style reads).

Backend: cargo fmt + clippy -D warnings clean, 524 tests green.
Frontend: npm run check clean, 301 vitest green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DdM7V7M3QPW7HEHg3gLov
2026-07-18 20:33:50 +00:00
Dionisio Pozo 2317d594e3 Merge pull request #610 from EdouardVanbelle/security/grants2 2026-07-18 16:07:17 +02:00
Claude 9729f033b2 perf: round 6 backend — CardDAV cursor streaming, borrowed NC id chain, binary UUID decode, one-alloc hex
Benchmark-gated (equivalence + BEFORE/AFTER in examples/bench_*, results
and reproduce commands in benches/ROUND6.md):

- CardDAV whole-book REPORT + depth-1 PROPFIND stream through a PG
  cursor (stream_contacts_by_book, 500-contact pages) instead of
  materialising every vCard twice: 8 000 contacts TTFB 37.4 → 7.6 ms
  (4.9x), peak heap 19.0 → 7.0 MiB (2.7x), wall -23%; REPORT and
  PROPFIND byte-identical to the buffered writers.

- NC numeric-id chain fully borrowed: get_or_create_file_ids/folder_ids
  take &[&str] and return HashMap<Uuid, i64>; batch_resolve_ids callers
  (PROPFIND pages, REPORT, trashbin, OCS search) pass id slices and look
  up via nc_id_of. 2.006 → 0.006 allocs/child (334x), 1.53x wall per
  500-child page. batch_check_favorites binds &[&str] as text[].

- file_blob_read_repository listing SELECTs drop id::text/folder_id::text
  server casts: rows decode binary Uuid (16 vs 36 bytes on the wire) and
  render once in row_to_file. A/B on 500-row pages: 1.225 → 1.044 ms
  mean (1.17x), p95 1.686 → 1.345 (bench_uuid_text_cast; single-row,
  param and min() sites left as-is deliberately).

- IncrementalHasher::finalize_hex renders through common::fmt::hex_lower
  instead of one format! per digest byte: 18 → 1 (md5) / 35 → 1 (sha256)
  allocs per chunk finalize, 14-15x wall.

- Share landing overlaps the access-count UPDATE with the unlock fetch
  via tokio::join! (one round-trip off every public link hit).

- REJECTED by benchmark and reverted: try_join_all fan-out of the
  batch-favorites authz pre-check — 42.6 → 56.4 ms cold, 0.15 → 0.23 ms
  warm against local-socket PG (bench_favorites_authz kept as evidence).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
2026-07-18 09:03:33 +00:00
Edouard Vanbelle eb884f6c8f fix(contact): use Permission::Create for creations 2026-07-17 19:01:05 +02:00
Edouard Vanbelle 38abe6766c fix(contact): use Permission::Delete for deletion verb 2026-07-17 19:01:05 +02:00
Claude 12dc648cff perf: round 4 — one-pass row paths, drive-selector cache, CalDAV single-parse, streamed Azure, batched hydration
Nine benchmark-gated changes (benches/ROUND4.md; every one ships with a
BEFORE/AFTER bench + equivalence gate, rollback rule as ROUND2/3):

- Row→entity path build: one-pass StoragePath::from_folder_and_name /
  from_joined + normalize_storage_name_owned + alloc-free Display —
  743→417 ns/file-row (1.78x), −5 allocs/row on every listing surface.
- WebDAV drive-selector: per-user readable_cache (single-flight, 30 s
  TTL, explicit invalidation incl. membership + group changes) replaces
  the grants join per request — 441 µs → 0.8 µs (~550x), 0 queries warm.
- CalDAV from_ical/update_ical_data: 8 full IcalParser runs per VEVENT
  → 1 (7.1x per PUT, 4.4x on 50-event imports); alloc-free split_vevents,
  chunk scan without the whole-body uppercase copy (1.4x), borrowed-key
  UID grouping (1.3x), REPORT props no longer cloned.
- PROPFIND emit: partition Vecs dropped (single-pass 404 list) + stack
  rendered RFC 3339/2822 dates, sizes, quoted etags (common::fmt,
  chrono-byte-identical, sweep-tested) on both DAV surfaces — 1.22x
  per page, 17.9→12.0 allocs/row.
- Grant-listing hydration: calendars/address books/playlists batch
  hydrate via = ANY($1) — 15 serial queries → 1 (~13x per sync poll).
- user-flags cache: get→insert → try_get_with single-flight (32→1
  queries per cold herd).
- Azure downloads: whole-blob Vec buffering → streamed SDK pages —
  TTFB 349→4 ms (87x), peak heap 480→1.9 MiB (254x) on 256 MiB blobs;
  new OXICLOUD_AZURE_ENDPOINT_URL override (Azurite/bench hook).
- Face indexing: unbounded per-image tokio::spawn → core-count
  semaphore, permit before blob read — peak heap 1175→176 MiB (6.7x).

Checks: cargo fmt, clippy --all-features --all-targets -D warnings,
cargo test --workspace (523 passed) + --features test_utils. hurl API
suite and dockerized integration DB not runnable in this environment —
left to CI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
2026-07-17 13:48:37 +00:00
Edouard Vanbelle dc2c8f5dcd fix(carddav): fix tel uri 2026-07-14 23:00:19 +02:00
Edouard Vanbelle 54b5b3bf4f feat(caldav+carddav): auto create default cal & card
automatically create default Calendar and default addressbook per user
    (no creation if user already have a such resource)

    default name are "Personal"

    this is using the user's life cycle like does the drives

    answers to issue #545
2026-07-14 14:30:05 +02:00
Edouard Vanbelle 0e341758af feat(calendar,addressbook): remove old sharing code 2026-07-08 01:03:25 +02:00
Edouard Vanbelle 0fcd617fd1 feat(calendar,addressbook): migrate share to authz engine
migrate DB entries to authz engine
    and wire authz engine to caldav and carddav
2026-07-08 01:03:25 +02:00
Claude 6a27c742d4 Fix CalDAV/CardDAV multiget loading entire collections into memory
A REPORT multiget for a handful of resources previously listed the
whole calendar/address book (every row incl. ical_data/vcard) and
filtered by href in Rust with an O(N×M) substring scan. Large
collections paid full-table latency, RAM and DB CPU on every sync.

- Add find_events_by_ical_uids / get_contacts_by_uids through every
  layer (domain repo trait → PG repo → storage port/adapter → use case
  → handler) using one indexed `= ANY($2)` query, mirroring the
  existing single-UID lookups.
- Extract UIDs from multiget hrefs with a shared, tested
  uid_from_multiget_href helper (percent-decoding, case-insensitive
  extension strip, collection hrefs rejected). Exact UID matching also
  removes the false positives the old substring filter allowed.
- CalDAV PROPFIND single-event path now uses the existing indexed
  get_event_by_ical_uid instead of listing all events.
- Honour the already-documented limit/offset query params on
  GET /api/address-books/{id}/contacts: optional ListQuery fields
  thread through ContactUseCase::list_contacts to a paginated repo
  query. Omitted params keep returning the full book (frontend and
  DAV listing/sync paths unchanged).

https://claude.ai/code/session_0193Hff42gaA962wThxMGSd1
2026-06-11 09:21:35 +00:00
Claude 7687766bf7 perf(dav-collections): indexed UID lookups for single-object operations
Every CalDAV PUT/GET/DELETE of one .ics loaded the ENTIRE calendar —
every row including its ical_data — and filtered with .find() in Rust,
so importing N events cost O(N²) rows transferred. CardDAV did the exact
same in four places (PROPFIND of one .vcf, PUT existence check, GET,
DELETE), with three JSONB deserializations per discarded contact. The
indexed repo queries (find_event_by_ical_uid, get_contact_by_uid)
existed all along with zero callers.

Wire them end to end: new `get_event_by_ical_uid` /
`get_contact_by_uid` use-case methods (same access checks as
list_events / list_contacts, per the service-layer authz rule) exposed
through the storage ports and adapters, and the seven handler sites now
resolve one row instead of the whole collection.

https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
2026-06-10 09:52:28 +00:00
Edouard Vanbelle badf35f08f chore: remove all executable attributes on non bash files 2026-04-01 23:14:42 +02:00
zjean 18518bedaf fix: resolve clippy warnings and rustfmt issues for CI compliance
Fix all clippy lints (collapsible if, clone on Copy, needless borrow,
redundant bindings, unused params) and apply rustfmt across the codebase.
Update test mocks to match Uuid-based trait signatures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 14:34:07 +01:00
Diocrafts 06ed0455ce perf: migrate all user/session/auth IDs from VARCHAR(36) to native UUID
- Schema: all ~15 VARCHAR(36) columns → UUID with DEFAULT gen_random_uuid()
- Domain entities: User, Session, DeviceCode, AppPassword, Share → id: Uuid
- DTOs: CurrentUser.id → Uuid (API boundary DTOs keep String for JSON)
- Auth middleware: parse JWT claims.sub (String) → Uuid at boundary
- All repository traits, port traits, service impls updated end-to-end
- Handlers: pass Uuid by value (Copy, 16 bytes) instead of String refs
- Settings chain: updated_by column → Uuid (was text, caused setup crash)
- Removed ~650 lines of String↔Uuid conversion boilerplate
- Eliminates per-request heap allocations for ID cloning
- 16-byte binary comparison vs 36-byte string comparison in all queries
- Native UUID indexing in PostgreSQL (btree on 16 bytes vs 36-char text)

85 files changed, 1090 insertions(+), 1739 deletions(-)
2026-03-07 14:59:32 +01:00
Dionisio f2d35ca792 feat: auto-persist JWT secret, remove setup token requirement
- JWT secret auto-generates and persists to <STORAGE_PATH>/.jwt_secret
- Remove setup token: first admin setup is open until system initialized
- Fix schema.sql: move CREATE EXTENSION pg_trgm/ltree to top
- Update login UI and auth.js to remove setup token fields
2026-03-05 22:12:53 +01:00
Claude 1b49135ca9 perf: replace dyn trait objects with concrete types to eliminate vtable overhead
Remove async-trait dependency and use native Rust async fn in traits.
Replace Arc<dyn Trait> with Arc<ConcreteType> throughout the codebase
to enable monomorphization and eliminate dynamic dispatch overhead.

Key changes:
- Remove write-behind cache (no implementation existed)
- Fix should_transcode static method call
- Use ContactStorageAdapter directly instead of dyn AddressBookUseCase
- Clean up unused trait imports across services and DI

https://claude.ai/code/session_01EbAFEfyJNLRmJHmmYDX3Tt
2026-03-03 15:36:42 +00:00
Dionisio 1ed20f425f perf: Phase 4+5 optimizations — uploads 10x, downloads 2x, concurrent 2x. moka cache, 512KB buffers, remove sync_all, hash-on-write, preloaded queries, bench.sh v3, gitignore storage/. 500MB upload 12.6s->1.3s (392MB/s). RSS 69-113MB, 0 swap. 2026-02-15 17:56:47 +01:00
Dionisio 4c98c5a657 style: apply cargo fmt to entire codebase
Standardize code formatting across all 173 Rust source files
using rustfmt. No functional changes - purely cosmetic.

This establishes a consistent code style baseline for the
project going forward.
2026-02-14 01:29:34 +01:00
Diocrafts a82faa5eaf refactoring hexagonal and clean architecture 2026-02-08 13:40:23 +01:00
Dionisio 52840e57df refactor: remove serde from domain entities for Clean Architecture compliance
- Remove Serialize/Deserialize from File, Folder, Session, User, Contact entities
- Create contact_persistence_dto.rs for JSONB persistence in infrastructure layer
- Update contact_pg_repository to use persistence DTOs
- Fix dependency on zip crate (downgrade from 7.2.0 to 2.1.0)
- Fix unused variable warnings in main.rs
- Move PathService import from domain to infrastructure
- Add missing fields to CoreServices and RepositoryServices
- Create proper service initialization in main.rs

Clean Architecture improvements:
- Domain layer no longer depends on serde framework
- Persistence concerns isolated to infrastructure layer
- TokenClaims in auth_service.rs is only exception (required for JWT)
2026-02-02 23:56:40 +01:00
DioCrafts 52d8250d51 adding card dav and cald dav 2025-04-13 01:04:04 +02:00