- provide is_shared and is_favorite information in DTO, information propagated as badge/buttons per items
regarding performances I try to be minimalis on SQL to prevent any perf regression
- remove old set of sharedids and favoriteids (was not functionnal anymore)
- fix date picker (no past selection) in grant
- fix contextmenu close on /files section
restore the legacy display with new element uploaded, when swimlane is in place
as the sort is done by server side just add new element in a "new elements" swimlane.
if user continue to scroll down in cursor/pages and item is found from server,
UI remove it from the new element and restore the position
other option: is user refresh it's page, server will restore the natural order
ensure conext menu is build on click: API call to determine if user has access to item's parent folder is done only if user request context menu
(lazy API call)
Two items from the ROUND17 deferred list, each benchmark-gated with a
BEFORE/AFTER equivalence gate and a rollback-on-regression check (ROUND2–17
discipline). See benches/ROUND18.md.
[C1] backend — CalendarEvent::update_ical_property / remove_ical_property
rewrote the ENTIRE ical_data body with format!("{}{}{}") on every call and
allocated two search needles per call. calendar_storage_adapter::update_event
fans a multi-field edit out into one call per changed field, so a full REST
edit paid one full-body (up to ~11 KB) allocation per property. The body is
now mutated in place (replace_range for an existing property, four insert/
insert_str for a new one, byte-identical spans) and the single "\nNAME:"
needle is built on the stack (the "\r\nNAME:" needle was redundant — the LF
form is its suffix). bench_round18_micro [C1]: 70 -> 2 allocs/op (68 fewer),
2.46x wall, emitted body byte-identical.
[F1] frontend — ResourceList itemIndexById rebuilt a fresh Map over the whole
accumulated list every infinite-scroll page (O(N)/page, O(N^2) drain) and,
being a new instance each page, re-fired the reap-stale effect (another O(N)
id Set/page). New ItemIndexBuilder extends a persistent Map with the fresh
page only and reuses the reference across appends; the reap-stale effect now
tests membership against it. round18.bench.test.ts [F1]: 40x50 drain 74.1 ->
6.4 ms (11.5x), deep-equal to the reference at every page, reference-contract
gate (same-ref append / new-ref rebuild).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoxFtikahM1N4PE5s3ZVH3
Benchmark-gated, same rule as rounds 2–14: every change ships with a
BEFORE/AFTER benchmark and an equivalence/safety gate; an AFTER that doesn't
beat its BEFORE is rolled back. The rule is encoded per harness (GATE FAIL
non-zero exit in the Rust examples, threshold expect() in vitest).
F1 — Grouped listings (trash / recent / favorites / shared-with-me)
re-bucketed the WHOLE accumulated list on every infinite-scroll page.
ResourceSectionsBuilder (new, off the reactive graph) re-buckets only the
fresh page and hands VirtualList the same rows array reference for untouched
buckets. 50×50 (2 500-item) drain: 63 750 → 2 500 bucketOf calls (25.5×),
12.5 → 1.3 ms wall (9.9×); O(N²/page) → O(N). Deep-equal to the full-rebuild
reference at every page for both a contiguous (date) and a non-contiguous
(trash-by-drive) group-by; reference-stability + fallback gated.
B1 — exif Make/Model: the display String was thrown away to allocate the
trimmed copy; display_value_trimmed trims in place (drain + truncate), 2 → 1
alloc per field (8 → 4 allocs/op, 1.26×).
B2 — content-index worker: text_extractor::supports (lowercases MIME +
extension) was called twice per file per drain batch; classify once into a
Vec<bool> and thread it through both uses. 256-file batch: 704 → 353 allocs,
34.5 → 16.7 µs (2.07×).
B3 — tantivy: skip SnippetGenerator::create on a zero-hit content search
(return Ok(vec![]) once top_docs.is_empty()); the per-hit loop was empty.
400-doc index: 1 575.6 → 1 237.2 ns (1.27×), widens with index size.
Harnesses: examples/bench_round15_micro.rs, examples/bench_round15_tantivy.rs,
frontend resourceSections.bench.test.ts; writeup in benches/ROUND15.md. Also
normalizes two round14 bench examples that were committed unformatted
(cargo fmt --all).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012o47jSrtL7xuNGTHXmtiYL
Benchmark-gated (BEFORE/AFTER + equivalence/safety gate per change), same
discipline as rounds 2-12. Full write-up in benches/ROUND13.md.
Shipped:
- V1 Grouped views windowed (files route + ResourceList). The grid arm was
the last unwindowed path (trash is grouped-by-default in grid): each
swimlane now feeds its own VirtualList, outer container a flex stack.
vitest gate: 800-item grouped grid mounts <120 .file-item (was 800).
- Q1 get_users_by_ids drops the <=512 KiB avatar image + ui_preferences
JSONB (notification path never reads them). 30-member fan-out 8.60 ->
0.25 ms (34.3x), ~7.7 MB off the wire.
- Q2 Login provisioning is_empty() -> SELECT EXISTS for calendar + address
book (every login). 0.193 -> 0.170 ms, widens with owned-row count.
- Q3 Recent-access prunes only when the upsert inserted (RETURNING xmax=0)
— a re-access can't grow the set. 0.567 -> 0.324 ms (1.75x).
- L1 Locale supported-codes precomputed once vs rebuilt per anonymous
request. 616 -> 17.3 ns (35.7x), 18 -> 1 allocs.
- H1 Duplicate /api TraceLayer removed (global stack already wraps it).
1.86 -> 1.42 us/request, -6 allocs.
- H2 client_ip span field: borrow-only ClientIpDisplay vs owned String.
187 -> 173 ns, -1 alloc.
Not shipped (discipline): the "media hooks read the blob 3x" lead was a
correctness bug, not a perf dup — the raw-path metadata/faces readers
resolve only for local+unencrypted+single-chunk blobs and silently produce
nothing otherwise. Flagged for maintainers; routing through read_blob_bytes
is a correctness fix (perf-neutral-to-negative), not a benchmark-gated
perf change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
Benchmark-gated round (benches/ROUND12.md; every change ships with a
BEFORE/AFTER harness + equivalence gates, one candidate rejected by its
own bench):
DB / query shapes (bench_round12_queries):
- NC sharee search: username-only projection instead of the 21-column row
(incl. the <=512 KiB avatar) per match, + gin_trgm_ops indexes on
auth.users for the leading-wildcard ILIKE (4.98x; 54.7x with index).
- Password login: delete the redundant full-row update_user — create_session
already stamps last_login_at in its own txn (4.45x per login).
- Email-verified stamp: narrow conditional UPDATE (8.9x); OIDC repeat login
now compares profile state in memory and issues ZERO queries when nothing
changed (was: full 17-column rewrite per login).
- Refresh rotation: revoke+insert+stamp fused into one transaction via new
rotate_session port method (1.18x).
- WOPI CheckFileInfo / authorize_wopi_access: require(Read) + get_file +
check(Update) overlapped with tokio::join!, original result precedence
(cold 1.34x).
- Upload quota gate: user-envelope + drive-cap checks fused into ONE
round-trip (check_upload_quotas) — the NC chunked PUT pays this per
chunk (1.81x, 2 -> 1 queries/chunk); shared verdict evaluators keep
error shapes byte-identical.
CPU / allocs (bench_round12_micro):
- sized_json: pre-sized listing serialization replacing axum Json's 128 B
seed + doubling-realloc chain on files/folder-resources/photos/search
responses (1.40x, 13 -> 2 allocs per 500-row page; byte-identical).
- Security headers: 4 SetResponseHeaderLayer folded into the CSP middleware
pass (5 layers -> 1; 1.43x per request, -26 allocs; header set gated
byte-identical incl. 304s).
- Media capture-metadata: single-read extraction — nom-exif now parses the
buffer kamadak already read (zero-copy Bytes) and videos open once with a
kind() dispatch; per-image opens 2-3 -> 1 (1.44x warm geomean, 1.6-3.2x
cold cache; extraction outputs gated identical incl. the MIME-mislabel
track fallback).
- Chunked-upload session ops: owner gate folded into the operation's own
DashMap lookup + stack-encoded uuid compare (5 -> 3 lookups, -2 allocs,
1.28x per chunk).
Blob cache (bench_blob_cache_index + round-3 regression guard):
- CachedBlobBackend index: tokio::sync::Mutex<LruCache> -> moka::sync::Cache
with byte weigher. The mutex serialized every cached chunk read and scaled
NEGATIVELY (2.08 -> 1.07 Mops/s from 1 -> 2 readers); moka probes are
lock-free (2.17x at K=2). Byte budget now enforced by moka (manual
current_size + collect_evictions machinery deleted); eviction listener
unlinks size-evicted files only (Replaced entries keep their file —
gated). Single-flight miss gate unchanged (16 concurrent misses -> 1
fetch re-verified via the round-3 harness).
- put_blob now populates the cache BEFORE the inner backend consumes the
source file (the old order failed 100% of the time — local renames,
S3/Azure delete the source — so the first read after a whole-file put
re-downloaded from the remote); inner-put failure invalidates the entry.
Frontend (vitest gates):
- List-view thumbnails request the 150px icon rendition instead of 400px
preview into a 40px slot (~7.1x fewer pixels, ~4-5x fewer bytes per
thumbnail across list views); grid keeps preview.
Rejected by its own bench (kept as evidence in bench_round12_micro §2):
- Single-pass compression predicate: the monomorphized And-chain already
costs ~4.6 ns / 0 allocs total; the fused node measured within noise.
New migration: 20260719000000_users_search_trgm.sql (trgm indexes).
Deferred with prepared design: grouped file/grid view virtualization
(single-VirtualRows flatten, the photos pattern) — next round's headline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
Conflict resolution re-applies the round-11 SPA optimizations onto the
refactored item-based ResourceList (which independently converged on the
favoriteIds-prop star — S2 is now upstream's shape):
- ResourceList.selectedItems: upstream reintroduced the full O(N)
items.filter per selection toggle; replaced with the round-11
id-index projection — O(k·log k), item order preserved
(benches/ROUND11.md §S1)
- favorites/recent pages: upstream's rewrite kept the host-side
selectedItems shadow + unused selectedIds mirror and ignored the
batchToolbar snippet param; re-applied the param-consuming shape and
deleted the shadows
Validation on the merged tree: clippy --all-features --all-targets
-D warnings clean; cargo test --workspace 524 passed; frontend npm run
check 0 errors; vitest 310 passed (upstream's new tests + round-11 gates)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
Backend (each change benchmark-gated with BEFORE replicas + equivalence
gates; see examples/bench_round11_micro.rs, bench_round11_queries.rs,
bench_log_writer.rs and benches/ROUND11.md — final numbers land in the
follow-up doc commit):
- StoragePath re-representation: single canonical joined String, segments
derived on demand; File/Folder drop the duplicated path_string field
(4000→1000 allocs per 500-row listing page)
- Display classifier fusion: classify_display shares one stack-lowered
extension across the three decision trees; call sites in FileDto,
folder/favorites/recent handlers, trash, path-resolver (+ interning
where Arc::from was still used)
- /status.php and /openapi.json memoized into OnceLock<Bytes> (openapi
rebuilt a 171 KiB spec per request: 2.8 ms → 18 ns)
- NC upload-session PROPFIND: write! + pre-sized body + stack RFC2822
dates (2.3-2.6x, 2582→772 allocs at 256 chunks)
- REST download: dead FileDto clone removed (capture mime/size + move)
- CalendarEventDto/TrashedItem into_parts moves (11 KiB ical_data memcpy
gone per CalDAV row); CardDAV getlastmodified stack render
- 4xx path: borrowed ErrorResponse serialize, ErrorKind::as_str,
not_found/already_exists clone kill
- vCard emit via write!; search page moved out with into_iter skip/take;
content-hit UUIDs parsed once; group last-user check via HashSet
- RateLimiter: lock-free get + insert (and_upsert_with variant REJECTED
by benchmark); CSRF token borrow-compare + borrowed cookie extraction
- Thumbnail/preview ETags built from as_str (Debug-identical bytes)
- Encrypted backend: encrypt_in_place_detached single-buffer write path,
chunk-sized reserve in collect_stream; retry labels made lazy
- PG: deferred upload registration 3→1 round-trips (persist_file CTE
template); direct_grant_cache for Calendar/AddressBook/Playlist authz
(single-flight + set_role/clear_role invalidation); expand_user
tokio::join!; geo clusters min(uuid)::text; recluster face assignment
batched into one UNNEST update
- People recluster cosine: norms precomputed once (bit-identical gate)
- NC capabilities poll logs demoted to debug; tracing-appender dep added
for the log-writer benchmark
Frontend:
- ResourceList.selectedEntries O(N)-per-toggle → id-index projection
O(k log k); favorites/recent consume the batchToolbar snippet param and
drop their duplicate filter + dead selectedIds mirror
- Recent: star state via new favoriteIds prop — a star click no longer
rebuilds all N entries
- admin timeAgo >30d fallback uses the cached Intl.DateTimeFormat
- vitest gates in src/lib/components/round11.bench.test.ts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
- add resource kind filter (file, folder, drive) in shared section (localStorage stored)
- add user preferences serverside store
- add client side dotfile filter (show/hide dotfiles) (user perf stored, default: dotfiles are shown)
for security trashed dotfile are always displayed
protection added: if a folder has only hidden items, a notification invite user to display it
if a user rename or create a hidden item, a notification tells it to user
Add an end-to-end and unit test suite for the SvelteKit frontend:
- Playwright e2e specs (tests/e2e/spa) with a throwaway container stack,
codegen scenarios, and an Istanbul-based coverage report pipeline.
- Vitest unit tests across API endpoints, components, stores and composables.
- `data-testid` hooks on interactive elements (AppShell, FileViewer,
ShareDialog, search, photos, files breadcrumbs, login/Nextcloud flows,
public share pages) so the e2e suite can target them deterministically.
- Serve the SPA app-shell CSP from a <meta> policy (svelte.config.js) plus a
middleware that skips the CSP header on HTML; move the Nextcloud Login Flow
v2 grant page to the SvelteKit /nextcloud/login route.
- `just front-codegen` recipe and start-server-spa.sh harness.
Make the test environment robust and consistent:
- Install a deterministic in-memory localStorage/sessionStorage in the Vitest
setup so storage behaves identically across Node versions (Node 26 ships a
native Web Storage global that otherwise shadows jsdom's).
- Pin devenv to Node 26 + PostgreSQL 18 and pin every CI job to Node 26.3.0
so the dev shell and CI run the same toolchain versions.
Repair the API/WebDAV (hurl) suite, which had drifted from the backend:
- Migrate the removed `/api/folders/{id}/listing` endpoint to `/resources`
(cursor-paginated `{items:[{resource_type,resource}]}` shape) across the
batch-copy, grants, nested-group, and WebDAV NC tests + the dav_helpers
wipe routine.
- Stop photos_etag from uploading the dedup-tracked fixture so the dedup
blob-lifecycle test can own its content-addressed blob exclusively.
- dedup_create now asserts the idempotent same-content re-upload (201 +
existing file id) instead of the stale 409 expectation.
Generated coverage reports, nyc output and the e2e server runtime data dir
are gitignored rather than committed.
Checkpoint of the in-progress frontend toolchain work (Vite pinned to ^6 after
the 7/8 rolldown build break, eslint-plugin-svelte v3 navigation/reactivity
fixes, CI/Dockerfile/manifest updates) together with three UI performance
optimizations (verified on the Vite 6 build):
- Critical CSS: move auth.css/music.css off the global path into their route
chunks (login/device/nextcloud-login, music) -- -25% gzipped critical CSS
(~5.4 KB) on every non-auth/non-music page load.
- relativeTimeAgo: cache the Intl.RelativeTimeFormat (was rebuilt per call, once
per row per render) -- 22.7x faster date formatting in large lists.
- Virtualize search results and grouped trash (list view) via VirtualList -- DOM
rows mounted stay ~constant (~27) instead of O(N) (94.6% fewer for 500 hits).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bundles the backend+frontend delta-upload (content-dedup) feature with a
batch of frontend fixes from this session.
Upload / dedup:
- Client-hashed delta & instant upload (deltaUpload, hashWasm vendor shim)
- Backend dedup batch endpoint (dedup_service, dedup_handler, routes)
- session store owned-hash helpers; unit tests + upload-strategy bench
Frontend UI/UX:
- Colour file-type icons in grid/list (per-type tinted tiles + glyph hue)
- Robust thumbnail fallback; PDFs now show their type icon (backend
generates no PDF thumbnails) instead of a blank tile
- Fix PDF preview: load via a same-origin blob: iframe — the API URL is
blocked by the global X-Frame-Options: DENY in the browser's framed
PDF viewer, matching the existing CSP `frame-src blob:` design
- Groups: localized virtual-group description (no DB schema-note leak),
add nav.groups to the 15 missing locales, fix primary-button contrast
- Repoint --color-text-light → --color-on-accent (was faint grey on accent)
- Nudge the admin role badge off the user-menu header divider
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends windowing to the remaining O(n)-DOM surfaces: the card-grid view of
ResourceList (recent / favorites / shared / shared-with-me / trash / search) and
the main file browser (`files/[...path]`), in both list and grid layouts.
- VirtualList gains a real grid mode: its inner window carries the caller's grid
class (`files-grid-view`) and lays out `columns` cards per windowed row. The
row pitch is auto-measured (grid card height tracks column width via the 4/3
aspect-ratio thumbnail) and re-measured on resize.
- `useVirtualWindow` now distinguishes scroll from resize and exposes a
`resizeTick`, so size-dependent layout (the grid pitch) is recomputed only when
it can actually change.
- `gridColumns(width)` (new util) mirrors the CSS `auto-fill` / `--grid-card-min`
/ gap so the windowed row count matches the browser's real wrapping exactly;
shared by both grid callers.
- The files browser flattens folders-then-files into one discriminated `entries`
list rendered through VirtualList (list: columns=1; grid: columns from width).
Grouped (swimlane) views stay fully rendered, as before — they're bounded.
ResourceList GRID, headless Chromium (1280x900), synthetic rows, before/after:
rows | mount→paint | DOM nodes | JS heap | scroll frame | jank frames
------+-------------+-----------+---------+--------------+------------
1000 | 979→75 ms | 19k→879 | 16→3 MB | 16→17 ms | 0→0
5000 | 4005→67 ms | 95k→879 | 72→3 MB | 37→17 ms | 26→0
20000 |12015→65 ms | 380k→879 |281→7 MB |197→17 ms |1249→19
Rendered DOM and heap are flat (O(visible)) regardless of dataset size; mount is
~185x faster and scroll holds ~60fps. Verified visually mid-scroll (5 columns,
4/3 thumbnail tiles, cards land at the expected indices). The files browser
shares the same VirtualList path (it can't be mounted headless — it depends on
$app routing/session — so it's validated via svelte-check + the production build
+ the shared, separately-benchmarked component).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8Vb9QHmLZnEMzHz7MrFy6
Large folders/collections rendered every row into the DOM, so memory and
scripting time grew linearly with item count. A 20k-row list took ~12.5s to
first paint, held ~380k DOM nodes / ~281MB JS heap, and scrolled at ~4fps.
Add a reusable, dependency-free `VirtualList` that windows rows against the
nearest scrollable ancestor (the existing `.content-area`), so the
single-scrollbar UX, sticky header and end-of-list sentinel are unchanged. It
reserves the full scroll height with a sized spacer and translates a small
window of rows; row height is auto-measured for the single-column case.
Wire it into ResourceList's flat list view (recent, favorites, shared,
shared-with-me, trash, search). Grouped sections and grid view are unchanged
for now and are the next callers of the same primitive.
Measured in headless Chromium (1280x900) with synthetic rows, before/after:
rows | mount→paint | DOM nodes | JS heap | scroll frame | jank frames
------+-------------+-----------+---------+--------------+------------
1000 | 632→70 ms | 19k→351 | 16→2 MB | 17→17 ms | 0→0
5000 | 4401→54 ms | 95k→351 | 72→3 MB | 47→17 ms | 136→0
20000 | 12577→56 ms | 380k→351 |281→6 MB |258→17 ms | 1443→0
Rendered DOM and heap are now flat (O(visible)) regardless of dataset size;
scroll holds 60fps with zero jank. Correctness verified by probing a mid-list
scroll (rows land at the expected indices, positioned within the viewport).
eslint: disable core `no-undef` for `.svelte` (TypeScript/svelte-check already
resolve identifiers, including `<script generics>` type params it can't see).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8Vb9QHmLZnEMzHz7MrFy6
Edouard Vanbelle