Re-uploading a partially-uploaded folder used to surface hundreds of spurious
"already exists" failures, and a file the watchdog aborted (or one the server
committed just before the client gave up) was lost.
Backend — save_file_with_blob_impl (the shared write path for both plain and
by-hash uploads): on a name conflict (23505), if the existing non-trashed file
holds byte-identical content (same folder, same name, same blob hash), return
that file as success instead of erroring. A different-content clash still
conflicts. Re-upload / re-sync becomes a clean no-op for everything already
stored — only the genuinely missing files transfer.
Frontend — uploadWithRetry: each file gets one automatic retry on a transient
failure (quota is never retried). With backend idempotency, retrying an
already-stored file is an instant no-op and a stalled/aborted file gets a real
second chance, so a folder upload self-completes instead of leaving gaps.
cargo test: 448 passed. npm run check: clean, 58 tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A folder upload with several large files could appear frozen for ~2 min: a few
concurrent uploads stalled and the old 120s per-file timeout neither aborted the
request (leaving zombie XHRs that exhaust the browser's per-host connection pool)
nor recovered quickly.
- uploadFileWithProgress now self-aborts on a stalled connection: the deadline
resets on every upload-progress tick (a slow but *moving* transfer is fine),
and once the body is sent the server gets a fixed window to respond; on a stall
xhr.abort() frees the connection immediately — no zombie, no cascade.
- Lower upload concurrency 4 -> 3 to reduce server contention from large
concurrent uploads.
- The outer per-file timeout is now just a generous backstop for a wedged delta
worker / by-hash request.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backend:
- POST /api/dedup/check-batch — returns the subset of submitted whole-file
BLAKE3 hashes the caller already owns, in one query (user-scoped,
anti-enumeration via idx_files_blob_hash). Lets a client learn which of N
files it can skip with a single round trip.
(dedup_service::user_owned_blob_references, dedup_handler, routes) + tests.
Frontend — upload pipeline:
- Instant ("by-hash") upload for content the caller already owns: hash every
in-band file, ONE /api/dedup/check-batch, create the owned ones with zero
content bytes, upload only the rest. Covers all sizes below the 8 MB delta
threshold (delta handles larger files). vendor/hashWasm computes the
whole-file BLAKE3 on the main thread.
- Resilient parallel uploads: bounded concurrency (4) + a per-file deadline,
so one stuck/slow/failing file no longer freezes the whole batch — it blocks
only its own lane and times out / is skipped while the rest proceed. Quota
exhaustion stops the run early; partial results are reported ("N uploaded,
M failed").
- Folder uploads (uploadTree) show live bell progress + a final result and go
through the same dedup + parallel pipeline.
- Storage bar ("Almacenamiento") refreshes after uploads/deletes
(session.refresh) instead of showing the stale login value.
Frontend — i18n / UI fixes:
- Fix literal {{count}} and {{percentage}}/{{used}}/{{total}} (param-name
mismatches) in the selection toolbar and storage line; add es strings.
- Remove the underline on user-menu link rows.
Benchmark (uploadStrategies.bench.test.ts) compares baseline / per-file / batch:
the batch collapses N per-file probes into one check (e.g. a WAN 1000-file run
drops from 1700 to 1001 round trips) while matching per-file's byte savings.
Also includes in-progress group virtual-description i18n work present in the
working tree (groups.ts, ResourceList, locale `groups` keys).
Verified: cargo clippy -D warnings (clean), backend 448 tests; frontend
npm run check (clean), 58 unit tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bundles the backend+frontend delta-upload (content-dedup) feature with a
batch of frontend fixes from this session.
Upload / dedup:
- Client-hashed delta & instant upload (deltaUpload, hashWasm vendor shim)
- Backend dedup batch endpoint (dedup_service, dedup_handler, routes)
- session store owned-hash helpers; unit tests + upload-strategy bench
Frontend UI/UX:
- Colour file-type icons in grid/list (per-type tinted tiles + glyph hue)
- Robust thumbnail fallback; PDFs now show their type icon (backend
generates no PDF thumbnails) instead of a blank tile
- Fix PDF preview: load via a same-origin blob: iframe — the API URL is
blocked by the global X-Frame-Options: DENY in the browser's framed
PDF viewer, matching the existing CSP `frame-src blob:` design
- Groups: localized virtual-group description (no DB schema-note leak),
add nav.groups to the 15 missing locales, fix primary-button contrast
- Repoint --color-text-light → --color-on-accent (was faint grey on accent)
- Nudge the admin role badge off the user-menu header divider
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backend — tail latency & throughput:
- FileContentCache, image transcode, and search now use moka single-flight
(try_get_with / get_or_load) so N concurrent misses for the same key
collapse to one disk read / transcode / query instead of a thundering herd.
Microbenchmark (128 concurrent on one hot key): 128 loads / p99 ~1023ms
before vs 1 load / p99 ~32ms after.
- DB: configurable per-statement timeout on the primary pool
(OXICLOUD_DB_STATEMENT_TIMEOUT_SECS, default 30; maintenance pool exempt) so
a runaway query can't pin a connection and starve the pool.
- DB: background pool-saturation monitor
(OXICLOUD_DB_POOL_MONITOR_INTERVAL_SECS) that WARNs as the primary pool nears
exhaustion — the early signal before tail latency cliffs.
- mimalloc: set MIMALLOC_PURGE_DELAY=0 (Dockerfile + compose) so freed pages
return to the OS and RSS tracks the live working set; benchmarked on
musl/aarch64 at ~400MB reclaimed vs 0MB with the default.
Frontend — UI / i18n fixes:
- i18n: fix literal "{{count}}" and "{{percentage}}/{{used}}/{{total}}" in the
selection toolbar and storage line — the call sites passed param names that
didn't match the locale placeholders; unify on `count` and pass the storage
template its params. Add es files.selected_count.
- sidebar: hide the drive picker when there's only one drive (the redundant
"Personal" row); remove the coloured left accent on the active nav item.
- logo: stop clipping the cloud's left bulge — viewBox recentred on the cloud's
true bbox with proportional SVG size so it keeps the same rendered scale.
- user menu: drop the default <a> underline on the link rows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Defer loading components that are off the initial render path until they
are first needed, keeping them out of the bundle that loads on page entry.
- Add `lazyComponent` composable: a tiny rune-based holder that dynamic-
imports a component on first `load()` and exposes it for `{@const}`
rendering, with the component type inferred from the module so prop/
binding type-checking stays intact.
- CommandPalette: loaded on the first Cmd/Ctrl+K and mounted open via a
new `autoOpen` prop. It was previously imported by AppShell, i.e. in the
initial chunk of every authenticated route (~27 KB). AppShell now owns
the shortcut that triggers the load.
- FileViewer + WopiEditor: loaded when a preview/editor first opens, across
files, recent, favorites and shared-with-me.
- PhotoLightbox / PlacesMap / PeopleView: loaded on first lightbox open or
when the places/people tab is selected (the latter also defers maplibre).
Once loaded a component stays mounted and behaves exactly as before, so
behavior is unchanged on the second use onward.
Last parity gap from the VanillaJS → Svelte migration (issue #500),
frontend-only — the backend endpoints already exist.
- Drag-out download: onItemDragStart now also sets the DataTransfer
`DownloadURL` type, so dragging a row/selection onto the OS desktop
downloads it — a single file directly (GET /api/files/{id}), a folder as a
zip (GET /api/folders/{id}/download?format=zip), and a multi-selection as
one server-zipped archive via the GET twin GET /api/batch/download
(DownloadURL can only point at a GET URL). The zip name is shared with the
in-app batch download via a new batchZipName() helper.
- Folder drag-drop upload: onDrop now walks dropped directory trees with
webkitGetAsEntry/createReader into {file, relativePath} rows and recreates
the tree server-side, instead of dropping only a folder's top-level files.
The recursive-upload core is extracted into uploadTree() and shared with the
folder picker (onUploadFolder), so both paths behave identically. Plain
multi-file drops keep the existing flat fast-path.
Stacked on #502 (shares onItemDragStart). Frontend gate green (svelte-check
0/0, eslint, stylelint, prettier) + 47 Vitest.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two parity gaps from the VanillaJS → Svelte migration (issue #500):
1. URL anchor when viewing a file. Opening a file now writes `?file=<id>`
to the URL, so a preview is bookmarkable, reload-restorable, and
Back/Forward open/close it. The viewer is driven from the URL via two
effects (URL→viewer with `untrack` so a user close can't be re-opened;
viewer→URL to drop the param on close with replaceState). Replaces the
write-less, load-only `maybeOpenDeepLink` (the `?file` reader that was
effectively dead because nothing ever set the param).
2. Multi-selection drag ghost. Dragging more than one item now sets a custom
drag image: a stack of the first few rows plus a count badge, reusing the
already-ported but orphaned `.drag-preview`/`.dragged-items`/
`.dragged-items-badge` styles. Previously a multi-item drag showed only the
browser's default single-row ghost with no count.
Also adds `static/geo/` to .prettierignore (the bundled minified world
basemap from #499 is a data asset and must stay byte-faithful — it was
failing `prettier --check`, blocking the gate).
Frontend gate green: svelte-check 0/0, eslint, stylelint, prettier, 47 Vitest.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every navigation rebuilt the breadcrumb with one `GET /api/folders/{id}` per path
segment (a depth-D folder = D requests, each no-store) purely to label the trail.
Add an id→name cache, populated wherever a name is already known:
- every listing names its children, so `cacheFolder` records them, and
- `getFolder` records the folder it fetched.
`buildCrumbs` now reads names from the cache and only fetches the ids it hasn't
seen. During normal step-by-step navigation each ancestor was named by its
parent's listing, so the breadcrumb resolves with ZERO extra requests; only a
cold deep-link fetches its unknown ancestors (still in parallel). Folder renames
update the cache immediately so the trail stays correct.
The cache is a small LRU (cap 1000 — names are tiny) and is independent of the
listing cache (names survive a listing invalidation).
Validated: 3 new unit tests (listing populates child names, getFolder records,
rename overwrites) → 46 frontend tests green; npm run check; headless render of
the real files route (list + grid) — no errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8Vb9QHmLZnEMzHz7MrFy6
Every folder navigation re-downloaded the full listing: `listFolder` cache-busted
with `?t=<ts>` + `Cache-Control: no-store`, so back/forward and re-entering a
folder always paid a full round-trip + payload.
Now the files browser caches listings in memory and serves SWR:
- On navigation it paints a previously-visited folder instantly from cache, then
revalidates with `If-None-Match` (the backend ETag covers folders + files +
favorite/share badges, so it's a faithful validator). Unchanged → 304 with an
empty body; changed → 200 refreshes cache + UI.
- A generation token guards against a slow in-flight response clobbering a newer
navigation; breadcrumbs now resolve independently so they never block the grid
paint.
- Mutations (create/upload/rename/move/copy/delete, incl. the move dialog) go
through `reload()`, which drops the cache and refetches fresh — no stale view
after an action.
API layer (`folders.ts`):
- `fetchFolderListing(id, { etag?, forceRefresh? })` does the conditional fetch
(200 → parsed listing + ETag, 304 → empty); `listFolder` stays as a
non-conditional wrapper for the move-dialog tree.
- A small LRU (cap 40) cache with `getCachedFolder` / `cacheFolder` /
`invalidateFolderCache`. `cache: 'no-store'` keeps the browser HTTP cache out
of the way; revalidation is driven entirely by our own ETag.
Net: instant back/forward navigation, and an unchanged folder revalidates with a
0-byte 304 instead of re-downloading the whole listing. Validated: 7 new unit
tests (conditional If-None-Match + 304, LRU eviction/recency, invalidation),
npm run check, and a headless render of the real files route (list + grid).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8Vb9QHmLZnEMzHz7MrFy6
The folder listing now carries the favorite/share badge state for exactly the
items it returns, so the files browser stops fetching favorites and outgoing
shares separately. This removes the last per-navigation badge round-trips AND
fixes the correctness hole of the previous approaches: badges were derived from
only the first 200 global favorites / shares, so a favorited or shared item
outside that window showed no badge. Now every listed item is correct, and the
work is scoped to the items on screen.
Backend (`GET /api/folders/{id}/listing`):
- `FolderListingDto` gains `favorite_ids` and `shared_ids` (sorted) — listing-
level metadata, so no churn to the many FileDto/FolderDto constructors.
- The handler computes both with two batched, index-backed queries run
concurrently: `FavoritesService::favorited_ids` (auth.user_favorites, ANY) and
`PgAclEngine::shared_resource_ids` (storage.role_grants by granted_by + ANY,
which already covers public links as 'token' grants — same membership the
/grants/outgoing/resources endpoint exposes). Both fold into the ETag.
- Public-share browsing passes empty sets (anonymous, read-only context).
Frontend:
- `listFolder` reads `favorite_ids` / `shared_ids`; the files view seeds local
badge sets straight from the listing and updates them optimistically on
favorite toggle / batch / share creation (via ShareDialog's `onshared`).
- Removes the session `badges` store + its fetches entirely — the listing is now
the single, authoritative, fetch-free source.
Net: favorite/share badges cost zero extra client requests per navigation and
are correct regardless of how many favorites/shares the user has. Validated:
cargo check + clippy -D warnings (backend; integration tests need Postgres,
unavailable here), frontend npm run check + unit tests, and a headless render of
the real files route (list + grid) with the new flags present — no errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8Vb9QHmLZnEMzHz7MrFy6
The files browser re-fetched the first 200 favorites AND the first 200 outgoing
shares on every folder navigation (two round-trips each time) just to render the
star / shared badges — work that grew with how much the user browsed, for data
that barely changes.
Move both id sets into a session-scoped `badges` store: `ensureLoaded()` fetches
once (concurrent callers share one in-flight request) and every later navigation
reads from cache, so browsing costs zero extra requests. Mutations keep the cache
in sync optimistically:
- favorite toggle / batch-favorite → `setFavorite` (revert on failure),
- share creation → `markShared`, wired through a new optional `onshared` callback
on ShareDialog (fired when a grant or public link is created). This also makes
the shared badge appear immediately instead of only after re-navigating.
Net effect per session: badge fetches drop from O(navigations) × 2 to 2 total.
The 200-item ceiling is unchanged from before; the fully-correct fix is per-item
flags on the listing endpoint (a backend change, noted in the store).
Verified: new badges store unit tests (load-once, concurrent de-dupe, optimistic
favorite/share, reset) and a headless render of the real files route in list and
grid (virtualization intact, no runtime errors).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8Vb9QHmLZnEMzHz7MrFy6
Extends windowing to the remaining O(n)-DOM surfaces: the card-grid view of
ResourceList (recent / favorites / shared / shared-with-me / trash / search) and
the main file browser (`files/[...path]`), in both list and grid layouts.
- VirtualList gains a real grid mode: its inner window carries the caller's grid
class (`files-grid-view`) and lays out `columns` cards per windowed row. The
row pitch is auto-measured (grid card height tracks column width via the 4/3
aspect-ratio thumbnail) and re-measured on resize.
- `useVirtualWindow` now distinguishes scroll from resize and exposes a
`resizeTick`, so size-dependent layout (the grid pitch) is recomputed only when
it can actually change.
- `gridColumns(width)` (new util) mirrors the CSS `auto-fill` / `--grid-card-min`
/ gap so the windowed row count matches the browser's real wrapping exactly;
shared by both grid callers.
- The files browser flattens folders-then-files into one discriminated `entries`
list rendered through VirtualList (list: columns=1; grid: columns from width).
Grouped (swimlane) views stay fully rendered, as before — they're bounded.
ResourceList GRID, headless Chromium (1280x900), synthetic rows, before/after:
rows | mount→paint | DOM nodes | JS heap | scroll frame | jank frames
------+-------------+-----------+---------+--------------+------------
1000 | 979→75 ms | 19k→879 | 16→3 MB | 16→17 ms | 0→0
5000 | 4005→67 ms | 95k→879 | 72→3 MB | 37→17 ms | 26→0
20000 |12015→65 ms | 380k→879 |281→7 MB |197→17 ms |1249→19
Rendered DOM and heap are flat (O(visible)) regardless of dataset size; mount is
~185x faster and scroll holds ~60fps. Verified visually mid-scroll (5 columns,
4/3 thumbnail tiles, cards land at the expected indices). The files browser
shares the same VirtualList path (it can't be mounted headless — it depends on
$app routing/session — so it's validated via svelte-check + the production build
+ the shared, separately-benchmarked component).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8Vb9QHmLZnEMzHz7MrFy6
DioCrafts