The global `script-src 'self'` CSP header blocked every inline <script>,
so the SvelteKit hydration bootstrap never ran and the SPA never mounted
(blank page behind the splash spinner). The anti-FOUC theme init in
app.html was blocked too.
Instead of weakening the policy with 'unsafe-inline', the backend now
builds `script-src` at startup from 'self' plus a SHA-256 hash of every
inline <script> in the served HTML shells. The policy stays strict (no
'unsafe-inline' for scripts) and the hashes are recomputed from the built
assets on each startup, so a frontend rebuild needs no header edit — even
though SvelteKit's bootstrap hash changes every build.
- web::content_security_policy builds the header; web::resolve_static_path
is extracted so serving and hashing read the exact same bytes.
- byte-exact inline-script extraction (skips src= externals), unit-tested
against a known SHA-256 vector and extraction edge cases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The SvelteKit migration gated build.rs's static-dist/OUT_DIR generation
behind OXICLOUD_RUST_ASSETS=1 (early return), but login_v2_handler.rs
still embedded the page with
`include_str!(concat!(env!("OUT_DIR"), "/nextcloud-login.html"))`. With
OXICLOUD_RUST_ASSETS unset (the default), that file is never written to
OUT_DIR, so a clean `cargo build` failed to compile. (#489)
Migrate the page off include_str! to an askama template
(templates/nextcloud/login.html), mirroring the existing
DrivePickerTemplate in the same handler. This drops the only
compile-time dependency on the legacy build.rs pipeline, so the
OXICLOUD_RUST_ASSETS=1 CI workaround is no longer needed and is removed
from ci.yml, load-smoke.yml and load-nightly.yml.
Also fix the second failure on #489: with OXICLOUD_RUST_ASSETS=1 the
release pipeline panicked in copy_dir_recursive because `static/locales`
is now a symlink to frontend/static/locales. entry.file_type() reports
the link itself (not its target), so the symlinked directory was routed
to fs::copy and failed with "the source path is neither a regular file
nor a symlink to a regular file". Classify entries with fs::metadata,
which follows symlinks, so symlinked directories are traversed.
static/nextcloud-login.html is removed (its content moved into the
template; no other consumer) and dropped from build.rs HTML_INCLUDE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CNCEMN6fC2xSmxqCVbstkd
The folder listing now carries the favorite/share badge state for exactly the
items it returns, so the files browser stops fetching favorites and outgoing
shares separately. This removes the last per-navigation badge round-trips AND
fixes the correctness hole of the previous approaches: badges were derived from
only the first 200 global favorites / shares, so a favorited or shared item
outside that window showed no badge. Now every listed item is correct, and the
work is scoped to the items on screen.
Backend (`GET /api/folders/{id}/listing`):
- `FolderListingDto` gains `favorite_ids` and `shared_ids` (sorted) — listing-
level metadata, so no churn to the many FileDto/FolderDto constructors.
- The handler computes both with two batched, index-backed queries run
concurrently: `FavoritesService::favorited_ids` (auth.user_favorites, ANY) and
`PgAclEngine::shared_resource_ids` (storage.role_grants by granted_by + ANY,
which already covers public links as 'token' grants — same membership the
/grants/outgoing/resources endpoint exposes). Both fold into the ETag.
- Public-share browsing passes empty sets (anonymous, read-only context).
Frontend:
- `listFolder` reads `favorite_ids` / `shared_ids`; the files view seeds local
badge sets straight from the listing and updates them optimistically on
favorite toggle / batch / share creation (via ShareDialog's `onshared`).
- Removes the session `badges` store + its fetches entirely — the listing is now
the single, authoritative, fetch-free source.
Net: favorite/share badges cost zero extra client requests per navigation and
are correct regardless of how many favorites/shares the user has. Validated:
cargo check + clippy -D warnings (backend; integration tests need Postgres,
unavailable here), frontend npm run check + unit tests, and a headless render of
the real files route (list + grid) with the new flags present — no errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8Vb9QHmLZnEMzHz7MrFy6
remove create_home_folder() & ensure_home_folder()
now: on_user_created() and on_user_login both() call provision_if_needed()
which calls **create_personal_drive_atomic()**
add a helper to find Personal drive for a user and also it's root directorry
Bring chroot to nextcloud capability: login on Nextcloud via username="{user}~{folder-uuid}
Doing a such login will chroot the folder folder-uuid
If user has several folder as root (parent=None), the login flow will request which
folder user want to chroot
now Drive is purely a metadata
each drive has always a root folder
this model minimize Oxicloud changes, and simplify
the Drive name is simply the folder's root's name
note: owner of Drive has more permission that an owner of the root folder
Bring the feature-rich main branch into the frontend Svelte rewrite
(PR #478, base bcn/frontend-svelte-rewrite). main moved well ahead of the
PR's branch point (b8a0018): it added the Places photo-map and People
(faces) backends, photos enhancements, the ReBAC→role-grants migration,
load tests, and more.
Conflicts resolved (4 files):
- Dockerfile: combine the explicit --bin allowlist (defence-in-depth from
main) with the SPA copy from the frontend build stage (PR).
- .github/workflows/ci.yml: keep the PR's Svelte frontend job
(svelte-check + eslint + stylelint + prettier + vitest); the legacy
static/-targeted tsc/locale/icon advisory steps don't fit the new
working-directory: frontend job and svelte-check supersedes them.
- justfile: keep both the new fe-* / dev recipes (PR) and the load-* k6
recipes (main).
- static/locales: keep the PR's symlink (-> ../frontend/static/locales);
main's new photos/people locale keys are folded into the Svelte locale
files alongside the ported views.
Backend (people/places/faces handlers, routes, DI, migrations) merged
cleanly. `cargo check --bins` passes. The new Places/People UI is not yet
in the Svelte app; that is ported in follow-up commits.
Phase 2 increment 6 — the People HTTP API (mounted only when
OXICLOUD_ENABLE_FACES is on; every handler is caller-scoped):
- GET /api/people list identity clusters
- GET /api/people/{id}/photos a person's photo file ids
- PATCH /api/people/{id} name / rename a person
- POST /api/people/{id}/hide hide / unhide
- POST /api/people/merge merge two clusters
- POST /api/people/recluster re-run clustering
- DELETE /api/people/data erase all face data (opt-out)
- GET /api/people/faces/{file_id} face boxes for lightbox tagging
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JW6ghFMDtnRYuYNzZhb47M
Three list endpoints resolved each resource with one query per id:
- GET /api/grants/incoming and /api/grants/outgoing used
join_all(ids.map(get_file)) + join_all(ids.map(get_folder)), so a single
page (limit ≤ 200) could demand ~200 concurrent connections from the
20-connection primary pool, causing acquire-timeouts and head-of-line
blocking under load.
- The NextCloud favorites REPORT (oc:filter-files) fetched get_file/
get_folder once per favorite — up to N serial round-trips per sync.
Add by-ids batch reads that mirror the existing get_file/get_folder column
mapping and NOT is_trashed filter:
- FileBlobReadRepository::get_files_by_ids / FolderDbRepository::get_folders_by_ids
(one SELECT ... WHERE id = ANY($1)), exposed as FileRetrievalService::
get_files_by_ids / FolderService::get_folders_by_ids returning DTOs.
- Both grant handlers and the favorites REPORT now issue two batch queries
total and look results up by id, preserving original order. Missing ids
(stale grants whose resource was deleted, or trashed/removed favorites)
drop out exactly as before. No auth-semantics change: these paths already
resolved ids vetted by the authorization engine / favorites table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAzLEQDaLak3dnrEN3YT35
JWT access tokens freeze role/identity at login (access 1h, refresh 7d)
and validated tokens are cached for 30s. The Bearer and cookie auth paths
trusted claims.role and never re-checked the account, so demoting an admin,
or disabling/deleting an account, did not revoke access until the token
expired. The app-password path already re-read role/active from the DB;
only the JWT/cookie path had the gap.
Re-validate the caller against the live user record on the token path via
the already-cached get_user_flags (role / is_external / active), bounded by
USER_FLAGS_CACHE_TTL and invalidated eagerly on set_user_active /
change_user_role / delete_user_admin:
- middleware/user.rs: new resolve_live_role helper (+ pure decide_live_role
core) — returns the *current* role, rejects deleted (NotFound) and
deactivated accounts, and fails open on transient lookup errors (mirrors
require_internal_user). Login/refresh remain the canonical active gate.
- middleware/auth.rs: auth_middleware (Bearer + cookie) now populates
CurrentUser with the live role and rejects revoked accounts (Bearer ->
401 AccountInactive; cookie -> fall through to 401/login redirect).
require_admin emits an audit line on denial.
- middleware/admin.rs: require_admin / require_authenticated re-check the
live record, return the live role, and audit admin denials.
Downstream admin gates (dedup_handler, subject_group_handler, OCS) inherit
the live role automatically via CurrentUser / require_authenticated.
Tests: decide_live_role policy (active / demoted / deactivated / deleted /
transient fail-open) and AccountInactive -> 401.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAzLEQDaLak3dnrEN3YT35
Phase 1 server side, gated on OXICLOUD_ENABLE_PLACES (off by default):
- migration: partial index on storage.file_metadata(longitude, latitude).
- FileBlobReadRepository::list_geo_clusters — plain-SQL grid aggregation
(no PostGIS) scoped to the caller's own non-trashed photos, returning a
centroid, count and a representative file id per non-empty cell.
- PlacesService (caller_id-scoped; user-scoped data needs no authz check,
mirroring RecentService) with a zoom→cell-size mapping.
- GET /api/photos/geo?bbox=w,s,e,n&zoom=N returning GeoCluster[]. The route
is mounted only when the Places service is present, and is registered in
the OpenAPI path list.
The map frontend (PMTiles serving + MapLibre module) is deferred pending
the basemap-sourcing and vendoring decision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JW6ghFMDtnRYuYNzZhb47M
list_media_files now LEFT JOINs storage.file_metadata and returns each
photo's pixel dimensions next to the sort date. The endpoint wraps FileDto
in a flattened PhotoDto carrying width/height, so the gallery can lay tiles
out at their true aspect ratio (justified layout) without a second per-file
metadata round-trip and without layout shift. FileItem gains optional
width/height. No change to FileDto or its other construction sites.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JW6ghFMDtnRYuYNzZhb47M
Standard CalDAV/CardDAV clients (Thunderbird, DAVx5, Apple
Calendar/Contacts) failed to connect, mounted collections read-only, or
could not discover address books, even though curl worked. Three
protocol-compliance gaps caused this:
1. Missing Basic-auth challenge on /caldav and /carddav.
The 401 returned for these surfaces carried no `WWW-Authenticate`
header (only /webdav did). Spec-compliant clients never send
credentials preemptively the way `curl -u` does — they wait for the
challenge — so Thunderbird never authenticated and failed with
"discovery failed" / 401. Extend the challenge to all DAV surfaces via
shared `is_dav_path` / `dav_basic_auth_challenge` helpers.
2. Calendars always advertised read-only.
The `current-user-privilege-set` write gate compared `owner_id`
against the literal string "current_user_id", which never matched a
real UUID, so `<D:write/>` was never emitted and clients mounted every
calendar read-only. Thread the caller's id through the CalDAV adapter
and grant write when the caller owns the calendar.
3. CardDAV discovery was incomplete.
There was no `/.well-known/carddav` route and the root PROPFIND
exposed neither `current-user-principal` nor `addressbook-home-set`,
so clients could not locate address books. Add the well-known redirect
and root/principal discovery responses mirroring the CalDAV adapter.
Adds unit tests for the auth challenge predicate, the calendar
owner/non-owner privilege split, and the CardDAV root/principal discovery
responses.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016cVV9nRQjP6G6a8zbNUWMw
prepare migration of permission to roles
this simplify drastically database (permission are now simply defined in code)
and will permit reuse of the same ReBAC engine to define owners of drives
mapping:
```
Role::Viewer => &[Permission::Read],
Role::Commenter => &[Permission::Read, Permission::Comment],
Role::Contributor => &[Permission::Read, Permission::Create],
Role::Editor => &[
Permission::Read,
Permission::Comment,
Permission::Create,
Permission::Update,
],
Role::Owner => &[
Permission::Read,
Permission::Comment,
Permission::Create,
Permission::Update,
Permission::Share,
Permission::Delete,
Permission::Manage,
],
```
handle_head was declaring `Content-Length: file.size` while writing
`Body::empty()` — on a keep-alive connection the client waits forever
for N bytes that never come. Hyper now derives Content-Length: 0 from
the actual body, which is honest about what's on the wire.
RFC 7231 §4.3.2 suggests HEAD return the same headers as GET, but
lying about Content-Length is worse than omitting it: NC and Sabre
clients use PROPFIND for size anyway, and curl -I (and any client
applying HEAD semantics) gets the same ETag/MIME/Last-Modified it
needs. Caught by the F6b test which uses `curl -X HEAD` to read the
current ETag before a conditional PUT.
Also adds `nc_status_propfind_depth0` to lib/dav_helpers.sh so the
F11/F11b assertions ("did the intermediate parent get auto-created?")
can compile.
When the client sends `MOVE /trashbin/{id}` with a `Destination` header,
handle_restore now resolves the destination path and returns 412
Precondition Failed if a live file or folder already sits there —
matching Sabre/DAV and the NC desktop client's expectation. There is
no `Overwrite: T` workflow for trash restore in either reference
implementation (silently replacing a live file with an undeleted one
is a footgun), so the refusal is unconditional.
The destination header is extracted at the dispatch site as an owned
String so the future stays Send-compatible (`&Request<Body>` is not
Sync because the body trait object is Send-only).
`extract_nc_subpath_from_dest` is promoted to `pub` so trashbin_handler
can share the same URL parser as handle_move.
Threads `new_name: Option<&str>` through FileWritePort::copy_file and
FileManagementUseCase::copy_file_with_perms so a same-folder
COPY /a.txt → /b.txt picks up the destination name via a single
COALESCE($3::text, name) in the CTE. Without it the new row inherits
the source's filename and collides on the (folder_id, name, user_id)
unique index — the "Already Exists" 500 M8 was hitting.
handle_copy in the native WebDAV surface now passes
`(file.name != dest_name).then(|| dest_name.into())`, keeping the
"same name in a different folder" case at None so existing semantics
are preserved.
Extends the N2/PUT lock guard introduced earlier to the rest of the
native mutator surface. Same helper, same If: capture before body
consumption, same 423-on-reject shape:
- handle_delete : check source path
- handle_proppatch : check source path
- handle_move : check source AND destination paths
- handle_copy : check destination path only (source isn't
modified by a copy)
The class-2 DAV advertisement in OPTIONS is now honest across the
full surface, not just PUT.
New tests N2c-N2f run while n-locked.txt is still LOCKed (before the
existing N3 UNLOCK). Each asserts 423 without the token and verifies
the operation didn't half-apply: file present after DELETE-423,
source untouched + no destination after MOVE-423, locked destination's
content unchanged after COPY-423.
Positive (with-token) coverage is implicit via the M-series happy-
path tests that exercise each method on unlocked resources — a
regression that hard-rejected every call would fail there too.
Closes F11.
The NC MKCOL handler previously had `mkdir -p` semantics:
sending MKCOL on /a/b/c/ where neither a nor b exists silently
created both intermediates and returned 201. Sabre/DAV and the
actual NC server both 409 on that — our auto-create deviated.
NC desktop walks ancestors one MKCOL at a time during sync so
nothing real depended on the old behaviour.
Drop the segment-walking creation loop. New flow:
target exists → 405
parent path missing → 409
parent ok, target new → 201
The race-recovery branch for the loop's per-segment create is
also gone — single parent lookup, single create, no window.
Test F11 flipped from pinned-201 to strict 409 and asserts the
intermediate parent was not silently created. F11b and F11c added
as regression guards for the success path and the 'target already
exists' case.
Closes M5 / M7 / M8a / M8b.
The optimized PathResolver and the read-side find_*_by_path queries
disagree on what counts as 'a path that hits a row'. After the drive-
refactor migration rewrote the path column to drop the
"My Folder - <user>/" prefix, files PUT through the WebDAV surface
stayed reachable by GET (legacy lookup) but vanished from the
optimized resolver (strict path-match). MOVE/DELETE/COPY 404'd on
every root-level file as a result.
Introduces resolve_or_legacy: optimized resolver first, then the
GET-style legacy lookups as a strict superset. Ownership is enforced
in both branches. handle_delete / handle_move / handle_copy each
collapsed from two near-identical resolver-only + legacy-only branches
into a single match using the helper — fewer lines, identical
semantics, root-level paths now resolve.
handle_copy also fixes M8b: copy_file_with_perms takes no destination
name, so a copy to a different filename in the same folder collided
with the source. After the copy, rename the new file when dest_name
differs from source name. Mirrors what handle_move already does.
Closes G4 / G5 / K5.
handle_move now resolves the destination once before the file/folder
dispatch and applies RFC 4918 §9.9.4:
- Overwrite: F on a collision → 412 Precondition Failed, source
untouched, destination untouched.
- Overwrite: T (or absent) on a collision → delete the existing
destination, then proceed → 204 No Content.
- No collision → 201 Created (unchanged).
The same destination lookup powers the 201-vs-204 status decision, so
adding the precondition guard adds zero extra DB hits on the happy
path.
handle_restore now catches the unique-index collision out of
restore_item and returns 412 instead of letting it bubble as 500.
Mirrors the G4 semantics for the trashbin surface (restore has no
Overwrite header so the refusal is unconditional; client resolves by
renaming the live file first).
Sabre/DAV's CorePlugin and our test pins agreed independently — NC
clients expect this exact behavior, and the new G5b/G5c positive-case
tests guard against a regression that hard-rejected every MOVE.
Closes N2. The native WebDAV PUT handler now consults the lock
store before accepting a write: if the target path is exclusively
locked, the request must carry the lock token in its If: header
or the server returns 423 Locked. Without a matching token, the
body is never consumed — a rejected PUT no longer wastes the
upload bandwidth or hits the CDC ingester.
Two helpers are introduced so the same enforcement plugs into
the other mutator methods (delete/move/copy/proppatch) when their
fixes land:
extract_if_header_tokens — angle-bracket-scoop view of If:
(sufficient for one-target writes;
full §10.4 tagged-list grammar would
only matter for multi-resource Ifs)
enforce_native_lock — Some(423) when locked + no/wrong
token, None otherwise
Test N2 flipped from pinned 204 to assert 423. Added N2b: same
PUT with the captured Lock-Token in If:(<...>) returns 204, so a
regression that hard-rejected every PUT would still fail loudly.
Closes F5/F6. The NC PUT handler now evaluates conditional
preconditions before body ingestion and returns 412 Precondition
Failed when they fail:
- If-None-Match: * on an existing target → 412 (create-if-absent)
- If-None-Match with matching ETag → 412 (weak compare)
- If-Match: * with no current representation → 412
- If-Match with no listed ETag strong-matching the current → 412
The lookup that drives the precondition reuses the same query the
handler already needed for the 201-vs-204 distinction, so this adds
no extra DB round-trip. Rejected requests skip body ingestion
entirely so a 412 doesn't waste megabytes of bandwidth + disk I/O.
Test F5/F6 flipped from 'pinned current 204' to assert 412, plus
mirror cases F5b/F6b/F6c/F6d covering the legitimate-success paths
so logical-operator regressions can't slip past silently.
- permit reuse of blob where file is trashed (by-hash and chunked)
- add hurl test on /api/files/by-hash
- add anti enumeration of blob (404 is always blob_not_owned_by_caller)
Backend — Photos timeline now groups by real capture date instead of upload time. New MediaMetadataService (FileLifecycleHook) extracts EXIF DateTimeOriginal from images and container creation_time from videos (mov/mp4/mkv) via nom-exif, timezone-correct (OffsetTimeOriginal), persisting captured_at so the existing media_sort_date trigger takes over. Adds POST /admin/photos/metadata/reextract to backfill existing media. Falls back to upload date when no embedded date exists.
Frontend — premium grid cards: combined metadata line (relative date · size, owner avatar when shared), custom selection checkbox with a clear checked state, uniform full-width 4:3 thumbnail tiles independent of filename length, centered file-type icons, and a hit-test fix so checkbox/star/kebab clicks reach the controls (the decorative thumbnail no longer captures pointer events). Notification messages internationalised across all 16 locales. Broader polish: design tokens, a11y/focus-visible states, brand + PWA assets.
Chore — bump semver-compatible dependencies (cargo upgrade); add nom-exif 3.6.1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 3 of the delta-sync plan — the inverse direction, so a future
client app holding an older local version can fetch only what changed:
- GET /api/files/{id}/manifest returns the file's chunk recipe
({file_hash, total_size, chunks}). Owner-scoped like the rest of the
delta surface (Read permission through the authz engine first, then
the chunk layer's possession standard; shared files use the regular
download endpoints). A manifest is immutable for a given file_hash,
so it is served with ETag = file_hash and If-None-Match answers 304 —
polling sync clients pay one header round-trip per unchanged file.
Legacy pre-CDC blobs are presented as a single-chunk manifest of
themselves, so clients need no special case.
- POST /api/files/delta/download streams the requested chunks as
[u32 BE length][bytes] frames in request order — the same wire format
the upload direction uses. Entitlement is the same possession rule as
negotiate/commit (chunks reachable through the caller's own files);
anything else returns 404 {not_available} — deliberately
indistinguishable from "never existed" — with a
delta_download.rejected audit event. Batches are bounded by the
chunk_max_bytes budget; Content-Length is exact (sizes come from the
dedup index) and peak RAM is one backend read frame.
Both endpoints share the delta rate limiter. New DedupService
primitives: manifest_chunk_list (with legacy fallback), chunk_sizes,
chunk_stream. OpenAPI regenerated; protocol doc gains the download
section; types.js maps the new wire shapes (plus the delta-upload
typedefs that a container reset had silently dropped from a previous
commit).
Verified end-to-end against PostgreSQL 16 with a simulated two-device
sync: device A uploaded 24 MB by bytes and delta-updated it (2 edits →
2 chunks); device B diffed the manifest against its WASM-chunked local
copy, needed 2/79 chunks, fetched 970 KB instead of 24 MB (96.1%
saved) and rebuilt the file byte-identical with the BLAKE3 verifying.
If-None-Match revalidation returned 304; a second user got 404 on both
the manifest and the chunk batch (with the not_available list and
audit lines); an unknown hash was indistinguishable from a denied one;
an empty hash list returned 400.
https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
Phase 1 of the delta-sync plan, server side. The CDC store already shares
unchanged chunks between file versions after the bytes arrive; these three
stateless endpoints move that detection to the client, so editing a few
bytes of a large file uploads ~1 MiB instead of the whole file:
- POST /api/files/delta/negotiate — given the file's chunk hashes, answer
which ones the caller must upload. User-scoped and purely advisory.
- PUT /api/files/delta/chunks — missing chunks as [u32 BE len][bytes]
frames (streaming parse, ≤1 MiB per frame, chunk_max_bytes per request).
Every hash is recomputed server-side; chunks land as ref_count=0 orphans
that a commit pins or the periodic GC sweeps — no session table.
- POST /api/files/delta/commit — pin one reference per distinct chunk with
a single UPDATE…RETURNING restricted to chunks the caller is entitled
to (reachable through their own non-trashed files, or unreferenced
orphans); anything else returns 409 {still_missing} for the client to
upload and retry. The pinned sequence is then RE-READ and the whole-file
BLAKE3 recomputed before any manifest exists — a declared file_hash is
never trusted, because a forged manifest would poison future whole-file
dedup hits for other users. The manifest accounting is shared with the
byte path (attach_manifest, extracted from store_from_stream); the file
row is created (201) or its content swapped by file_id (200). Owners of
the exact file_hash short-circuit to a pure reference bump.
Supporting pieces: GIN index on chunk_manifests.chunk_hashes (containment
probes were sequential scans), claimable/pin/release/store-loose/verify
primitives on DedupService, update-by-id with Update-permission AuthZ on
FileUploadService, per-caller rate limiter (240/min), audit events with
stable reasons (rate_limited, chunk_verification_failed,
file_hash_mismatch), OpenAPI + docs/delta-upload-protocol.md, framing
parser unit tests and a PG-gated integration suite covering the
entitlement matrix (owned/foreign/orphan/unknown), orphan registration
and the verification read.
Verified end-to-end against PostgreSQL 16 with a node client hashing via
the vendored WASM: a 24 MB file delta-committed in 96 fixed-size chunks;
a 3-byte edit then negotiated missing 1/96 and synced with 278 KB on the
wire vs 24 MB (98.9% saved), downloading byte-identical. A second user
probing the same chunks got nothing (negotiate: all missing; commit: 409
with all 96 still withheld); a forged file_hash returned 400 plus the
audit line; a commit referencing one never-uploaded chunk returned 409
naming exactly that hash; the GIN index serves containment probes
(Bitmap Index Scan) once the planner favors it.
https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
Phase 0 of the delta-sync plan. Re-uploading a file the user already has
(another device, a restore, a duplicate) used to transfer every byte just
for the server to discard them as a dedup hit. The frontend now computes
the file's BLAKE3 locally and, on a hit, registers the file with a single
~150-byte metadata call.
Server — POST /api/files/by-hash:
- All checks live in the application service per the AuthZ rule:
Create permission on the target folder via the authorization engine,
hash ownership via the existing user-scoped query (a non-owned hash
returns 404 — same shape as "no such blob" — and emits an
instant_upload.rejected audit event), quota on the logical size.
- On success: one ref_count bump + the existing save_file_with_blob row
registration (compensation included); is_new_blob=false so lifecycle
hooks skip thumbnail regeneration. ~10 ms warm.
- The storage-usage service is now built before the application services
and injected, instead of only living on AppState.
Client — WASM BLAKE3 + worker:
- wasm/oxicloud-hash: the exact same blake3 crate the server uses,
compiled with WASM SIMD128 (~660 MB/s measured) so browser hashes match
server content addresses bit for bit. Built by scripts/build-wasm.sh;
the artifacts (45 KB wasm + 8 KB glue) are vendored like pdf.js — no
npm dependencies, no wasm toolchain needed for regular builds.
- static/js/workers/hashWorker.js streams the File in 8 MiB slices off
the main thread (constant RAM at any file size).
- features/files/instantUpload.js orchestrates: threshold (8 MiB — below
it the round-trips cost more than the bytes), user-scoped
/api/dedup/check, by-hash registration, and silent fallback to the
normal byte upload on any miss, race or unsupported environment.
Wired into both uploadFiles and uploadFolderEntries.
- biome.json vendors exclusion fixed to cover nested directories
(previous vendors were .mjs and never matched the *.js include).
Verified end-to-end against PostgreSQL 16: node-driven WASM hash equals
the server's content_hash for a 20 MB file; by-hash returns 201 in ~10 ms
warm with a 151-byte request (vs 20,971,873 bytes for the byte upload);
the copy downloads byte-identical and the manifest ref_count goes 1→2;
a second user probing the same hash gets exists:false and 404 plus the
audit line; duplicate name → 409, malformed hash → 400; worker and wasm
are served with correct MIME (application/wasm).
https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
Every upload surface previously wrote each byte to disk twice: the HTTP
body was spooled to a temp file (or assembled from chunk parts), then
mmap-re-read for FastCDC analysis, and finally the new chunks were
written to the blob backend. CDC could not start until the last byte
arrived, so large uploads paid receive + reread + rewrite latency.
The dedup engine now chunks, hashes and settles the stream WHILE it
arrives (fastcdc AsyncStreamCDC + incremental BLAKE3):
- Each batch of distinct chunks is pinned-or-classified by ONE
`UPDATE … RETURNING` (no check-then-bump TOCTOU; pinned chunks can't
be reclaimed mid-upload), and only chunks the store doesn't have are
written — a full dedup hit performs zero content writes.
- Durability before visibility is preserved: one batched fsync sweep,
then one batched INSERT, then the manifest. Identical concurrent
uploads are resolved at the manifest INSERT via ON CONFLICT (the
loser releases its references and becomes a dedup hit).
- A drop guard rolls back pins and surfaces written-but-unregistered
chunks to GC if the request future is cancelled mid-stream.
- MIME sniffing now peeks the first bytes in-flight; client-requested
MD5/SHA-256 checksums are computed by a stream tee — the post-upload
re-read of the assembled file is gone.
All surfaces converge on the new interfaces::upload_ingest helper:
REST multipart, WebDAV PUT, NextCloud PUT, WOPI PutFile, the dedup
endpoint, and both chunked-upload completions (which now stream their
ordered parts straight into the store instead of writing an assembled
file — chunk parts persist until finalize, so completion is genuinely
retryable). The legacy blob re-chunk migration streams from the
backend with no spool file either.
Legacy removed: store_from_file + mmap CDC analysers + temp-path
plumbing through every port (pre_computed_hash, save_file_from_temp,
update_file_content_from_temp), upload_spool + assembled-file
assembly in both chunked services, create_file/update_file byte-slice
variants (no callers), common::temp, the OXICLOUD_UPLOAD_TMPDIR
config, and the memmap2 dependency.
Verified end-to-end against PostgreSQL 16: 8 MB upload (26 chunks),
identical re-upload (dedup hit, zero writes), 3-byte edit re-upload
(26 chunks, 1 written), byte-identical downloads, Range across chunk
boundaries, concurrent identical-upload race (manifest ref 2), and
trash-empty reclaiming exactly the unshared chunk while the shared 25
survive for the edited file. The empty/sub-8KB multipart path found a
post-EOF re-poll panic in the MIME peek (fixed with fuse + regression
test).
https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
JWT validation cache now stores Arc<TokenClaims> and validate_token
returns Arc<TokenClaims>. On a cache hit — the 99% path for every
authenticated request — the moka lookup was deep-cloning the whole
claims struct (5 Strings: sub, jti, username, email, role) on every
call. It is now a refcount bump. Read-only callers (admin middleware)
go through Deref and allocate nothing; the auth middleware clones only
the three fields it moves into CurrentUser (was 5 clones, now 3), and
the admin paths clone only role (was 5, now 1). A new test asserts the
hit path returns a pointer-equal Arc.
TokenServicePort::validate_token is the single trait method touched;
its only implementor is JwtTokenService and the only production callers
are the auth and admin middleware (the WOPI handler uses a separate
WopiTokenService).
Dockerfile: rust:1.94.1-alpine3.23 -> rust:1.96-alpine3.24 and
alpine:3.23.3 -> alpine:3.24.0 for the runtime stage.
https://claude.ai/code/session_0193Hff42gaA962wThxMGSd1
A REPORT multiget for a handful of resources previously listed the
whole calendar/address book (every row incl. ical_data/vcard) and
filtered by href in Rust with an O(N×M) substring scan. Large
collections paid full-table latency, RAM and DB CPU on every sync.
- Add find_events_by_ical_uids / get_contacts_by_uids through every
layer (domain repo trait → PG repo → storage port/adapter → use case
→ handler) using one indexed `= ANY($2)` query, mirroring the
existing single-UID lookups.
- Extract UIDs from multiget hrefs with a shared, tested
uid_from_multiget_href helper (percent-decoding, case-insensitive
extension strip, collection hrefs rejected). Exact UID matching also
removes the false positives the old substring filter allowed.
- CalDAV PROPFIND single-event path now uses the existing indexed
get_event_by_ical_uid instead of listing all events.
- Honour the already-documented limit/offset query params on
GET /api/address-books/{id}/contacts: optional ListQuery fields
thread through ContactUseCase::list_contacts to a paginated repo
query. Omitted params keep returning the full book (frontend and
DAV listing/sync paths unchanged).
https://claude.ai/code/session_0193Hff42gaA962wThxMGSd1
Request-path thumbnail generation (REST get_thumbnail and NC preview)
read the full source blob into RAM and decoded it with no concurrency
bound — a first-view gallery of K images stacked K full-size buffers
and K parallel decodes on the blocking pool. The background hook had
the inverse ordering problem: it read the blob eagerly and only then
queued on the decode semaphore, so N concurrent uploads held N
originals in memory while waiting.
All three paths now acquire the decode semaphore first and read the
blob under the permit, capping peak RAM at permits x image size:
- new ThumbnailService::get_thumbnail_from_blob defers the blob read
into the moka init closure (cache and disk hits never touch the
blob); both handlers use it and no longer pre-read.
- get_thumbnail_from_bytes (direct-bytes variant) now also takes a
permit before decoding; shared generate_and_persist core keeps the
two entrypoints duplicate-free.
- generate_all_sizes_background_from_blob (renamed from _from_bytes)
checks blob existence and disk-dedup state, then acquires the
permit, then reads; the redundant pre-read spawn wrapper in
ThumbnailRefreshHook is gone.
https://claude.ai/code/session_01QxwJDHqQhbMkHK333QtMme
Same whitespace-only reformat as branch claude/jolly-johnson-yso7z7:
PR #326 landed three files that fail cargo fmt --check and its CI run
skipped the Rustfmt job, breaking the check for every later Rust PR.
https://claude.ai/code/session_01GpprjxjtXFYLfXNkoKnHuL
Every CalDAV PUT/GET/DELETE of one .ics loaded the ENTIRE calendar —
every row including its ical_data — and filtered with .find() in Rust,
so importing N events cost O(N²) rows transferred. CardDAV did the exact
same in four places (PROPFIND of one .vcf, PUT existence check, GET,
DELETE), with three JSONB deserializations per discarded contact. The
indexed repo queries (find_event_by_ical_uid, get_contact_by_uid)
existed all along with zero callers.
Wire them end to end: new `get_event_by_ical_uid` /
`get_contact_by_uid` use-case methods (same access checks as
list_events / list_contacts, per the service-layer authz rule) exposed
through the storage ports and adapters, and the seven handler sites now
resolve one row instead of the whole collection.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
Two DAV-surface fixes that replicate patterns the codebase already had:
NC PROPFIND (folder case) previously loaded EVERY child via unbounded
list_files/list_folders and serialized the entire multistatus into one
Vec (~2 KB per entry — a 50k-file folder meant ~100 MB of buffer per
request, repeated constantly by sync clients). It now mirrors the native
WebDAV handler's streaming builder: children are fetched in pages of
PROPFIND_BATCH_SIZE (500), each page's favorites and oc:fileids are
resolved with two batch queries, and the XML is yielded chunk by chunk —
memory stays O(batch) and the first byte flows immediately. The
single-file PROPFIND keeps a small buffered variant; the multistatus
opening tag is factored into a shared helper so the namespace set cannot
diverge.
WebDAV GET (native and NC) ignored the Range header and never compared
the ETag it emitted, so mount-style clients (rclone, davfs2, Finder)
re-transferred whole files on every seek, resume, or revalidation. New
shared `interfaces::range_requests` helpers — same semantics as the REST
download endpoint, which now reuses the 304 helper too — give both GETs
If-None-Match → 304, Range → 206/416, and Accept-Ranges advertising.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
DioCrafts