The thumbnail ETag was "thumb-{file_id}-{size}-{format}", sent with
Cache-Control: public, max-age=31536000, immutable. Replacing a file's
content preserves its id — file_upload_service rebuilds the entity with
parts.id and a new hash, then fires on_file_updated, which deletes and
regenerates the thumbnails — so the server produced a new thumbnail while
still advertising the old ETag. Because `immutable` tells a conforming
browser not to revalidate at all inside the freshness window, clients kept
rendering the previous image for up to a year, unfixably.
Keyed on the content hash the directive becomes honest: a thumbnail is a
pure function of (source bytes, size, format), so that triple identifies
the response. New content yields a new ETag.
The same change fixes the opposite direction. A copy, or any dedup twin,
had a different id and therefore a different ETag, so clients refetched
bytes they already held even though both are served from the same derived
blob. Now identical content agrees on an ETag and revalidates to 304
across files, users and copies.
Both thumbnail endpoints were affected: the REST handler and the
NextCloud preview handler.
Cost is one PK lookup ahead of the 304 decision, where the id-keyed
version needed none — paid for by no longer serving stale images. It is
partly recovered: both handlers already resolved the same hash further
down for the render path, and that second lookup is now gone, so the
cache-miss path is unchanged and only the 304 path pays. The resolved
hash is also handed to get_cached_thumbnail instead of None, saving the
service its own lookup.
No new disclosure: content_hash is already on FileDto and returned by
GET /api/files/{id}.
Tests: thumbnail_etag_content_keyed.hurl covers invalidation — overwrite
in place via WebDAV PUT, assert the ETag changed, assert a client holding
the stale one gets 200 rather than 304. derived_blob_copy.hurl gains the
sharing direction: a copy answers with the SAME ETag and revalidates to
304, which is the one externally observable consequence of content-keying
and was not previously testable.
Step 8 of docs/plan/derived-blobs.md. "What follows a file on copy" was
written twice — the copy_file CTE and storage.copy_folder_tree — and had
already drifted: the tree path bumped storage.blobs only, missing
manifests, which was silent data loss on any multi-chunk file. Fixing it
meant writing the same logic a second time. Step 9 adds a file-keyed
satellite table, which would mean a third and fourth.
Two SQL functions:
storage.add_blob_references(TEXT[]) — the manifest-first reference
contract for SQL callers, returning hashes that matched no registry
row. Set-based so the tree path keeps its single-statement cost; a
per-row helper would have made a 10k-file copy 10k calls.
storage.copy_file_satellites(UUID[], UUID[]) — dead properties plus
the blob reference. The body is the copy-semantics declaration: what
is absent (comments, favorites, content-keyed derived rows) is listed
with its reason, so the taxonomy is executable rather than documented
elsewhere and drifting.
Both copy paths now call it. The single-file path becomes a real
transaction, which also fixes the reference being best-effort: a failed
add_reference used to log a warning and leave a copy holding no
reference at all — the exact shape that gets its content reaped. It
cannot be a CTE arm, because data-modifying CTEs share one snapshot and
the function must read the row the INSERT just wrote.
Verified against a scratch PG with all migrations applied: multi-chunk
manifest 1→2, single-chunk alias bumped at manifest level only (the
NOT EXISTS guard), chunks behind a manifest untouched, dead properties
duplicated, length mismatch rejected, repeats counted.
tests/api/derived_blob_copy.hurl covers it end-to-end and answers the
question the copy raises: content_derived_blobs is NOT copied. A copy
carries the same blob_hash, so it resolves the same derived row — the
test asserts byte-identical thumbnails from both copy paths, then
deletes the original, runs GC, and requires both copies to still serve.
That last step only passes if the references are real.
OPAQUE (RFC 9807) implementation (using `opaque-ke` crate)
with opaque authentfication, server will never receive the password (in the auth=password mode)
this is a must have to create trust with users to permit end to end encryption in the future
(we cannot know if user use the same password/passphrase for his asymetric key or his oxicloud auth,
this is why server must never have the password)
pass1: prepare server
add /api/folders/{id}/ancestors
this API to iterate parent up to the drive root or the shared folder
this will help UI to build the breadcrumb in 1 API call
and to identify the root element (is it a drive users has access to or
a shared folder ?)
ui: now only 1 API call is now required to build the breadcrumb
Close the two coverage gaps in the external-mounts feature — the admin
REST surface and the admin UI tab both previously had no automated tests.
Backend (hurl):
- tests/api/external_mounts.hurl — black-box coverage of the admin mount
endpoints (list / create / delete): input validation (empty name,
non-existent host path → 400), create → 201 with the full view, the
mount appears in the list, delete → 204 then 404, and the /api/admin
middleware denies a non-admin on every verb → 403.
- server.env: enable OXICLOUD_ENABLE_EXTERNAL_MOUNTS for the test server;
run.sh: register the new scenario.
Frontend (vitest):
- admin/page.test.ts — opening the mounts tab loads + lists mounts,
the create form calls createExternalMount, and delete goes through the
confirm modal to deleteExternalMount.
Tooling:
- devenv.nix: add b3sum (used by tests/api/run.sh's storage-integrity check).
webdav_patch.hurl and nc_webdav_patch.hurl existed with real coverage
since the original PATCH commits but were never added to
tests/api/run.sh, so just api-test/CI silently skipped them. Wire both
in, fix nc_webdav_patch.hurl's header-after-[BasicAuth] ordering bug
that meant it had never actually passed, and add two new
consistency-focused files chaining PATCH operations with
cross-protocol/cross-surface verification:
- webdav_patch_consistency.hurl: chained overwrites with ETag-change
checks, GET/HEAD/PROPFIND cross-protocol agreement, quota-507
leaving the file byte-for-byte unchanged, direct_put_max_bytes
prefix/suffix regression coverage.
- nc_webdav_patch_consistency.hurl: Editor/Viewer/Outsider permission
matrix, cross-surface lock interop, quota-507 via the NC surface.
Running these surfaced two real bugs in the NC PATCH handler, both
fixed here:
- The write step mapped every error (including a legitimate anti-enum
permission denial) to a raw 500 instead of AppError::from(e), unlike
the plain surface. A Viewer without Update permission got a 500
leak instead of the expected 404.
- nc_to_internal_path() didn't strip the leading '/' that chroot.path
carries from StoragePath::to_string(), so a LOCK taken via /webdav/
silently failed to block PATCH via /remote.php/dav/ on the same
file — the exact-string lock-store lookup never matched. Added a
regression unit test.
automatically create default Calendar and default addressbook per user
(no creation if user already have a such resource)
default name are "Personal"
this is using the user's life cycle like does the drives
answers to issue #545
- add resource kind filter (file, folder, drive) in shared section (localStorage stored)
- add user preferences serverside store
- add client side dotfile filter (show/hide dotfiles) (user perf stored, default: dotfiles are shown)
for security trashed dotfile are always displayed
protection added: if a folder has only hidden items, a notification invite user to display it
if a user rename or create a hidden item, a notification tells it to user
ensure OIDC is supported during nextcloud login
flow is:
1. nextcloud
2. oxicloud login ( direct pass or OIDC according config)
3. drive picker (if user has multiple drive)
4. success page + backchannel login to nextcloud
resolve_quota only ever reported the caller's personal envelope,
ignoring the drive_id already resolved at every PROPFIND call site —
shared drives with their own quota showed the wrong numbers. Adds
AppState::resolve_webdav_quota, shared by both WebDAV surfaces:
nil drive_id or personal drive -> account envelope, shared drive ->
its own storage.drives quota/used_bytes.
Also adds quota-used-bytes/quota-available-bytes to the NextCloud-
compatible surface, which previously had no RFC 4331 support at all.
Registers webdav_quota_properties.hurl and the new
nc_webdav_quota_properties.hurl in tests/api/run.sh — neither was
wired into the suite before this change.
add env variable `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX`
which is by default:
`OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX="@drive"`
so `/webdav/` -> points to user's personal drive (**backward compatibilit**y)
`/web/dav/@drive/{uuid|drive name}/` points to the respective drive
if admins want directly `/webdav/` pointing to list of drives they need to:
`OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX=""`
+ ensure lock is per user (RFC 4918 §9.11)
fix: #554
Native + NC surfaces: DAV: displayname/getetag, oc:fileid/
permissions, nc:has-preview all 403 and never land in the store;
mixed request shows per-property granularity (protected prop 403
alongside an ordinary custom prop 200); oc:favorite regression
guard confirms its special-case still works despite being on the
protected list.
this test is a duplicated with the move in litmus test suite
it has been added because not everybody do have litmus installed
and no clue if litmus will be kept in the future
Edouard Vanbelle