link are checking that email matches, +email alias are normalize into email
if email is already used on another account, link is not possible
not usurpation risk as the IDP is choosen by the admin
OPAQUE (RFC 9807) implementation (using `opaque-ke` crate)
with opaque authentfication, server will never receive the password (in the auth=password mode)
this is a must have to create trust with users to permit end to end encryption in the future
(we cannot know if user use the same password/passphrase for his asymetric key or his oxicloud auth,
this is why server must never have the password)
pass1: prepare server
add `auto_redirect_if_standalone_oidc` in `OXICLOUD_AUTH_POLICIES`
let admin decide to redirect immediately to IdP if OIDC is the only auth method enabled
permit an admin to specify `oidc` only as the only method to login/register
note that if OIDC is enabled, the engine always append oidc in OXICLOUD_AUTH_METHODS
we could move to an explicit declaration in a major release
Prevent services accessing directly to localstorage and prefer using an astraction layer
to expose full blob. The abstraction layer (dedup services) will cover backend storage
election (local, s3, ...), encryption, etc
This change permit audio_metadata_service, media_metadaa_service, face_indexing_service to handle
blobs without worring of the backend.
note: prefered way to handle blob is the streamed way. Some services may not have this possibility
add:
- OXICLOUD_REGISTRATION_ALLOWED_EMAIL_DOMAINS to specify list of domains allowing a self registration
- OXICLOUD_REQUIRE_VERIFIED_EMAIL=true|false
- OXICLOUD_AUTH_METHODS=password,magic_link (login methods, OIDC is on top of this)
- OXICLOUD_AUTH_POLICIES=permit_magic_link_for_password_users (OIDC is on top)
add env variable `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX`
which is by default:
`OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX="@drive"`
so `/webdav/` -> points to user's personal drive (**backward compatibilit**y)
`/web/dav/@drive/{uuid|drive name}/` points to the respective drive
if admins want directly `/webdav/` pointing to list of drives they need to:
`OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX=""`
+ ensure lock is per user (RFC 4918 §9.11)
fix: #554
Every upload surface previously wrote each byte to disk twice: the HTTP
body was spooled to a temp file (or assembled from chunk parts), then
mmap-re-read for FastCDC analysis, and finally the new chunks were
written to the blob backend. CDC could not start until the last byte
arrived, so large uploads paid receive + reread + rewrite latency.
The dedup engine now chunks, hashes and settles the stream WHILE it
arrives (fastcdc AsyncStreamCDC + incremental BLAKE3):
- Each batch of distinct chunks is pinned-or-classified by ONE
`UPDATE … RETURNING` (no check-then-bump TOCTOU; pinned chunks can't
be reclaimed mid-upload), and only chunks the store doesn't have are
written — a full dedup hit performs zero content writes.
- Durability before visibility is preserved: one batched fsync sweep,
then one batched INSERT, then the manifest. Identical concurrent
uploads are resolved at the manifest INSERT via ON CONFLICT (the
loser releases its references and becomes a dedup hit).
- A drop guard rolls back pins and surfaces written-but-unregistered
chunks to GC if the request future is cancelled mid-stream.
- MIME sniffing now peeks the first bytes in-flight; client-requested
MD5/SHA-256 checksums are computed by a stream tee — the post-upload
re-read of the assembled file is gone.
All surfaces converge on the new interfaces::upload_ingest helper:
REST multipart, WebDAV PUT, NextCloud PUT, WOPI PutFile, the dedup
endpoint, and both chunked-upload completions (which now stream their
ordered parts straight into the store instead of writing an assembled
file — chunk parts persist until finalize, so completion is genuinely
retryable). The legacy blob re-chunk migration streams from the
backend with no spool file either.
Legacy removed: store_from_file + mmap CDC analysers + temp-path
plumbing through every port (pre_computed_hash, save_file_from_temp,
update_file_content_from_temp), upload_spool + assembled-file
assembly in both chunked services, create_file/update_file byte-slice
variants (no callers), common::temp, the OXICLOUD_UPLOAD_TMPDIR
config, and the memmap2 dependency.
Verified end-to-end against PostgreSQL 16: 8 MB upload (26 chunks),
identical re-upload (dedup hit, zero writes), 3-byte edit re-upload
(26 chunks, 1 written), byte-identical downloads, Range across chunk
boundaries, concurrent identical-upload race (manifest ref 2), and
trash-empty reclaiming exactly the unshared chunk while the shared 25
survive for the edited file. The empty/sub-8KB multipart path found a
post-EOF re-poll panic in the MIME peek (fixed with fuse + regression
test).
https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
ensure files does not exeed OXICLOUD_MAX_UPLOAD_SIZE, prefer to deny from header rather consuming bandwidth
add OXICLOUD_DIRECT_PUT_MAX_BYTES for direct PUT (non chunked), admins can fine tune their prefered values
explain OXICLOUD_CHUNK_DIR and OXICLOUD_UPLOAD_TMPDIR
and also the OXICLOUD_CHUNK_MAX_BYTES & OXICLOUD_UPLOAD_TMPDIR
to help administratorrs to defined correctly their storage architecture
- add coalesced protection to avoid mail bombing if an invited goes many grant in a short period
- add resentd method in share menu item (work for both internal and external users)
- user can disable email notification via his properties
- add env variable from admin to disable notifications
- remove the hardcoded list of locales in favor of a discovry on start time
- server will stop on badly formatted locale .json
- add server.* entries for serer side translation
server side translation will be used for templating and email
note: no json in some embded html (like in /magic), amount of work was similar
- SMTP has a mock to enable end to end test and validate the whole path
(via OXICLOUD_SMTP_MOCK)
- add email normalisation ( including punicode)
- api to share to external user
initially server was accepting multiple instances on same port (Linux and MacOS only)
this can create issues during development (if a dev forget another running instance...)
Add OXICLOUD_REUSE_PORT variable to activate it, so only admins knowing this feature can activate itt
(permits multiple instance + let the OS schedduler to decide which process will handle a request)
if not enabled, other instance will receive exit with a:
`Error: Os { code: 48, kind: AddrInUse, message: "Address already in use" }`
Edouard Vanbelle