Every GET of a JPEG ≤5 MB with `Accept: image/webp` ran a full decode +
lossless-WebP encode on the rayon pool (hundreds of ms of CPU), found
the result LARGER than the original — the `image` crate's WebP encoder
is lossless-only, which practically never beats an already-lossy photo —
and threw the work away. Nothing recorded the verdict, so the next GET
of the same file repeated it, forever. A 20-photo gallery saturated the
transcode pool on every visit.
- `can_transcode` no longer accepts `image/jpeg` — photos skip the
pipeline entirely. PNG/GIF stay (lossless WebP genuinely shrinks them).
- Negative verdicts for the remaining formats are now remembered: an
empty-Bytes sentinel in the moka cache (expires with the 10-min TTL)
plus a zero-byte `.skip` marker on disk that survives restarts.
`invalidate()` removes the marker alongside the cached transcode.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
Two DAV-surface fixes that replicate patterns the codebase already had:
NC PROPFIND (folder case) previously loaded EVERY child via unbounded
list_files/list_folders and serialized the entire multistatus into one
Vec (~2 KB per entry — a 50k-file folder meant ~100 MB of buffer per
request, repeated constantly by sync clients). It now mirrors the native
WebDAV handler's streaming builder: children are fetched in pages of
PROPFIND_BATCH_SIZE (500), each page's favorites and oc:fileids are
resolved with two batch queries, and the XML is yielded chunk by chunk —
memory stays O(batch) and the first byte flows immediately. The
single-file PROPFIND keeps a small buffered variant; the multistatus
opening tag is factored into a shared helper so the namespace set cannot
diverge.
WebDAV GET (native and NC) ignored the Range header and never compared
the ETag it emitted, so mount-style clients (rclone, davfs2, Finder)
re-transferred whole files on every seek, resume, or revalidation. New
shared `interfaces::range_requests` helpers — same semantics as the REST
download endpoint, which now reuses the 304 helper too — give both GETs
If-None-Match → 304, Range → 206/416, and Accept-Ranges advertising.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
Every WebDAV / CalDAV / CardDAV request paid one full-row user fetch in
require_internal_user_layer just to read `is_external` (and the NC Basic
Auth middleware repeated it right after its own cache hit). That SELECT
includes the `image` column — a data URI of up to 512 KiB — so a sync
client issuing hundreds of PROPFINDs per minute dragged hundreds of MB
of avatar bytes out of Postgres to evaluate a boolean.
- New `UserFlags { role, is_external, active }` + a repo query selecting
only those three columns (inherent method, mirroring `update_image`).
- `AuthApplicationService::get_user_flags`: moka cache, 30 s TTL,
10k capacity. `change_user_role` / `set_user_active` invalidate
eagerly, so admin changes still apply immediately; anything else is
visible within the TTL — preserving the documented "no token rotation
needed" semantics at a per-request cost of zero DB round-trips when
warm.
- `require_internal_user`, `require_admin_user` and the NC Basic Auth
external check now go through the flags lookup.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
Four related fixes to stop buffering files in page memory and stop
hammering layout from the selection paths:
- Inline viewer media: video/audio fetched the ENTIRE file into a blob
before the first frame (a 2 GB video = 2 GB of tab heap, no
progressive playback, no seek). The <video>/<audio> src now points
straight at the same-origin API URL — cookies travel automatically
and the browser streams with native Range requests, exactly like the
music player already did.
- Downloads (file, folder ZIP, batch ZIP, viewer button): the
fetch → blob → objectURL pattern materialized the whole payload in
RAM before the save dialog appeared (a 10 GB batch ZIP risked
crashing the tab) with no download progress UI. New shared
utils/download.js hands the URL to the browser, which streams to
disk with its own progress UI. Batch download uses the existing GET
endpoint (same URL contract as the drag-out DownloadURL, now shared
via buildBatchDownloadUrl); selections whose id list cannot fit in a
URL keep the buffered POST fallback. Trade-off: failed downloads now
surface in the browser's download shelf instead of an in-app toast.
- Rubber-band lasso: every mousemove (>100/s) walked all cards
interleaving getBoundingClientRect() reads with class writes — up to
N forced reflows per event, freezing the frame rate on folders with
thousands of loaded rows. Card rects (+ item info) are now snapshot
once per drag (rebuilt on scroll), and a single rAF pass per frame
compares against the cached geometry, touching only cards whose
selection state changed.
- batchToolbar: getSelection() rebuilt the selection by scanning every
.file-item in the document (the TODO admitted it); it now reads the
_selected Map that every selection path already keeps in sync.
clear() scopes its DOM sweep to #files-list (the only container the
toolbar manages) instead of the whole document.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
The global childList observer scheduled replaceIconsInElement() with no
scope on ANY node insertion — including bare text nodes — so every
notification-bell progress tick during an upload and every
infinite-scroll batch re-scanned the whole document with an attribute
substring selector. Cost grew with total DOM size (5-10k nodes), not
with what was inserted.
Queue the added element roots per animation frame and scan just those
subtrees (an inserted <i> itself is caught via its parent). Text-node
churn no longer triggers any scan at all. The icon replacement is
idempotent, so overlapping roots are harmless.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
_bindItemEvents attached direct click listeners to the favorite star,
the shared badge and every custom action button of every row — ~3+
listeners per row on top of the container delegation the component
already wires, i.e. ~10-15k listeners with a few thousand rows loaded
via load-more, plus the per-row binding cost on every render.
The container's delegated click handler already dispatches by
closest() checks; move the three button behaviours there (they return
before the card-open branch, preserving the old stopPropagation
semantics) and drop _bindItemEvents entirely. Row creation is now
pure innerHTML — no listener allocation per row.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
displaySearchResults inserted up to 100 results one at a time via
addItem({ scroll: true, highlight: true }) — each insert ran two DOM
scans (duplicate guard + row lookup), a smooth scrollIntoView and a
highlight pulse, so every live-search keystroke triggered ~100 competing
smooth-scrolls and O(n²) container scans.
Add filesView.renderItems(): a single component.render() pass (one
DocumentFragment, zero per-item scans/scrolls). addItem keeps the
scroll/highlight affordances for its documented purpose — optimistic
single-item inserts after upload/create.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
`uploadFiles` sent files one at a time (await per file), so dropping N
files meant N sequential round-trips. Reuse the bounded-concurrency pool
already proven in `uploadFolderEntries` (CONCURRENCY = 10): independent
files now upload up to 10 at a time — ~10x faster for many small files.
Per-file progress (XHR → bell), the legacy progress bar, timeout
notifications and the quota short-circuit are preserved. On a quota error
the pool stops pulling new files while in-flight uploads finish (matching
`uploadFolderEntries`) instead of the old hard `break`.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
The entrypoint ran "chown -R" on the storage volume on every container
start. Storage is a content-addressable blob store that can hold millions
of objects, so a blind recursive chown re-stats and rewrites the inode of
every blob on each boot, adding minutes of startup time and saturating the
disk on spinning media.
Guard the chown behind a top-level ownership check: only recurse when the
directory root is not already owned by the oxicloud user. The first boot
fixes a freshly mounted (root-owned) volume; every later boot is a no-op.
The same guard is applied to the static dir, factored into a shared helper.
The target UID is resolved via "id -u oxicloud" instead of hardcoding 1001.
https://claude.ai/code/session_01GpprjxjtXFYLfXNkoKnHuL
Deleting a multi-photo selection fired one `DELETE /api/files/{id}` per
photo in series — 300 selected photos meant 300 sequential round-trips
with the UI blocked on `confirm()`. Switch to `POST /api/batch/trash`
(already used by the file view), chunked to the backend's MAX_BATCH_SIZE
of 1000, so a selection of any size collapses to ceil(N/1000) requests.
Only items the server reports as successfully trashed are removed from
the grid; the selection bar refreshes to reflect any failures.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
Resolving the stable numeric oc:fileid for every child in a NextCloud
listing issued one `INSERT ... ON CONFLICT DO UPDATE` per entry — a write
(row rewrite + WAL + dead tuple) even when the mapping already existed.
A Depth:1 PROPFIND of a folder with N children meant N sequential write
round-trips on a read-only operation that sync clients repeat constantly.
- Repository: replace the single `get_or_create` (DO UPDATE) with
`get_or_create_many` — one idempotent bulk `INSERT ... SELECT unnest(...)
ON CONFLICT DO NOTHING` (existing rows untouched) plus a single
`SELECT ... WHERE object_id = ANY(...)`. Two statements instead of N.
- Service: add an Arc-backed moka cache (uuid -> i64; the mapping is
immutable, so warm entries never go stale) and batch APIs
`get_or_create_file_ids` / `get_or_create_folder_ids` that only query
the misses. Warm listings cost zero queries.
- Handlers (PROPFIND, REPORT favorites/search, trashbin, OCS unified
search): pre-resolve all ids in two batched queries — file and folder
run concurrently via `tokio::join!` — and turn the XML/JSON emission
into a synchronous map lookup.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
The .dockerignore excluded a non-existent doc/ directory (the real one is
docs/) and did not exclude tests/, images/, charts/, or tools/. None of
these are COPYed by the Dockerfile, yet ~24 MB of them were sent to the
daemon as build context on every build. Fix the typo and exclude the
unused asset/tooling directories so the context stays minimal.
https://claude.ai/code/session_01GpprjxjtXFYLfXNkoKnHuL
sqlx's postgres driver implements the wire protocol in pure Rust and TLS
goes through rustls, so libpq is never linked. Confirmed via Cargo.lock:
no pq-sys, native-tls, or openssl-sys in the dependency tree.
Removing postgresql-dev from the build base and libpq from the runtime
image shrinks the final image by ~3 MB and removes a C library from the
attack/patch surface. perl/make/gcc/musl-dev are kept for the C builds of
aws-lc-sys.
https://claude.ai/code/session_01GpprjxjtXFYLfXNkoKnHuL
Closes#323.
LoginLockoutService cached failed-attempt counters keyed only on
the username, so any caller that could reach the auth endpoint and
guess (or enumerate) a username could lock that account out for the
entire lockout window — the rate limiter happily lets each IP make
its share of bad-password attempts before clamping, which is enough
to trip the per-account threshold in seconds. The reporter
demonstrated a complete DOS by spoofing X-Forwarded-For with
OXICLOUD_TRUST_PROXY_HEADERS=true.
Fix: change the lockout cache key from `username` to `username|ip`.
A flood from one IP locks that IP out of that account, but a
legitimate user coming from a different IP is unaffected.
Changes:
- LoginLockoutService::{check, record_failure, record_success} take
client_ip as a second argument; cache key is built via Self::key
(`format!("{username}|{ip}")`).
- middleware/rate_limit.rs: factor out extract_client_ip_from_parts
(HeaderMap + Option<&SocketAddr>) so handlers that don't take a
full Request<B> can still derive the same client identifier
extract_client_ip uses. extract_client_ip now delegates to it.
- auth_handler.rs login: derive client_ip from headers (the only
signal available without ConnectInfo) and pass it through to all
three lockout calls.
- nextcloud/basic_auth_middleware.rs: do the same with the full
Request via extract_client_ip.
Tests:
- Updated existing 4 unit tests to thread an IP arg.
- New does_not_lock_out_other_ips_for_same_account: lock from IP1,
assert IP2 still allowed (the #323 regression).
- New success_resets_only_the_acting_ip: a successful login from
IP2 must NOT clear an attacker's lockout from IP1.
Verification:
- `cargo build` ✅
- `cargo test login_lockout` → 6 passed (4 existing thread an IP
arg without behaviour change, 2 new pin the per-IP scoping).
Signed-off-by: SAY-5 <say.apm35@gmail.com>
The main search endpoint took `limit: params.limit.unwrap_or(100)` with no
ceiling, and that value flows straight into the SQL LIMIT of
search_files_paginated / search_files_in_subtree. A client passing
?limit=<huge> would make Postgres return that many rows into memory and into
the result cache. The suggestions endpoint already clamps (.min(20)); search
did not.
Cap at MAX_SEARCH_LIMIT (500). total_count still reflects the full match set
(COUNT(*) OVER()), so deeper results stay reachable via offset.
https://claude.ai/code/session_01UtfkS3nZF1vrF5jNAps6wV
store_chunks bumped storage.blobs.ref_count once per chunk *occurrence* (it
looped over the full chunk list, duplicates included), but
remove_manifest_reference decrements once per *distinct* chunk
(WHERE hash = ANY(chunk_hashes) matches each row a single time). For any file
that repeats a chunk -- zero-filled regions in disk/VM images, repeated
document structures, concatenated archives -- storing added +N while deleting
removed -1, so the blob's ref_count never returned to 0 and the chunk was
never garbage-collected: a permanent storage leak.
Count per distinct chunk on the store side too, matching deletion. This also
makes it faster:
- existing chunks: one batched `UPDATE ... WHERE hash = ANY($1)` instead of
one UPDATE per occurrence;
- a brand-new chunk repeated within a file is read, uploaded and INSERTed
once instead of once per occurrence.
The manifest still stores the full per-occurrence chunk sequence (needed to
reassemble the file). Forward fix: blobs already over-counted by the old path
stay over-counted (a reconcile/verify pass could recompute them), but the
bias is upward (leak), so no data is ever deleted early.
CDC tests pass (12); fmt + clippy clean.
https://claude.ai/code/session_01UtfkS3nZF1vrF5jNAps6wV
The /api router added its own predicate-less CompressionLayer (routes.rs),
nested inside the global predicate-aware one in main.rs. As the inner layer it
compressed responses first, so the global predicate that skips already-
compressed media was bypassed for every /api response: video/audio/image/zip
downloads got Brotli-compressed (CPU + first-byte latency for ~0 bytes saved)
and lost their Content-Length (forced to chunked -> no client progress bar).
- Remove the redundant /api CompressionLayer; /api now flows through the
single global layer in main.rs.
- Make that predicate smarter: compress by default so nothing shrinkable is
missed, and skip ONLY already-compressed types. It no longer blanket-excludes
image/*, so image/svg+xml (text, ~70% shrink) now compresses; raster formats
are listed individually. Added the previously-missed already-compressed
types: Office (docx/xlsx/pptx), ODF, epub, jar, apk, 7z/rar/bzip2/zstd/xz,
woff/woff2 fonts, icons.
Net: media downloads keep Content-Length and skip pointless compression, while
text/JSON/JS/CSS/SVG/XML/ttf/otf/wasm still compress. fmt + clippy clean.
https://claude.ai/code/session_01UtfkS3nZF1vrF5jNAps6wV
StoragePath::join deep-cloned the whole Vec<String> (every segment String)
just to append one element. Take self by value and push in place. All callers
pass owned values except PathService::create_file_path, which holds a borrow
and now clones explicitly — the same copy the old &self join already made.
File::with_name / with_folder / with_size took &self and rebuilt the struct,
cloning every carried-over field (id, mime_type, folder_id, blob_hash, ...).
Consume self and mutate only the fields that change. Behaviour is identical;
the fallible builders now drop the input on Err, which is fine for these
rename/move/resize transforms (all current callers replace the file).
Impact is small in practice — with_folder/with_size have no callers and
with_name is test-only, while the one hot join caller (create_file_path)
must copy segments regardless — but the consuming form is the idiomatic one.
Verified: cargo fmt + clippy --all-features --all-targets -D warnings clean;
domain tests (path_service::, entities::file::) pass — 30 + 6.
https://claude.ai/code/session_01UtfkS3nZF1vrF5jNAps6wV
webdav encode_uri_path runs on every PROPFIND href and did
.map(...).collect::<Vec<_>>().join("/"), allocating a String per segment plus
a joined Vec. Write each utf8_percent_encode Display adapter straight into a
single preallocated String. Behavior is identical (split on '/', encode each
segment, join with '/'), including leading/trailing-slash edge cases.
subject_group list / list_with_counts each issued a second SELECT COUNT(*)
round-trip for the total. Fold it into the page query via COUNT(*) OVER() —
the pattern folder_db_repository already uses — halving the round-trips.
total_count is read from the first row and is 0 on an empty page, matching
folder_db_repository's documented convention.
https://claude.ai/code/session_01UtfkS3nZF1vrF5jNAps6wV
list_incoming_grants and list_grants_on_resource ran fetch_all with no
LIMIT, so a pathological number of grants on one resource (or targeting one
subject) would be pulled fully into memory. These back the grant-management
endpoints ("Manage sharing", "Shared with me"), not the hot require() path.
A blind LIMIT is unsafe here: apply_role reads the full grant set to compute
an add/remove diff, so a silently truncated list would be acted on as if
complete (stale grants never revoked). Instead fetch MAX_GRANT_ROWS + 1 and
reject with an audit line (authz.grant_list_rejected / reason=over_row_cap)
when the cap is exceeded, bounding worst-case RAM without ever returning a
partial set. The check is shared via PgAclEngine::guard_grant_row_cap. Cap is
10_000 — orders of magnitude above any realistic single-resource/subject
grant count.
https://claude.ai/code/session_01UtfkS3nZF1vrF5jNAps6wV
CachedBlobBackend held its single tokio::Mutex<LruCache> across filesystem
syscalls, serializing every concurrent cache operation behind one lock:
- get_blob_stream / get_blob_range_stream: held across File::open()/seek()
- delete_blob: held across remove_file()
- initialize: held across the full cache-dir walk
- eviction (insert + fetch paths): held across remove_file() loops
Now the lock only guards the in-memory LRU. Presence checks bump recency
and release the guard before touching the filesystem (a vanished file falls
through to the existing fetch-and-cache path, covering the race), and
eviction selects victims under the lock then unlinks them after releasing
it. The duplicated eviction loop is extracted into
CachedRef::collect_evictions.
db: set test_before_acquire(false). With warm min_connections and a bounded
max_lifetime, the liveness ping sqlx issues on every acquire() costs more
than the rare dead connection it catches; stale sockets surface as a query
error and the pool recycles them either way.
https://claude.ai/code/session_01UtfkS3nZF1vrF5jNAps6wV
a prefix path as been add to avoid collision if OXICLOUD_UPLOAD_TMPDIR=OXICLOUD_CHUNK_DIR
upgrade to this version will convert previous upload chunked into the prefixed version at server start
ensure files does not exeed OXICLOUD_MAX_UPLOAD_SIZE, prefer to deny from header rather consuming bandwidth
add OXICLOUD_DIRECT_PUT_MAX_BYTES for direct PUT (non chunked), admins can fine tune their prefered values
explain OXICLOUD_CHUNK_DIR and OXICLOUD_UPLOAD_TMPDIR
and also the OXICLOUD_CHUNK_MAX_BYTES & OXICLOUD_UPLOAD_TMPDIR
to help administratorrs to defined correctly their storage architecture
purpose: avoid current scheme:
1. write to disk
2. reopen file to read data and digest it
now: digest is done while writing data to disk
all other implementation than Nextcloud are corrrect
add OXICLOUD_CHUNK_MAX_BYTES which correspond to the max upload chunk allowed
(differs from OXICLOUD_MAX_UPLOAD_SIZE which is the max total size of a file)
hurl test validate the change
Streams the request body straight to the chunk file with peak heap of
~one HTTP frame, regardless of chunk size or the configured cap. The
`storage.chunk_max_bytes` config (env `OXICLOUD_CHUNK_MAX_BYTES`,
default 100 MB) bounds a single PUT — separate from `max_upload_size`
which governs whole-file uploads. Without this separation, a client
could submit a chunk up to the whole-file cap (10 GB default) and
monopolise server memory.
- add /ocs/v1.php/cloud/user route (same as /ocs/v2.php/cloud/user )
- add also /remote.php/dav/avatars/{user}/{size}.png route (used by Nextcloud sync)
note: webp are transcoded in png, no cache but using pragma HTTP header
PROPFIND on upload chunk is necessary for application to resume the upload
it is used at least by Nextcloud Android app
issue has been raised via #415
unable to test, so I am using a playbook in test/webdav to simulate a
PROPIND on remote.php/dav/uploads/{user}/{session}
This solve issue with 2 migrations made the same day, due to merge on pull request, DB migration is blocking
the same version prefix:
- 20260625000000_files_user_size_index.sql (Dio)
- 20260625000000_folder_tree_modified_at.sql (Ed)
They were renamed to ...0001 and ...0002 (disjoint versions) + protection like "IF NOT EXISTS"
I have opt for an automated clean up of old entry:
`DELETE FROM _sqlx_migrations WHERE version = 20260625000000;`
runned on startup
affected users: Dio, myself and any dev that wanted to work on this project since eb0ba58158
When a new browser visits the login page, the language selector runs first.
After the user selects a language and clicks continue, the code checked
system status and correctly showed the login panel when `initialized=true`
— but did not hide the "Set up administrator" link.
That link was only hidden by `showInitialPanel()`, which returns early
(without reaching the hide logic) whenever `isFirstRun()` is true. So on
any browser that had not previously stored the locale key, the link stayed
visible and clickable, leading users back to the admin setup panel even
after an admin already existed.
Fix: hide the link in the language-continue handler's `else` branch,
mirroring the same guard already present in `showInitialPanel()`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude