The SvelteKit app under /frontend has fully superseded the legacy
vanilla-JS/CSS frontend in /static, which was only ever served by a
debug `cargo run` / `PROFILE=dev` and never shipped to production.
Remove it together with the whole subsystem that existed only to
support it (~54k lines).
Frontend & assets:
- Delete /static (js/, css/, *.html, sw.js, basemaps/, locales symlink).
- Relocate the brand/PWA assets (logo/, favicon.ico, manifest.webmanifest)
to frontend/static/ so they ship with the SPA. This also fixes the
favicon, which app.html referenced but was missing from the prod bundle.
- Migrate the Nextcloud login-flow redirects from /nextcloud-error.html
to the SvelteKit /nextcloud/error route.
Web layer:
- Simplify resolve_static_path: drop the PROFILE=dev branch; always prefer
the Vite static-dist/ build, fall back to the configured path.
- Resolve i18n locales from the served SPA dir with a frontend/static
fallback so `just dev` works without a prior build.
Build:
- Prune build.rs from 1262 to ~70 lines (git metadata only); the Rust asset
pipeline and the OXICLOUD_RUST_ASSETS rollback flag are gone.
- Drop the now-unused build-dependencies (oxc_*, lightningcss).
- Remove the COPY static lines from the Dockerfile (cacher + builder).
Tooling & docs:
- Delete biome.json, jsconfig.json, tools/check-*.py, identifier.sh.
- Remove the legacy front-* justfile recipes; repoint the design-system
scripts (locales, dead-tokens, brand-drift, token-docs) at the frontend,
and drop check-contrast/check-headings (coupled to the old token
taxonomy / multi-page HTML).
- Repoint docs/DESIGN-SYSTEM.md links; remove 5 superseded docs/plan/*.
Backend dead code:
- Remove the dead `folder_repo` field from FileBlobWriteRepository.
- Remove the deprecated GET /api/folders/{id}/listing endpoint
(superseded by /resources).
Verified: cargo clippy (all-features/all-targets) clean, cargo test
--workspace 448 passed, cargo fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements the last Phase 2 piece: a working face detector/embedder behind
the new `faces-onnx` cargo feature (mirrors how `plugins` gates wasmtime).
Inert by default — the default build is unchanged and ships the no-op
analyzer.
Pipeline (InsightFace/immich pattern): SCRFD detection with 5-point
landmarks → least-squares similarity alignment to the canonical 112×112
template → ArcFace embedding → L2-normalized 512-d vector.
- face_geometry.rs (always compiled, unit-tested): SCRFD anchor/distance
decode, NMS, the closed-form (complex-number) similarity transform,
bilinear affine warp, NCHW normalization, L2-norm, Laplacian sharpness.
11 unit tests cover the error-prone math with no model needed.
- onnx_face_analyzer.rs (feature `faces-onnx`): wires the geometry to ONNX
Runtime via `ort` (load-dynamic, so libonnxruntime is dlopen'd at runtime
and the crate builds without it). Inference runs on spawn_blocking; each
session is serialized behind a Mutex. Loads via `ort::init_from` (fallible)
not ORT's lazy loader, which would panic under `panic = "abort"`.
- config: FacesConfig + OXICLOUD_FACES_{ORT_DYLIB,DETECTOR_MODEL,
EMBEDDER_MODEL,DET_SIZE,DET_THRESHOLD,NMS_THRESHOLD,INTRA_THREADS}.
- di: build_face_analyzer() loads the real analyzer when the feature is
compiled in and runtime+models are configured; any missing piece or load
failure degrades to the no-op analyzer (logged) so startup never fails.
- ort/ndarray added as optional deps; example.env documents the setup.
Models and the ONNX Runtime dylib are operator-provided at runtime and are
never committed. Cannot be exercised in CI (no models/dylib); the geometry
is unit-tested and the ONNX seam is isolated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JW6ghFMDtnRYuYNzZhb47M
Backend — Photos timeline now groups by real capture date instead of upload time. New MediaMetadataService (FileLifecycleHook) extracts EXIF DateTimeOriginal from images and container creation_time from videos (mov/mp4/mkv) via nom-exif, timezone-correct (OffsetTimeOriginal), persisting captured_at so the existing media_sort_date trigger takes over. Adds POST /admin/photos/metadata/reextract to backfill existing media. Falls back to upload date when no embedded date exists.
Frontend — premium grid cards: combined metadata line (relative date · size, owner avatar when shared), custom selection checkbox with a clear checked state, uniform full-width 4:3 thumbnail tiles independent of filename length, centered file-type icons, and a hit-test fix so checkbox/star/kebab clicks reach the controls (the decorative thumbnail no longer captures pointer events). Notification messages internationalised across all 16 locales. Broader polish: design tokens, a11y/focus-visible states, brand + PWA assets.
Chore — bump semver-compatible dependencies (cargo upgrade); add nom-exif 3.6.1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both sides added a parameter to create_application_services and a
setup step before it: this branch's storage-usage/quota service (for
the instant-upload path) and main's Tantivy content index (for
SearchService). The resolution keeps both — the signature takes both
arguments and the build runs storage usage as step 3c and the content
index as 3d.
https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
/api/search now finds files by CONTENT as well as by name: BM25-ranked
matches over extracted text (PDF, Office OOXML/ODF, plain text/code)
with typo-tolerant fuzzy terms and search-as-you-type prefix matching,
served from an embedded Tantivy index at {storage}/.search-index.
Pipeline (all off the request path, mirroring tree-etag + thumbnails):
- statement triggers on storage.files append to a durable dirty queue
(storage.search_index_dirty) - every write surface (REST, WebDAV,
NextCloud, WOPI, trash) is covered, crash-safe by construction
- ContentIndexWorker drains the queue on the maintenance pool, extracts
text once per unique BLAKE3 blob (storage.blob_extracted_text cache:
N copies = 1 extraction, renames/moves = 0 re-extraction) and applies
batched single-writer Tantivy commits; queue rows are deleted only
after the commit succeeds (at-least-once, idempotent upserts)
- the index is a derived artifact: a version-marker mismatch wipes and
reseeds it from Postgres, which remains the single source of truth
SearchService merges content hits into the existing name search: hits
are hydrated through ONE SQL round-trip that re-applies user scope,
trash state and every active filter (a stale index id can never leak),
scored below name matches, and returned with a plain-text snippet and
a match_source field. Index failure or
OXICLOUD_ENABLE_CONTENT_SEARCH=false degrades to name-only search; a
discard-only janitor keeps the trigger-fed queue bounded while disabled.
The frontend renders the snippet under the file name in list view.
New dependencies: tantivy 0.26, zip 8.6 (deflate only), pdf-extract 0.10.
https://claude.ai/code/session_01Sc7F4xbo83YbFAQ4xEeDrX
Every upload surface previously wrote each byte to disk twice: the HTTP
body was spooled to a temp file (or assembled from chunk parts), then
mmap-re-read for FastCDC analysis, and finally the new chunks were
written to the blob backend. CDC could not start until the last byte
arrived, so large uploads paid receive + reread + rewrite latency.
The dedup engine now chunks, hashes and settles the stream WHILE it
arrives (fastcdc AsyncStreamCDC + incremental BLAKE3):
- Each batch of distinct chunks is pinned-or-classified by ONE
`UPDATE … RETURNING` (no check-then-bump TOCTOU; pinned chunks can't
be reclaimed mid-upload), and only chunks the store doesn't have are
written — a full dedup hit performs zero content writes.
- Durability before visibility is preserved: one batched fsync sweep,
then one batched INSERT, then the manifest. Identical concurrent
uploads are resolved at the manifest INSERT via ON CONFLICT (the
loser releases its references and becomes a dedup hit).
- A drop guard rolls back pins and surfaces written-but-unregistered
chunks to GC if the request future is cancelled mid-stream.
- MIME sniffing now peeks the first bytes in-flight; client-requested
MD5/SHA-256 checksums are computed by a stream tee — the post-upload
re-read of the assembled file is gone.
All surfaces converge on the new interfaces::upload_ingest helper:
REST multipart, WebDAV PUT, NextCloud PUT, WOPI PutFile, the dedup
endpoint, and both chunked-upload completions (which now stream their
ordered parts straight into the store instead of writing an assembled
file — chunk parts persist until finalize, so completion is genuinely
retryable). The legacy blob re-chunk migration streams from the
backend with no spool file either.
Legacy removed: store_from_file + mmap CDC analysers + temp-path
plumbing through every port (pre_computed_hash, save_file_from_temp,
update_file_content_from_temp), upload_spool + assembled-file
assembly in both chunked services, create_file/update_file byte-slice
variants (no callers), common::temp, the OXICLOUD_UPLOAD_TMPDIR
config, and the memmap2 dependency.
Verified end-to-end against PostgreSQL 16: 8 MB upload (26 chunks),
identical re-upload (dedup hit, zero writes), 3-byte edit re-upload
(26 chunks, 1 written), byte-identical downloads, Range across chunk
boundaries, concurrent identical-upload race (manifest ref 2), and
trash-empty reclaiming exactly the unshared chunk while the shared 25
survive for the edited file. The empty/sub-8KB multipart path found a
post-EOF re-poll panic in the MIME peek (fixed with fuse + regression
test).
https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
TL;DR:
fix duplicate filename via:
```
docker exec <container> migrate-nfc-filenames --dry-run # preview
docker exec <container> migrate-nfc-filenames # execute
```
== issue ==
Last week I uploaded Capture d'écran 2026-06-03 à 20.04.24.png from the web. It synced down to Nextcloud on my Mac. Two minutes later, the Web UI was showing the file twice.
Both rows had:
- the same name
- the same size
- the same content hash
So why two rows? Because to PostgreSQL, the names weren't the same.
Web upload (browser → Postgres):
"é" stored as 1 codepoint (U+00E9) bytes: c3 a9 ← NFC
NiextCloud client (macOS → Postgres):
"é" stored as 2 codepoints (e + U+0301) bytes: 65 cc 81 ← NFD
macOS's APFS keeps filenames in NFD (decomposed); browsers send NFC (composed). Visually é and é are identical. To WHERE name = $1 they're two different keys. Our UNIQUE index on (folder_id, name, user_id) never fired — and the row count quietly drifted every time a Mac user touched an accented
filename.
== The fix is two halves ==
1. No new duplicates — every name-receiving boundary (file upload, NC PUT, rename, MOVE, path lookup) now NFC-normalizes before touching the database. The storage invariant becomes "every stored name is NFC".
2. Clean up existing data — one-shot migrate-nfc-filenames binary walks storage.files, NFC-normalizes any non-NFC row, and resolves the collisions we've accumulated. Same-content duplicates go to trash (recoverable); different-content collisions get renamed with a .duplicate suffix.
== use of the clean up ==
example of use (do not forget to define env **DATABASE_URL**)
either
`cargo run --bin migrate-nfc-filenames -- --dry-run`
or
`cargo build --bin migrate-nfc-filenames`
`./target/debug/migrate-nfc-filenames --dry-run`
example:
```
% ./target/debug/migrate-nfc-filenames --dry-run
=== NFC filename migration (DRY RUN — no writes) ===
Loaded 543 non-trashed file rows
NORMALIZE 163451b5-5e6c-404b-9b1e-f4b01a2b7269 user=42433185-4717-416d-9a15-4580fff171ec 'Capture d’écran 2026-03-20 à 14.44.50.png' → 'Capture d’écran 2026-03-20 à 14.44.50.png'
NORMALIZE 827dddec-4dd5-48c2-a120-dec5289f7d29 user=969deca6-7935-4f12-a430-4d636b62fa3e 'Capture d’écran 2026-04-03 à 15.43.38.png' → 'Capture d’écran 2026-04-03 à 15.43.38.png'
NORMALIZE 09559934-a620-472d-9ba8-fc3cfeb6dc6f user=a0643a21-0092-4a84-9dde-7ac4e76bc1a5 'Capture d’écran 2026-06-03 à 20.05.38.png' → 'Capture d’écran 2026-06-03 à 20.05.38.png'
NORMALIZE 5ce6dbf9-0562-4758-8783-671aa9069590 user=a0643a21-0092-4a84-9dde-7ac4e76bc1a5 'Capture d’écran 2026-06-05 à 11.07.25.png' → 'Capture d’écran 2026-06-05 à 11.07.25.png'
DEDUP newer=26bcf82b-99cc-45c8-9d69-dd7e5c4484ff (trash, same blob) older=df3adc67-a778-424d-a817-b930c75f3b06 user=a0643a21-0092-4a84-9dde-7ac4e76bc1a5 hash=0d2cc7b0ffce2850
=== Summary ===
scanned : 543
already in NFC : 538
normalized in place (no collision) : 4
dedup-trashed (same content) : 1
renamed to .duplicate : 0
DRY RUN — no rows were written. Re-run without --dry-run to apply.
```
once valid remove --dry-run
- remove the hardcoded list of locales in favor of a discovry on start time
- server will stop on badly formatted locale .json
- add server.* entries for serer side translation
server side translation will be used for templating and email
note: no json in some embded html (like in /magic), amount of work was similar
- SMTP has a mock to enable end to end test and validate the whole path
(via OXICLOUD_SMTP_MOCK)
- add email normalisation ( including punicode)
- api to share to external user
Azure SDK crates (azure_core, azure_storage, azure_storage_blobs) were
enabling reqwest/default-tls → native-tls → openssl-sys via their
default 'enable_reqwest' feature. This breaks the Alpine musl Docker
build which lacks libssl.a/libcrypto.a.
- Set default-features = false on all 3 Azure crates
- Enable 'enable_reqwest_rustls' + 'hmac_rust' (pure Rust, no OpenSSL)
- Pin reqwest to ^0.12 (azure_core 0.21 requires reqwest ^0.12)
- openssl-sys is now fully eliminated from the dependency tree
- Dockerfile: add dummy src/bin/generate-openapi.rs in cacher stage to satisfy
Cargo.toml [[bin]] target resolution during dependency caching
- docker-publish.yml: add explicit ref to checkout steps so workflow_dispatch
builds the correct tag instead of HEAD of main
- docker-publish.yml: increase timeout to 180min for multi-arch QEMU builds
- Bump version to 0.5.4
Closes#250
- Add utoipa v5 dependency with ToSchema derives on all REST API DTOs
- Annotate free-function handlers with #[utoipa::path] (trash, share, favorites, recent)
- Create ApiDoc struct with OpenApi derive registering 37 schemas across 7 tags
- Add generate-openapi binary outputting resources/gen/openapi.json
- Serve OpenAPI spec at GET /api/openapi.json (public, no auth)
- Add justfile with common dev commands (build, test, lint, check, openapi, db)
Replace manual schema.sql application with sqlx's built-in migration
system. Migrations are embedded at compile time and tracked in the
_sqlx_migrations table. Pending migrations run automatically on startup.
- Move db/schema.sql → migrations/20260307000000_initial_schema.sql
- Remove apply_schema() and split_sql_statements() from db.rs
- Add run_migrations() using sqlx::migrate!() macro
- Remove docker-compose schema.sql mount (app handles it now)
- Enable sqlx "migrate" feature in Cargo.toml
Future schema changes: add a new timestamped .sql in migrations/.
Closes#190
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace std::fs::read() + update_rayon() with update_mmap_rayon()
for file hashing, eliminating full-file heap allocation (500MB file
no longer needs 500MB of RAM to hash)
- Enable blake3 'mmap' feature in Cargo.toml
- Lower hash_bytes rayon threshold from 10MB to 128KB
- Remove dead constants HASH_BLOCK_SIZE and RAYON_HASH_THRESHOLD
Extract EXIF orientation, GPS coordinates, camera info, and timestamps
from uploaded images using kamadak-exif. Store metadata in a new
file_metadata PG table. Apply EXIF orientation to thumbnail generation
so images display correctly. Add /api/files/{id}/metadata endpoint.
- JWT secret auto-generates and persists to <STORAGE_PATH>/.jwt_secret
- Remove setup token: first admin setup is open until system initialized
- Fix schema.sql: move CREATE EXTENSION pg_trgm/ltree to top
- Update login UI and auth.js to remove setup token fields
- Allow async_fn_in_trait lint crate-wide (internal project, 413 warnings)
- Add integration_tests feature to Cargo.toml to fix unexpected cfg warnings
- Collapse nested if statements into single conditions (13 locations)
- Add type_complexity allows on pg repository functions (12 locations)
- Fix dead code warnings in test modules with allow attributes
- Fix E0599 by gating new_stub() for integration_tests feature
- Add result_unit_err and result_large_err allows where appropriate
- Apply rustfmt formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement a complete Nextcloud client compatibility layer so that
Nextcloud desktop/mobile sync clients can connect to OxiCloud.
Key additions:
- Login Flow v2 (device auth) with OIDC bridge support
- WebDAV handler compatible with Nextcloud clients (PROPFIND, GET,
PUT, DELETE, MKCOL, MOVE, COPY, HEAD, PROPPATCH)
- OCS API endpoints (user info, capabilities, notifications stubs,
sharees, unified search)
- Basic Auth middleware with app password verification, account
lockout integration, and blake3-keyed auth cache
- App password management: create, list, revoke via both native
API (JWT-authenticated profile page) and Nextcloud OCS endpoints
- Nextcloud file ID mapping (oc:fileid) with persistent DB storage
- Chunked upload support (Nextcloud v2 chunking protocol)
- Trashbin WebDAV interface
- Avatar (SVG placeholder) and preview (redirect) handlers
- User profile page with app password management UI
- URL user validation on all DAV routes (403 on mismatch)
- Database schema for app_passwords and nextcloud_object_ids tables
All services are behind a `nextcloud.enabled` config flag and
cleanly separated under src/interfaces/nextcloud/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Enable HTTP/2 auto-detection in axum (add 'http2' feature)
- Remove direct hyper dependency with 'full' features (already transitive via axum/reqwest)
- Replace std::sync::RwLock with tokio::sync::RwLock in oidc_service.rs and file_system_i18n_service.rs to prevent async deadlocks
- Add type aliases (FileRow, FolderRow, FolderRowPaginated, FolderRowOptUser) to reduce type complexity
- Simplify redundant closures in app_password_handler and webdav_handler
- Remove needless borrow in auth_handler
- Collapse nested if/let chains in login_lockout, webdav_lock, auth, rate_limit
- Box LockEntry in acquire() Err variant to fix large enum variant warning
- Rename DeviceCodeStatus::from_str to parse to avoid should_implement_trait lint
- Add #[allow(clippy::too_many_arguments)] and #[allow(clippy::result_unit_err)] where appropriate
- Convert integration_tests from cargo feature to custom cfg attribute
- Add check-cfg lint config in Cargo.toml for integration_tests cfg
The standalone `md5` (0.8.0) crate is replaced with `md-5` (0.10) from
the RustCrypto ecosystem, which shares `digest v0.10` with sha2, argon2,
blake2 and other crates already in the dependency tree — eliminating one
redundant implementation.
https://claude.ai/code/session_01V23pGpfNw5ujZvtwRFG6qy
Remove async-trait dependency and use native Rust async fn in traits.
Replace Arc<dyn Trait> with Arc<ConcreteType> throughout the codebase
to enable monomorphization and eliminate dynamic dispatch overhead.
Key changes:
- Remove write-behind cache (no implementation existed)
- Fix should_transcode static method call
- Use ContactStorageAdapter directly instead of dyn AddressBookUseCase
- Clean up unused trait imports across services and DI
https://claude.ai/code/session_01EbAFEfyJNLRmJHmmYDX3Tt
Replace push_str(&format!(...)) pattern in generate_full_calendar_ical,
generate_event_ical, and generate_vevent with direct write!() into a
pre-sized String buffer.
Before: ~5N+1 heap allocations for N events (temporary Strings created
by format!(), copied into the main buffer, then dropped).
After: 1 allocation (the initial String::with_capacity). All write!()
calls format directly into the destination buffer with zero intermediate
Strings.
All content from 003_add_device_codes.sql and 004_add_trigram_indexes.sql
was already absorbed into db/schema.sql (the single source of truth).
The migrations had additional problems:
- Broken numbering (started at 003, missing 001/002)
- 004 used CREATE INDEX CONCURRENTLY which fails inside sqlx transactions
- No production flow ever invoked the migrate binary
Removed: db/migrations/, src/bin/migrate.rs, migrations Cargo feature,
[[bin]] migrate target, and doc/database-migrations.md.
- WebDAV PUT and WOPI PutFile: replace sha2::Sha256 with blake3::Hasher (~5x faster hashing, compatible with dedup service)
- Fix TEXT↔UUID JOIN anti-pattern in favorites and recent_items repos (enables PK index usage)
- Add LIMIT 500 to get_favorites query to prevent unbounded memory allocation
- Remove unused lru crate from Cargo.toml (superseded by moka)
- Replace tokio features=["full"] with explicit feature list (removes signal, process, test-util)
- Replace basic TcpListener::bind with socket2 tuned socket
- TCP_NODELAY: disable Nagle's algorithm (-5 to 40ms latency on small responses)
- SO_REUSEADDR: port available immediately after server restart
- SO_REUSEPORT: ready for multi-worker scaling (Linux)
- TCP_KEEPALIVE: detect dead connections within 60s/10s interval
- listen(2048): high backlog for WebDAV connection bursts
- Eliminate redundant create_dir_all calls from upload hot path
DioCrafts