derived-blobs.md — consolidates several review rounds.
BLOCKER found while building the coverage matrix: the zero-ref manifest
sweep in dedup_gc (dedup_service.rs:2574) deletes a manifest when
`ref_count <= 0 OR NOT EXISTS (SELECT 1 FROM storage.files ...)`. That
OR hardcodes "storage.files is the only thing that can reference a
manifest", so a thumbnail manifest held only by content_derived_blobs
is deleted on the next GC run, its chunks dereferenced and the bytes
reaped. Promoted to prerequisite 0 and delivery step 2. It is also the
missing half of the unreconciled chunk_manifests.ref_count: that OR is
the hack that made the drift survivable.
Adds the 13-edge consistency coverage matrix (rows 1-6 and 13 covered,
7-11 not), and records that backend_consistency needs NO change — the
backend holds chunks, which neither new table references.
BlobReferenceSource correction: `ref_level()` was wrong because
FilesReferenceSource spans both levels (chunk for legacy manifest-less
rows, Blob for CDC rows). Replaced with
`ref_count_sql(level, alias) -> Option<String>`.
Also: migration of existing sidecar content; schema trim to the columns
nothing else owns (no size/format/codec/renderer; content_type kept as
non-key since it removes today's byte-sniffing); ext-{file_id}.jpg
corrected — the client generator ships and covers PDF, which has no
server-side rasteriser, so file_attached_blobs is required rather than
deferred; uploaded_by on the shares NOT NULL/no-FK convention; the
mermaid relation map; copy and version semantics with the
copy_file_satellites consolidation; the storage.files vs file_metadata
table-identity fix; DedupService -> BlobHandler recorded as decided.
NEW hidden-system.md — retires auth.users.image TEXT (inline base64
avatar, up to 512 KiB, already worked around with a narrow projection
after it was measured detoasting M avatars per group fan-out) in favour
of *_file_id pointers at ordinary storage.files rows in one shared
hidden system drive. Because storage.files is already a
BlobReferenceSource, a file pointer costs zero new reference sources
and zero new consistency edges. Records why the alternatives lose,
the drive's required properties (hidden at enumeration, trash off,
quota exempt, boot fail-fast), per-kind visibility in code, the
secrets exclusion rule, the avatar migration, and the future object
catalogue.
Docs only; no code or schema changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Enrich OxiCloud to maximise the use of `dedup` Engine
2 cases will be covered:
- blobs issues from other blobs (thumbnail automatic generation from blob)
- by filename (ex: thumbnail uploaded from users)
A local cache will be added when blobs are remote (S3 or similar)
link are checking that email matches, +email alias are normalize into email
if email is already used on another account, link is not possible
not usurpation risk as the IDP is choosen by the admin
this make OIDC compliant with the invariant binding (issuer and subject)
admin can now rename their provider without breaking
clarifing federation_kind: report the kind of federation wired not the allowed login method
hybryd login method are still allowed
Prevent services accessing directly to localstorage and prefer using an astraction layer
to expose full blob. The abstraction layer (dedup services) will cover backend storage
election (local, s3, ...), encryption, etc
This change permit audio_metadata_service, media_metadaa_service, face_indexing_service to handle
blobs without worring of the backend.
note: prefered way to handle blob is the streamed way. Some services may not have this possibility
Two chronic problems fall out:
1. **Split-brain config.** Admin edits DB via the panel; app boot ignores DB.
Migration completes; live backend hasn't moved. Admin has to remember to
copy env vars into `.env` and restart. Two sources of truth for the same
setting. Cutover is a manual multi-step flow; users routinely get it wrong.
2. **Migration data-loss window on concurrent writes.** The copy walks
`storage.blobs` in hash order. A blob whose hash is lex-lower than the
current cursor, written to source AFTER migration passed it, is never
copied to target. `passed=true, findings=0` completion does NOT guarantee
target has every blob. Silent.
3. **Migration target selection is fragile.** DTO passes the whole S3 config
at trigger time; secrets sit plaintext in `admin_settings`. Any future
pluggable-storage story compounds this (Azure, GCS, WebDAV-as-source, …).
This plan replaces the split-brain model with a single-source-of-truth
architecture:
- `.env` declares **N named storage entries** (immutable per-deploy).
- `admin_settings.storage.active_backend_name` holds ONE row — which named
entry the app currently runs on. That's the whole runtime config.
- Migration is the atomic transition from one active entry to another. Server
is put in read-only mode for the copy window; on completion, the active
pointer flips; a restart cuts over.
purpose is to design a job registry with a scheduler
thi aim to drive in the same way any services requiring execution of periodic background tasks
plugins could benefeciate it
purpose is mostly to add a normative way to implement consistency check per services
this is to edutcate implementors adding any new services
goal is to ensure data quality with oxicloud and resumable jobs by default
add env variable `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX`
which is by default:
`OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX="@drive"`
so `/webdav/` -> points to user's personal drive (**backward compatibilit**y)
`/web/dav/@drive/{uuid|drive name}/` points to the respective drive
if admins want directly `/webdav/` pointing to list of drives they need to:
`OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX=""`
+ ensure lock is per user (RFC 4918 §9.11)
fix: #554
what about exposing dead props into rest API
let user to store preferences, labels on resources
| Use case | What it looks like | Why dead-props help |
|---|---|---|
| Photo annotations | captions, ratings (1-5), notes per photo | already keyed by `file_id`; round-trips via WebDAV without re-implementing |
| Web-UI tags / labels | `oxi:user:tag/project=alpha`, color flags, "archived" markers | per-resource user metadata without new tables |
| Folder UI preferences | default sort, default view mode, "favourite" flag | persistent per-folder, shared across users on shared drives |
| Cross-protocol bridge | Thunderbird sets `oxi:lastsync=...` via PROPPATCH → web UI reads it via REST | one store, two surfaces — visibility goes both ways |
| Workflow / approval state | `reviewed_by=alice`, `due=2026-09-15` | ad-hoc state per resource without schema sprawl |
| Third-party integrations | external apps store scratch space per resource | lower barrier than implementing WebDAV |
The SvelteKit app under /frontend has fully superseded the legacy
vanilla-JS/CSS frontend in /static, which was only ever served by a
debug `cargo run` / `PROFILE=dev` and never shipped to production.
Remove it together with the whole subsystem that existed only to
support it (~54k lines).
Frontend & assets:
- Delete /static (js/, css/, *.html, sw.js, basemaps/, locales symlink).
- Relocate the brand/PWA assets (logo/, favicon.ico, manifest.webmanifest)
to frontend/static/ so they ship with the SPA. This also fixes the
favicon, which app.html referenced but was missing from the prod bundle.
- Migrate the Nextcloud login-flow redirects from /nextcloud-error.html
to the SvelteKit /nextcloud/error route.
Web layer:
- Simplify resolve_static_path: drop the PROFILE=dev branch; always prefer
the Vite static-dist/ build, fall back to the configured path.
- Resolve i18n locales from the served SPA dir with a frontend/static
fallback so `just dev` works without a prior build.
Build:
- Prune build.rs from 1262 to ~70 lines (git metadata only); the Rust asset
pipeline and the OXICLOUD_RUST_ASSETS rollback flag are gone.
- Drop the now-unused build-dependencies (oxc_*, lightningcss).
- Remove the COPY static lines from the Dockerfile (cacher + builder).
Tooling & docs:
- Delete biome.json, jsconfig.json, tools/check-*.py, identifier.sh.
- Remove the legacy front-* justfile recipes; repoint the design-system
scripts (locales, dead-tokens, brand-drift, token-docs) at the frontend,
and drop check-contrast/check-headings (coupled to the old token
taxonomy / multi-page HTML).
- Repoint docs/DESIGN-SYSTEM.md links; remove 5 superseded docs/plan/*.
Backend dead code:
- Remove the dead `folder_repo` field from FileBlobWriteRepository.
- Remove the deprecated GET /api/folders/{id}/listing endpoint
(superseded by /resources).
Verified: cargo clippy (all-features/all-targets) clean, cargo test
--workspace 448 passed, cargo fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The normalized cursor-paginated /resources API is live and the bundled
frontend already uses it for favorites, recent, trash and grants. These five
deprecated old-format endpoints had no remaining frontend or protocol
consumers (the Nextcloud handlers call the service layer directly, not these
HTTP routes), so remove them for a uniform API and less duplicate listing
logic:
- GET /api/folders/{id}/contents → use /api/folders/{id}/resources
- GET /api/folders/{id}/contents/paginated → use /api/folders/{id}/resources
- GET /api/favorites → use /api/favorites/resources
- GET /api/recent → use /api/recent/resources
- GET /api/trash → use /api/trash/resources
Removes the HTTP handlers, their routes, OpenAPI path registrations, and the
now-dead list_folder_contents{,_paginated}_impl helpers + unused imports. The
underlying service methods (favorites_service.get_favorites,
trash_service.get_trash_items, etc.) are KEPT — the Nextcloud OCS/trashbin
handlers depend on them.
Deliberately NOT removed: GET /api/folders/{id}/listing. It is still the Files
view's primary data path and offers ETag/304 conditional caching plus
one-shot favorite/share badge sets that /resources does not yet provide;
migrating it needs a separate parity pass on /resources first.
Updates the OpenAPI structure test and the two docs that referenced the
removed paths. `cargo clippy -D warnings` clean; 443 lib tests pass; OpenAPI
regenerates with the 5 paths gone and the /resources replacements present.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ensure that a user that don't havee permission to a drive cannot search elements in this drive
note: current design: Index are associated to drive, so if a document / directory is shared,
the shared resource will not be in index for targetted user
design is explicitely as is to reduce complexity
now Drive is purely a metadata
each drive has always a root folder
this model minimize Oxicloud changes, and simplify
the Drive name is simply the folder's root's name
note: owner of Drive has more permission that an owner of the root folder
+ request non enumeration on search (a user must not guess existance
of any files he does not have access)
```
┌─────────────────────────────┬──────────────────────────────────────────────────────────────────┬───────────────────────────────────────────────────────────────────────┐
│ Section │ Scope │ Why │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Photos │ Default Personal Drive (capability flag to include other drives) │ High noise risk from non-photo images in shared drives │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Music (library + playlists) │ Cross-drive (capability flag to exclude) │ Low noise risk; audio in shared drives is usually intentional content │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Favorites │ Cross-drive │ Personal curation │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Recent items │ Cross-drive │ Personal history │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Tantivy search │ Cross-drive │ Discovery tool │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Trash (D2) │ Per-drive (owner-actioned) │ Already specified │
└─────────────────────────────┴──────────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────────────────┘
```
Records the real SCRFD+ArcFace analyzer behind the faces-onnx feature,
the FacesConfig env vars, and the deviation (unit-tested geometry split
from the feature-gated ONNX seam). Notes what stays optional: per-user
consent gate, lightbox face boxes, and a periodic re-cluster scheduler.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JW6ghFMDtnRYuYNzZhb47M
Edouard Vanbelle