Commit Graph

1717 Commits

Author SHA1 Message Date
Dionisio Pozo 08f2175455 Merge pull request #535 from swissiety/webdav-litmus-compliance
[QA] integrate litmus (WebDAV server protocol compliance test suite) into ci workflow and fix uncovered compliance issues
2026-06-30 17:16:13 +02:00
M.Schmidt 6ca6649e50 fix(webdav): achieve 100% litmus compliance (59/59 tests)
Complete RFC 4918 litmus test suite compliance (basic 16/16,
copymove 13/13, props 30/30) by fixing six categories of failures:

- PropPatchOp: parse PROPPATCH in document order (RFC 4918 §9.2)
  so that remove-then-set and set-then-remove yield different results
- Null namespace: resolve_name now correctly maps xmlns="" to empty-ns
  QualifiedName instead of falling through to DAV: namespace
- Unicode chars: handle quick-xml 0.39's Event::GeneralRef for character
  references (𐀀) — the parser emits these as GeneralRef, not Text
- MOVE preserves dead props: call rename_resource on MOVE, clearing
  stale destination data even when source has no registered properties
- Malformed PROPFIND: return 400 when body lacks a complete <propfind>
  element (test 2: bare <foo>, test 3: invalid xmlns:prefix="" binding)
- PROPPATCH document order: process ops via Vec<PropPatchOp> instead of
  separate (sets, removes) to honour interleaved remove/set sequences
2026-06-30 08:58:13 +02:00
Dionisio Pozo 3a7ca6722c Merge pull request #534 from EdouardVanbelle/feat/drive 2026-06-29 23:47:34 +02:00
Edouard Vanbelle 934d6b4af0 feat(drive): NC chunked upload respect drive's quota
personal drive are based on user's quota
2026-06-29 23:21:17 +02:00
Edouard Vanbelle 583af517c2 fix(drive): fix drive_id on copy to other drive
- fix alsot drive_id exposition
2026-06-29 23:21:17 +02:00
Edouard Vanbelle e81b297f68 feat(drive): can move|copy to other drives 2026-06-29 23:21:13 +02:00
M.Schmidt 43cf4a2b2b fix(webdav): RFC 4918 handler compliance (COPY/MOVE/MKCOL/PROPFIND/PROPPATCH)
- COPY: return 403 on self-copy, 409 when destination parent missing,
  409 (not 500) when overwriting a locked or conflicting resource
- MOVE: same parent-missing and conflict handling as COPY
- MKCOL: return 405 when collection already exists, 409 when parent
  is missing (no auto-creation of ancestors)
- PUT: return 201 Created for new resources, 204 No Content for overwrites
- PROPFIND: use client-facing URI path (not internal home-folder path)
  for DAV:href values so responses match the request URI
- PROPFIND: include dead properties from DeadPropertyStore in responses
- PROPPATCH: persist set/remove operations to DeadPropertyStore
2026-06-29 22:33:09 +02:00
Edouard Vanbelle ee92d365b9 feat(drive): ensure drive_id updated on file|folder moved to another drive 2026-06-29 21:11:40 +02:00
Edouard Vanbelle acf2311cea test(oidc): ensure that static-dist is built to validate tests 2026-06-29 21:10:26 +02:00
Dionisio Pozo bfb1751f17 Merge pull request #532 from EdouardVanbelle/test/oidc
test(OIDC): sanity check with OIDC
2026-06-29 08:44:12 +02:00
M.Schmidt 1cf48b0dfa fix(webdav): RFC 4918 PROPFIND adapter compliance
- Return unknown properties in a separate 404 propstat (RFC 4918 §9.2)
  via folder_prop_is_known/file_prop_is_known classifiers
- Detect unclosed <propfind> element and return 400 Bad Request
- Add write_folder/file_entry_with_dead_props to include stored dead
  properties in PROPFIND responses
- Change write_folder/file_requested_props to accept &[&QualifiedName]
  to avoid clone overhead when using partition output
2026-06-28 23:51:38 +02:00
Edouard Vanbelle ada4b021bf test(oidc): test OIDC login and refresh flow
- ensure Oxicloud s correctly functional with OIDC login
    - refresh token works
    - givenname, familyname, picture are imported in users' informations
    - test relogin (server is not using same path)
    - test email verified
    - test admin group provided from IdP
2026-06-28 23:22:37 +02:00
Edouard Vanbelle 324fcd486f fix(oidc): fix user's picture copy when OIDC login 2026-06-28 23:20:02 +02:00
Orville Bennett f99764fcff fix: remove panic=abort so catch_unwind guards PDF extraction. Fixes #530 2026-06-28 14:14:19 -04:00
Dionisio Pozo 8ed8fe8a20 Merge pull request #527 from EdouardVanbelle/feat/drive 2026-06-26 22:44:56 +02:00
M.Schmidt b5afa1a855 feat(webdav): implement RFC 4918 dead property store
Add an in-memory DeadPropertyStore backed by RwLock<HashMap> that
stores arbitrary client-supplied XML properties per resource path.
Wire it into AppState so PROPPATCH can persist dead props and PROPFIND
can retrieve them across requests.
2026-06-26 21:19:03 +02:00
Edouard Vanbelle 9697c63210 deps(security): bump pdf-extract → 0.12.0 to pull lopdf 0.42.0
lopdf <0.42.0 has an unbounded-recursion stack overflow on deeply
nested PDF objects (advisory 2026-06-21). The vector through OxiCloud
is the search-index text extractor — anyone who can upload a file can
ship a malicious PDF, and the existing catch_unwind in
text_extractor::extract_pdf does not save us: a stack overflow aborts
the process, it is not a panic.

pdf-extract 0.12.0 requires lopdf ^0.42 which adds the depth bound;
the only consumer call (`extract_text_from_mem`) is API-compatible,
no source changes needed.
2026-06-26 18:57:44 +02:00
Edouard Vanbelle 26d3c692ba feat(drive): policies management from UI 2026-06-26 18:57:35 +02:00
Edouard Vanbelle 66f2aaa250 feat(drive): add drive policie
- add policy forbid_external_sharing
    - add policy forbid_sharing
    - add polocy forbid_cross_drive_move
    - add policy forbid_owner_role_change
2026-06-26 14:04:36 +02:00
Edouard Vanbelle ddb131da8b feat(drive): add policy forbid_public_links 2026-06-26 13:59:02 +02:00
Edouard Vanbelle cfd783cbd3 feat(drive): personal drives have the user's quota in commun 2026-06-26 13:59:02 +02:00
Dionisio Pozo 996ace8a1d Merge pull request #524 from EdouardVanbelle/fix/cascading-deletion-and-file-life-cycle
fix(fileLifeCycle): add missing on_file_deleted() with cascading deletion
2026-06-26 12:35:25 +02:00
Dionisio Pozo 2fb2bcb8f3 Merge pull request #526 from EdouardVanbelle/feat/recent
feat(recent): update recent list on server side
2026-06-26 07:35:32 +02:00
Edouard Vanbelle f4032e9e23 feat(recent): update recent list server side
initially the recent was done client side
    recent files are now directly updated on serverside when accessing a file

    note: nextcloud and webdav voluntary not included
2026-06-26 01:44:19 +02:00
Edouard Vanbelle fefba83cfc fix(fileLifeCycle): add missing on_file_deleted with cascading del
on_file_deleted() was missing on folder deletion or on cascading deletion
2026-06-25 23:36:50 +02:00
M.Schmidt 5a001c4120 fix(webdav): remove spurious calendar_rest_handler module declaration
The declaration was brought in by a cherry-pick but that handler does
not exist on this branch, causing a compilation error.
2026-06-25 22:44:27 +02:00
Dionisio Pozo 11e34762e3 Merge pull request #520 from EdouardVanbelle/feat/drive-d1 2026-06-25 22:10:12 +02:00
Edouard Vanbelle 81d50bde33 fix(GC): race condition in dedup_service
Stamp `orphaned_at = now()` on the ref-0 row so it sits inside the
     GC grace window for the duration of this test. Without it,
     `orphaned_at IS NULL` is treated by `garbage_collect` as
     "pre-migration, immediately reapable" — and any sibling test in
     the shared pool that calls `garbage_collect()` (e.g.
     `garbage_collect_respects_grace_and_cross_checks`) would race
     with the pin below and delete the row first.
2026-06-25 19:00:19 +02:00
Bradley Nelson 94e5aec4af test(mounts): cover download range logic + add just coverage recipe
- Extract the WebDAV/REST mount-download Range decision (parse + validate +
  satisfiable) into a pure `plan_mount_range`, unit-tested across no-range,
  malformed, inclusive, open-ended, suffix, and out-of-bounds cases. This was
  the one untested branch of `download_mount_file` (its I/O deps are already
  integration-covered).
- Add `just coverage [filter]` and `just coverage-integration [filter]`
  recipes wrapping cargo-llvm-cov (HTML + summary; integration variant adds
  the testcontainers tests).
2026-06-25 01:11:45 -06:00
Bradley Nelson 07320a3925 feat(mounts): P4 — admin CRUD endpoints + frontend admin tab
Mounts are now configurable at runtime without DB surgery.

Backend (admin-gated, /api/admin/external-mounts):
- GET    list all configured mounts
- POST   create: validate the provider config up front, create a mount-root
         folder under the admin's drive, insert the row, hot-reload the registry
- DELETE remove the mount row + its root folder (host content is left intact),
         then hot-reload
- ExternalMountRepositoryPort gains create/delete (PG impl); registry.reload()
  runs in-process so changes are live immediately
- audit lines event="external_mount.config" action=create|delete

Frontend:
- admin.ts endpoints: listExternalMounts / createExternalMount / deleteExternalMount
- new "External Mounts" tab in the admin page: add form (name + host path +
  read-only) and a list with delete

Integration test for the repo create/delete round-trip (testcontainers).
2026-06-25 01:07:03 -06:00
Bradley Nelson d8229650ef feat(mounts): P3 — WebDAV/NextCloud path resolution + browse/download
Mounts are now browsable and downloadable over both WebDAV surfaces
(/webdav/ and NextCloud /remote.php/dav) and NextCloud clients. The work
funnels through the path-based service methods all three surfaces already use,
so both WebDAV handlers gain mount support with no handler changes.

- get_folder_by_path / get_file_by_path resolve a path that descends past a
  mount root to a synthetic ext: DTO (the mount root itself stays a real row)
- list_folders_paginated_with_perms / list_files_batch_with_perms branch to the
  provider, so PROPFIND Depth:1 enumerates mount directory contents in both
  WebDAV handlers
- get_file_stream / get_file_range_stream branch ext: file ids to the provider,
  so WebDAV GET streams mount content (Pin<Box<dyn Stream>> re-boxed)
- router injected into FileRetrievalService; new MountRouter::find_path delegates
  to the registry's (drive_id, mount_path) index
- WebDAV mkdir/delete/move/rename by path work via path→ext:id→the P2 service
  methods (no extra wiring)

Fixes a leading-slash normalization bug in the registry path index: materialized
folder paths arrive both as `Personal/Media` and `/Personal/Media`; keys and
lookups now normalize the leading slash (would have broken real WebDAV paths).

Known follow-up: WebDAV PUT (upload/update by path) still ingests to the CAS;
streaming a WebDAV PUT straight to the provider needs a pre-ingest branch in the
two WebDAV PUT handlers (mirrors the REST upload branch).

Integration test: get_folder_by_path/get_file_by_path resolution, PROPFIND
Depth:1 folder+file listing, and content streaming on a real mount + Postgres.
2026-06-25 00:51:16 -06:00
Bradley Nelson 8e3e31da4d feat(mounts): P2 — read-write REST for external mounts
Adds full CRUD on mount contents, mirroring the P1 read pattern (handlers/
services classify; authorization stays in the service via the mount-root
folder grant; the provider does the I/O).

- mkdir / rename / delete / move-within branch inside FolderService and
  FileManagementService (router injected into both)
- streaming upload via a new ExternalUploadService: the upload handler detects a
  mount destination BEFORE the CAS ingest and streams the multipart body
  straight to the provider (no BLAKE3/dedup). `write_stream` now takes a
  lifetime-bound boxed stream so the borrowing multipart field can be passed
  without buffering.
- deletes on mounts are permanent (no trash): the trash-first folder handler
  routes `ext:` ids straight to the provider delete; file delete goes through the
  branched delete_and_cleanup
- cross-backend move/copy (mount ↔ native, or between mounts) is forbidden
  (UnsupportedOperation); the mount root itself cannot be renamed/moved/deleted
- every mutation emits a `target:"audit" event="external_mount.write"` line
- shared mount_dto builders synthesize FolderDto/FileDto from a provider MountStat

Tests: 529 unit + integration tests for mkdir/rename/delete, file rename/delete,
streaming upload, cross-boundary forbid, and stranger-denied — all against real
Postgres + a real provider (testcontainers).
2026-06-25 00:30:10 -06:00
Bradley Nelson 3c31695579 feat(mounts): external file mounts P1 — pluggable provider + read-only REST
Adds the foundation for external file mounts: admin-configured backends
(raw host filesystem in v1; sftp/webdav/… as future provider kinds) surfaced
as a folder inside a user's drive. Mount contents are virtual/live-passthrough
— read straight from the backend, never stored in storage.files — and are a
deliberately separate, limited storage type (no dedup/sharing/trash/search).
The feature is dark by default (OXICLOUD_ENABLE_EXTERNAL_MOUNTS=false).

P1 scope (this PR): data model, the pluggable provider abstraction, and the
read-only REST surface (mount listing + download). Read-write (P2),
WebDAV/NextCloud path resolution (P3), and the admin UI (P4) follow.

Core model
- Mount root = a real storage.folders row; authorization for everything inside
  collapses onto that folder UUID (ltree-ancestry grant cascade).
- Children are virtual, addressed by ext:<mount_id>:<base64url(node_id)> where
  node_id is provider-owned and opaque to the rest of the system.
- A lock-free (arc-swap) MountRegistry maps mount-root UUID -> provider; a thin
  MountRouter::classify() is the single cheap hook handlers call before parsing
  an id as a UUID. With no mounts configured it always returns Regular, so
  existing code paths are unchanged.

Added
- migrations/20260805000000_external_mounts.sql (storage.external_mounts, kind + config JSONB)
- domain/services/external_mount_id (id envelope + virtual etags)
- application/ports/external_mount_ports (ExternalMountProvider, MountProviderFactory, repo port)
- infrastructure local_fs_mount_provider (tokio::fs, symlink-escape-safe) + factory
- application MountRegistry + MountRouter, pg ExternalMountRepository
- DI wiring (AppState.mount_router), FeaturesConfig.enable_external_mounts
- listing branch (FolderService::list_mount_dir_with_perms + folder_handler) and
  download branch (FileRetrievalService stat/open mount methods + file_handler)

Authorization stays in the service layer (authz.require(Resource::Folder(mount_id)));
handlers only classify. Cross-backend operations are out of scope for P1.

Tests: 529 unit tests + 5 testcontainers integration tests (real Postgres 17),
including end-to-end authorization (owner allowed, stranger denied). Line
coverage of the new modules is 84–100% (cargo-llvm-cov). Known gap:
file_handler::download_mount_file (HTTP glue) needs a full-app test (P4).
2026-06-24 23:52:01 -06:00
Edouard Vanbelle 4a99eb2453 chore(ci): make space on CI runners 2026-06-25 00:41:31 +02:00
Edouard Vanbelle 2604ac3309 refactor: format syntaxt 2026-06-25 00:07:16 +02:00
Edouard Vanbelle 2643484860 test(storage_cleanup_check.sh): add support of drive cleanup 2026-06-25 00:05:36 +02:00
Edouard Vanbelle 8387473b26 bigfix(gc): fix leak with GC and blob lifecycle
call fire_blob_hooks to respect lifecycle

    dedup_service::garbage_collect_with_grac must call fire_blob_hooks() once blob are dropped
    so other services like thumbnail can proceed to their cleanup
2026-06-24 23:51:25 +02:00
Edouard Vanbelle 90474aa885 chore(test): blob lifecyclc with thumbnail cleanup
renable thumbnail test, ensure that blob lifecycle correctly
    trigger thumbnail cleanup on blob deletion

    need to call `/api/admin/internal/trigger-gc?force=true`
2026-06-24 23:50:37 +02:00
M.Schmidt 5913fff047 feat(webdav): RFC 4918 prop classification and 404 propstat 2026-06-24 23:08:51 +02:00
Edouard Vanbelle 6b8e2ba49c test(drive): check quota calculus 2026-06-24 23:01:34 +02:00
Edouard Vanbelle a9a8604322 feat(drive): add quota calculus per drive
- quota per drive
    - add 2 internal API endpoints to test purpose
        disabled by default, enable it via `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true`

        this enable:

            /api/admin/internal/trigger-sweep
                to sweep the trash and recalculated quota

            /api/admin/internal/trigger-gc
                to garbage orphan blobs

        use full for end to end tests and validate lifecycles
2026-06-24 22:28:31 +02:00
Edouard Vanbelle b73f176024 feat(drive): permanent deletion per drive 2026-06-24 21:56:27 +02:00
Edouard Vanbelle 7d24015fc4 feat(drive): add drive deletion
- conditions: drive must be empty
    - deletion forbidden on main personal drive
2026-06-24 21:17:27 +02:00
Edouard Vanbelle 33cfa876d0 chore(frontend/test): normalize test environement
normalize environment to avoid such issues during tests
    on non EN local machine:

    ```
    AssertionError: expected 'il y a 2 ans' to match /year/

   ❯ src/lib/utils/time.test.ts:24:60
       22|
       23|  it('formats past times in the largest matching unit', () => {
       24|   expect(relativeTimeAgo(Date.now() - 2 * 31_536_000_000)).toMatch(/year/);
    ```
2026-06-24 20:53:03 +02:00
Dionisio Pozo 1d0fa27991 Merge pull request #519 from EdouardVanbelle/feat/drive-d1 2026-06-24 08:26:00 +02:00
Edouard Vanbelle 754e6239da fix(drive): correct integration test 2026-06-24 08:02:40 +02:00
Edouard Vanbelle 338915a912 security(quinn-proto): bump to 0.11.15 (RUSTSEC-2026-0185 patched) 2026-06-24 02:41:35 +02:00
Edouard Vanbelle 289cf19270 feat(drive): user can rename drive
- only owners can rename root folders name (aka the drive name)
    - add UI to rename drive's name
2026-06-24 02:03:21 +02:00
Edouard Vanbelle 6034dd47d9 feat(drive): improve drive edition from owners 2026-06-24 02:03:17 +02:00
Edouard Vanbelle d77846119f feat(drive): UI: add drive edition for admin 2026-06-24 01:20:44 +02:00