Commit Graph

135 Commits

Author SHA1 Message Date
Edouard Vanbelle 75601beb43 security(wopi): add authz to Wopi 2026-07-05 23:31:44 +02:00
Edouard Vanbelle 1786fe4111 security(nextcloud): chroot-aware display paths + recent race fix
strip_chroot_prefix replaces the hardcoded "Personal/" strip
    in NC trashbin PROPFIND, OCS unified search, and REPORT
    (favorites + search). Handles composed chroots, drops
    cross-chroot items instead of surfacing malformed paths, and
    fixes the leading-slash mismatch (FolderDto path has '/', DB
    paths don't) that silently dropped every NC trashbin item
    post-D3. OCS keeps a first-segment fallback (results
    legitimately span drives, no single chroot).

    uploads_handler switches to nc_to_internal_path(chroot, …)
    for the two remaining hardcoded "Personal/" sites, closing
    the D1 TODO markers.

    RecentService::record_item_access is split from a new
    record_item_access_internal (no authz) used by
    RecentRecordingHook. Round 1's authz.require widened the
    tokio::spawn race past tests/api/recent.hurl step 7; the
    internal path skips the redundant Read gate — upstream
    _with_perms already enforced it.

    Tests: 8 unit tests pin strip_chroot_prefix (leading slash,
    composed chroots, sibling-leak rejection, partial-prefix,
    empty-chroot). drives_membership.hurl step 21b/22b cover
    Editor upload → 201 / Viewer upload → 404 fresh + overwrite
    with fixture cleanup at 30c. test_nc_move_copy_delete_trash
    K1 pins the actual original-location value.
2026-07-05 22:53:05 +02:00
Edouard Vanbelle b95e740b2f security(music): ensure read permission via authz 2026-07-05 22:53:05 +02:00
M.Schmidt 3501857a70 test(webdav): cover protected-property PROPPATCH rejection
Native + NC surfaces: DAV: displayname/getetag, oc:fileid/
permissions, nc:has-preview all 403 and never land in the store;
mixed request shows per-property granularity (protected prop 403
alongside an ordinary custom prop 200); oc:favorite regression
guard confirms its special-case still works despite being on the
protected list.
2026-07-01 22:54:56 +02:00
Edouard Vanbelle 09339ea63f feat(drive): UI: show policiesto drive's members
and add tests
2026-07-01 22:35:08 +02:00
Edouard Vanbelle 01ff7dab0b feat(drive): impl policy photo + music policies
add `include_in_photo_index` and `include_in_music_index` policies
    both true for default personal drive

    photo is implemented
    music is not yet implemented
2026-07-01 21:57:37 +02:00
M.Schmidt 94e0145855 test(webdav): cover NC dead-props PROPPATCH/PROPFIND contract
Round-trip, upsert, remove, 404 on missing resource, survives MOVE,
reaped on DELETE, oc:favorite regression guard, folder coverage.
2026-07-01 19:59:23 +02:00
Edouard Vanbelle d8adae3572 feat(dead-props): ensure replication on copy 2026-06-30 23:48:44 +02:00
Edouard Vanbelle cb7b653a15 feat(webdav): bind dead prop to res. id rather path 2026-06-30 23:35:25 +02:00
Edouard Vanbelle cf1479d113 test(webdav): cover nested folder move
this test is a duplicated with the move in litmus test suite
    it has been added because not everybody do have litmus installed
    and no clue if litmus will be kept in the future
2026-06-30 22:55:49 +02:00
Edouard Vanbelle 21ac3a178f test(webdav): cover the dead properties 2026-06-30 20:32:16 +02:00
Edouard Vanbelle 85bc6c51de fix(test): fix quota test (race condition in test) 2026-06-30 20:27:15 +02:00
Edouard Vanbelle 1e2882973b fix(test): correct due to commit 43cf4a2bg
- MKCOL is now better protected
    - Webdav now handle 201 (created) 204 (overritten)
2026-06-30 20:18:19 +02:00
Edouard Vanbelle 583af517c2 fix(drive): fix drive_id on copy to other drive
- fix alsot drive_id exposition
2026-06-29 23:21:17 +02:00
Edouard Vanbelle ee92d365b9 feat(drive): ensure drive_id updated on file|folder moved to another drive 2026-06-29 21:11:40 +02:00
Edouard Vanbelle 66f2aaa250 feat(drive): add drive policie
- add policy forbid_external_sharing
    - add policy forbid_sharing
    - add polocy forbid_cross_drive_move
    - add policy forbid_owner_role_change
2026-06-26 14:04:36 +02:00
Edouard Vanbelle ddb131da8b feat(drive): add policy forbid_public_links 2026-06-26 13:59:02 +02:00
Edouard Vanbelle cfd783cbd3 feat(drive): personal drives have the user's quota in commun 2026-06-26 13:59:02 +02:00
Edouard Vanbelle f4032e9e23 feat(recent): update recent list server side
initially the recent was done client side
    recent files are now directly updated on serverside when accessing a file

    note: nextcloud and webdav voluntary not included
2026-06-26 01:44:19 +02:00
Edouard Vanbelle 2643484860 test(storage_cleanup_check.sh): add support of drive cleanup 2026-06-25 00:05:36 +02:00
Edouard Vanbelle 90474aa885 chore(test): blob lifecyclc with thumbnail cleanup
renable thumbnail test, ensure that blob lifecycle correctly
    trigger thumbnail cleanup on blob deletion

    need to call `/api/admin/internal/trigger-gc?force=true`
2026-06-24 23:50:37 +02:00
Edouard Vanbelle 6b8e2ba49c test(drive): check quota calculus 2026-06-24 23:01:34 +02:00
Edouard Vanbelle b73f176024 feat(drive): permanent deletion per drive 2026-06-24 21:56:27 +02:00
Edouard Vanbelle 7d24015fc4 feat(drive): add drive deletion
- conditions: drive must be empty
    - deletion forbidden on main personal drive
2026-06-24 21:17:27 +02:00
Edouard Vanbelle 289cf19270 feat(drive): user can rename drive
- only owners can rename root folders name (aka the drive name)
    - add UI to rename drive's name
2026-06-24 02:03:21 +02:00
Edouard Vanbelle a5b24a7453 feat(drive): add /api/drive
- permit shared drive creation from oxicloud admin (for now)
    - prepare other personal drive creation (Not implemented), need to validate
    quota policies and strategy first
    - add hurl test to verify permissions
2026-06-23 23:50:57 +02:00
Edouard Vanbelle 184520c17a feat(drive): plug trash to drives 2026-06-23 22:04:20 +02:00
Edouard Vanbelle 6f1f44f962 feat(drive): prepare E2E tests on drive membership 2026-06-23 20:26:34 +02:00
Bradley Nelson e3823ce470 test(e2e): Playwright + Vitest coverage harness and test instrumentation
Add an end-to-end and unit test suite for the SvelteKit frontend:

- Playwright e2e specs (tests/e2e/spa) with a throwaway container stack,
  codegen scenarios, and an Istanbul-based coverage report pipeline.
- Vitest unit tests across API endpoints, components, stores and composables.
- `data-testid` hooks on interactive elements (AppShell, FileViewer,
  ShareDialog, search, photos, files breadcrumbs, login/Nextcloud flows,
  public share pages) so the e2e suite can target them deterministically.
- Serve the SPA app-shell CSP from a <meta> policy (svelte.config.js) plus a
  middleware that skips the CSP header on HTML; move the Nextcloud Login Flow
  v2 grant page to the SvelteKit /nextcloud/login route.
- `just front-codegen` recipe and start-server-spa.sh harness.

Make the test environment robust and consistent:
- Install a deterministic in-memory localStorage/sessionStorage in the Vitest
  setup so storage behaves identically across Node versions (Node 26 ships a
  native Web Storage global that otherwise shadows jsdom's).
- Pin devenv to Node 26 + PostgreSQL 18 and pin every CI job to Node 26.3.0
  so the dev shell and CI run the same toolchain versions.

Repair the API/WebDAV (hurl) suite, which had drifted from the backend:
- Migrate the removed `/api/folders/{id}/listing` endpoint to `/resources`
  (cursor-paginated `{items:[{resource_type,resource}]}` shape) across the
  batch-copy, grants, nested-group, and WebDAV NC tests + the dav_helpers
  wipe routine.
- Stop photos_etag from uploading the dedup-tracked fixture so the dedup
  blob-lifecycle test can own its content-addressed blob exclusively.
- dedup_create now asserts the idempotent same-content re-upload (201 +
  existing file id) instead of the stale 409 expectation.

Generated coverage reports, nyc output and the e2e server runtime data dir
are gitignored rather than committed.
2026-06-22 00:05:06 -06:00
DioCrafts 0c40c69f9b perf(photos): ETag/304 conditional revalidation on the timeline
GET /api/photos sent only X-Next-Cursor — no ETag — so every gallery
re-mount rebuilt up to 500 PhotoDtos, serde-serialized the whole vector,
and shipped the full body even when nothing changed.

The handler now emits a lightweight content-derived ETag
(hash of before + limit + max(modified_at) + row count) and honours
If-None-Match, with Cache-Control: private, no-cache so the SPA's default
fetch cache mode always revalidates. An unchanged "navigate away and back"
becomes an empty 304 instead of a full rebuild + reserialize + transfer.
The DB query still runs (the cheap part); the win is skipping the DTO
build, serialization, and body bytes.

Proven end-to-end (throwaway Postgres + server, 7 images):
  1st GET (no If-None-Match)      -> 200  4586 bytes + ETag
  2nd GET (If-None-Match matches) -> 304     0 bytes
  3rd GET (If-None-Match stale)   -> 200  4586 bytes (correctly invalidated)
~655 B/photo, so a full 500-row first page saves ~320 KB + a 500-DTO
build/serialize per unchanged revalidation. Unlike a cold load this is the
common gallery-navigation path, so it hits real user-facing latency.

Regression test: tests/api/photos_etag.hurl (added to the api-test suite).
Methodology in benches/PHOTOS-ETAG.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 00:22:49 +02:00
Edouard Vanbelle 74b33744ed chore(test): disable storage_cleanup_check.sh until GC trigger lands 2026-06-19 16:06:38 +02:00
Edouard Vanbelle dd9e3b8868 feat(drive): test Tantivy lookup with drive
ensure that a user that don't havee permission to a drive cannot search elements in this drive
    note: current design: Index are associated to drive, so if a document / directory is shared,
    the shared resource will not be in index for targetted user
    design is explicitely as is to reduce complexity
2026-06-19 16:06:38 +02:00
Edouard Vanbelle e7f4826778 feat(drive): complete updated_by created_by 2026-06-19 16:06:38 +02:00
Edouard Vanbelle 16ea08b093 feat(drive): improve Drive model
now Drive is purely a metadata
    each drive has always a root folder
    this model minimize Oxicloud changes, and simplify
    the Drive name is simply the folder's root's name
    note: owner of Drive has more permission that an owner of the root folder
2026-06-19 16:06:37 +02:00
Edouard Vanbelle eab7a609b9 feat(drive): start implementation of drive
- add storage.drives
    - prepare migration phase
    - add created_by and updated_by on storage.folders
2026-06-19 16:06:37 +02:00
Edouard Vanbelle 72129af0bd refactor(role): use grant only
- remove permission centric mode
    - finalize migration drop all tables with permissions
    - ensure roles are ENUM (owner is always displayed first)
2026-06-18 02:47:06 +02:00
Edouard Vanbelle f168c4578f feat(roles): prepare migration ReBAC to roles
prepare migration of permission to roles
    this simplify drastically database (permission are now simply defined in code)
    and will permit reuse of the same ReBAC engine to define owners of drives

    mapping:

    ```
        Role::Viewer => &[Permission::Read],
        Role::Commenter => &[Permission::Read, Permission::Comment],
        Role::Contributor => &[Permission::Read, Permission::Create],
        Role::Editor => &[
            Permission::Read,
            Permission::Comment,
            Permission::Create,
            Permission::Update,
        ],
        Role::Owner => &[
            Permission::Read,
            Permission::Comment,
            Permission::Create,
            Permission::Update,
            Permission::Share,
            Permission::Delete,
            Permission::Manage,
        ],
    ```
2026-06-18 02:47:06 +02:00
Dionisio Pozo b8a0018785 Merge pull request #473 from EdouardVanbelle/fix/nextcloud+webdav
fix(nextcloud+webdav) fix bugs found via end to end tests
2026-06-17 12:42:31 +02:00
Edouard Vanbelle 619c24e1e9 test(api): unpin folder-share file fetch; assert in/out-of-scope
The KNOWN BUG pin for `GET /api/s/{folder-token}/file/{file_id}` was
stale — the route now returns 200 + body for files inside the share's
subtree and 404 for anything outside it. Replaces the sidestep
comment with two positive assertions:

  - in-share: 200 + Content-Disposition references the file name
  - out-of-share (caller-owned file in a different folder): 404,
    matching "no such file" so the response can't be used to
    enumerate foreign file ids

Coverage now exercises the actual recipient-side download path that
NC desktop and web clients use; the file-share variant (item_type=file)
moves down to test 8b.

Adds a teardown DELETE for the outsider hello.txt so the next test in
the runner (permissions.hurl) can re-upload its own hello.txt into
admin's home folder without hitting the live-name unique index.
2026-06-17 09:32:37 +02:00
Edouard Vanbelle 68abb891d7 feat(by-hash): allow /by-hash even if blob is trashed
- permit reuse of blob where file is trashed (by-hash and chunked)
    - add hurl test on /api/files/by-hash
    - add anti enumeration of blob (404 is always blob_not_owned_by_caller)
2026-06-16 22:41:55 +02:00
Edouard Vanbelle 6d65f7eb09 chore(test): add new API coverage
ensure better API test coverage on important routes

  1. tests/api/public_shares.hurl — create a share token, verify, list contents, fetch a file, fetch a folder zip, then revoke and re-verify with the token. Same pattern as grants.hurl. ~30 min, biggest security ROI.
  2. tests/api/auth_session_lifecycle.hurl — login → refresh → use new token → logout → refresh-rejected → login-again. Covers the session-family invalidation contract.
  3. tests/api/admin_user_ops.hurl — admin disables / re-enables / changes role / resets password / sets quota for a fixture user. Five POSTs.
  4. tests/api/groups_effective_members.hurl — nested groups: A contains B contains user X; effective-members returns X. Two scenarios, but it's the ReBAC contract under the Drive refactor.
  5. tests/api/search_basic.hurl — upload foo.txt, search "foo", get the result; cross-user: bob can't search alice's foo.
2026-06-13 19:03:07 +02:00
Edouard Vanbelle da65987e58 chore(test): retry storage check to prevent race condition
case: server on CI may be overloaded which can create race
    condition between file/bolb deletion andd check
2026-06-13 19:01:41 +02:00
Edouard Vanbelle ce69bbe151 feat(bin): add --version and --config parameters
--config is to prevent Oxicloud loading any other .env, usefully when you want to test with different environements
2026-06-13 19:01:41 +02:00
Edouard Vanbelle 140cbb440e chore(test): fix deprecated call or syntax 2026-06-13 19:01:40 +02:00
Edouard Vanbelle 595273277b test(e2e): webdav + nextcloud full e2e test coverage
add a full coverage of Webdav and Nextcloud
    purpose: prepare move to Drives and ensure no regression at all

    test scenarios are in docs/plan/BASELINE_TESTS_NC_WEBDAV.md

    current existing bugs identified via these tests:

      ┌──────────┬─────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┐
      │   Bug    │ Surface │                                            Pin location                                            │
      ├──────────┼─────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
      │ G4/G5/K5 │ NC      │ AlreadyExists → 500 instead of 412 (handle_move + trashbin restore)                                │
      ├──────────┼─────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
      │ G9       │ NC      │ Folder DELETE not row-recursive — orphan descendants stay live                                     │
      ├──────────┼─────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
      │ M5/M7    │ Native  │ resolve_path_for_user mismatch — PUT writes, GET reads via lenient lookup, MOVE/DELETE can't find  │
      │          │         │ via strict                                                                                         │
      ├──────────┼─────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
      │ M8       │ Native  │ COPY discards destination filename — collides with source                                          │
      ├──────────┼─────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
      │ N2       │ Native  │ LOCK creates the token, mutators don't check it — class-2 advertisement is aspirational            │
      └──────────┴─────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘
2026-06-13 19:01:40 +02:00
Edouard Vanbelle 50ea406719 feat(OXICLOUD_DIRECT_PUT_MAX_BYTES): add a security limit on direct PUT
ensure files does not exeed OXICLOUD_MAX_UPLOAD_SIZE, prefer to deny from header rather consuming bandwidth
    add OXICLOUD_DIRECT_PUT_MAX_BYTES for direct PUT (non chunked), admins can fine tune their prefered values
2026-06-09 11:06:44 +02:00
Edouard Vanbelle 964d66e7f8 featchukn-upload): client can provide full file hash completion 2026-06-09 09:55:27 +02:00
Edouard Vanbelle 41aad26702 feat(upload): cover chunk upload + add support of different digest hash
Prefer stream storage rather using buffered (in memory)

  note: on many unix like tmpfs are in-memory, sungle PUT are sized limited

  Storage map (NC stands for Nextcloud gateway)

  ┌───────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────┐
  │                   Streaming surface                   │                            Destination                             │                Configurable via                 │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST chunked PUT /api/uploads/{id} chunk              │ {storage_path}/.uploads/{upload_id}/chunk_{NNNNNN}                 │ OXICLOUD_STORAGE_PATH (the .uploads subdir is   │
  │                                                       │                                                                    │ hard-wired)                                     │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST chunked assemble (during /complete)              │ {storage_path}/.uploads/{upload_id}/assembled                      │ same                                            │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ NC chunked PUT /dav/uploads/.../{chunk}               │ {storage_path}/.uploads/nextcloud/{user}/{upload_id}/{chunk_name}  │ same                                            │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ NC chunked assemble (during MOVE)                     │ {storage_path}/.uploads/nextcloud/{user}/{upload_id}/.assembled    │ same                                            │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ NC single-file PUT /dav/files/.../{path} (via         │ OXICLOUD_UPLOAD_TMPDIR if set, else OS default temp (/tmp on       │ OXICLOUD_UPLOAD_TMPDIR                          │
  │ spool_body_to_temp)                                   │ Linux)                                                             │                                                 │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST WebDAV PUT /webdav/{path} (via                   │ same as above                                                      │ OXICLOUD_UPLOAD_TMPDIR                          │
  │ spool_body_to_temp)                                   │                                                                    │                                                 │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST multipart upload /api/files/upload               │ {storage_path}/.dedup_temp/upload-{uuid}                           │ OXICLOUD_STORAGE_PATH (hard-wired subdir)       │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ WOPI PutFile                                          │ OS default temp via NamedTempFile::new() (no override)             │ (none — bug worth tracking)                     │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ Final blob storage (after fsync + rename)             │ {storage_path}/.blobs/{ab}/{abc…}.blob                             │ OXICLOUD_STORAGE_PATH                           │
  └───────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────┘

  one caveat: a malicious user can create many chunked upload and saturate local storage
2026-06-09 09:52:09 +02:00
Edouard Vanbelle 5e638691ad security(upload): cap upload size to prevent memody/disk consumption
add OXICLOUD_CHUNK_MAX_BYTES which correspond to the max upload chunk allowed
    (differs from OXICLOUD_MAX_UPLOAD_SIZE which is the max total size of a file)
    hurl test validate the change

    Streams the request body straight to the chunk file with peak heap of
    ~one HTTP frame, regardless of chunk size or the configured cap. The
    `storage.chunk_max_bytes` config (env `OXICLOUD_CHUNK_MAX_BYTES`,
    default 100 MB) bounds a single PUT — separate from `max_upload_size`
    which governs whole-file uploads. Without this separation, a client
    could submit a chunk up to the whole-file cap (10 GB default) and
    monopolise server memory.
2026-06-09 09:26:33 +02:00
Edouard Vanbelle 5a18ec4bac test(hash_content): check that hash matches 2026-06-06 18:49:18 +02:00