Commit Graph

77 Commits

Author SHA1 Message Date
Edouard Vanbelle cb7b653a15 feat(webdav): bind dead prop to res. id rather path 2026-06-30 23:35:25 +02:00
Edouard Vanbelle cf1479d113 test(webdav): cover nested folder move
this test is a duplicated with the move in litmus test suite
    it has been added because not everybody do have litmus installed
    and no clue if litmus will be kept in the future
2026-06-30 22:55:49 +02:00
Edouard Vanbelle 21ac3a178f test(webdav): cover the dead properties 2026-06-30 20:32:16 +02:00
Edouard Vanbelle 85bc6c51de fix(test): fix quota test (race condition in test) 2026-06-30 20:27:15 +02:00
Edouard Vanbelle 1e2882973b fix(test): correct due to commit 43cf4a2bg
- MKCOL is now better protected
    - Webdav now handle 201 (created) 204 (overritten)
2026-06-30 20:18:19 +02:00
Edouard Vanbelle 583af517c2 fix(drive): fix drive_id on copy to other drive
- fix alsot drive_id exposition
2026-06-29 23:21:17 +02:00
Edouard Vanbelle ee92d365b9 feat(drive): ensure drive_id updated on file|folder moved to another drive 2026-06-29 21:11:40 +02:00
Edouard Vanbelle 66f2aaa250 feat(drive): add drive policie
- add policy forbid_external_sharing
    - add policy forbid_sharing
    - add polocy forbid_cross_drive_move
    - add policy forbid_owner_role_change
2026-06-26 14:04:36 +02:00
Edouard Vanbelle ddb131da8b feat(drive): add policy forbid_public_links 2026-06-26 13:59:02 +02:00
Edouard Vanbelle cfd783cbd3 feat(drive): personal drives have the user's quota in commun 2026-06-26 13:59:02 +02:00
Edouard Vanbelle f4032e9e23 feat(recent): update recent list server side
initially the recent was done client side
    recent files are now directly updated on serverside when accessing a file

    note: nextcloud and webdav voluntary not included
2026-06-26 01:44:19 +02:00
Edouard Vanbelle 2643484860 test(storage_cleanup_check.sh): add support of drive cleanup 2026-06-25 00:05:36 +02:00
Edouard Vanbelle 90474aa885 chore(test): blob lifecyclc with thumbnail cleanup
renable thumbnail test, ensure that blob lifecycle correctly
    trigger thumbnail cleanup on blob deletion

    need to call `/api/admin/internal/trigger-gc?force=true`
2026-06-24 23:50:37 +02:00
Edouard Vanbelle 6b8e2ba49c test(drive): check quota calculus 2026-06-24 23:01:34 +02:00
Edouard Vanbelle b73f176024 feat(drive): permanent deletion per drive 2026-06-24 21:56:27 +02:00
Edouard Vanbelle 7d24015fc4 feat(drive): add drive deletion
- conditions: drive must be empty
    - deletion forbidden on main personal drive
2026-06-24 21:17:27 +02:00
Edouard Vanbelle 289cf19270 feat(drive): user can rename drive
- only owners can rename root folders name (aka the drive name)
    - add UI to rename drive's name
2026-06-24 02:03:21 +02:00
Edouard Vanbelle a5b24a7453 feat(drive): add /api/drive
- permit shared drive creation from oxicloud admin (for now)
    - prepare other personal drive creation (Not implemented), need to validate
    quota policies and strategy first
    - add hurl test to verify permissions
2026-06-23 23:50:57 +02:00
Edouard Vanbelle 184520c17a feat(drive): plug trash to drives 2026-06-23 22:04:20 +02:00
Edouard Vanbelle 6f1f44f962 feat(drive): prepare E2E tests on drive membership 2026-06-23 20:26:34 +02:00
Bradley Nelson e3823ce470 test(e2e): Playwright + Vitest coverage harness and test instrumentation
Add an end-to-end and unit test suite for the SvelteKit frontend:

- Playwright e2e specs (tests/e2e/spa) with a throwaway container stack,
  codegen scenarios, and an Istanbul-based coverage report pipeline.
- Vitest unit tests across API endpoints, components, stores and composables.
- `data-testid` hooks on interactive elements (AppShell, FileViewer,
  ShareDialog, search, photos, files breadcrumbs, login/Nextcloud flows,
  public share pages) so the e2e suite can target them deterministically.
- Serve the SPA app-shell CSP from a <meta> policy (svelte.config.js) plus a
  middleware that skips the CSP header on HTML; move the Nextcloud Login Flow
  v2 grant page to the SvelteKit /nextcloud/login route.
- `just front-codegen` recipe and start-server-spa.sh harness.

Make the test environment robust and consistent:
- Install a deterministic in-memory localStorage/sessionStorage in the Vitest
  setup so storage behaves identically across Node versions (Node 26 ships a
  native Web Storage global that otherwise shadows jsdom's).
- Pin devenv to Node 26 + PostgreSQL 18 and pin every CI job to Node 26.3.0
  so the dev shell and CI run the same toolchain versions.

Repair the API/WebDAV (hurl) suite, which had drifted from the backend:
- Migrate the removed `/api/folders/{id}/listing` endpoint to `/resources`
  (cursor-paginated `{items:[{resource_type,resource}]}` shape) across the
  batch-copy, grants, nested-group, and WebDAV NC tests + the dav_helpers
  wipe routine.
- Stop photos_etag from uploading the dedup-tracked fixture so the dedup
  blob-lifecycle test can own its content-addressed blob exclusively.
- dedup_create now asserts the idempotent same-content re-upload (201 +
  existing file id) instead of the stale 409 expectation.

Generated coverage reports, nyc output and the e2e server runtime data dir
are gitignored rather than committed.
2026-06-22 00:05:06 -06:00
DioCrafts 0c40c69f9b perf(photos): ETag/304 conditional revalidation on the timeline
GET /api/photos sent only X-Next-Cursor — no ETag — so every gallery
re-mount rebuilt up to 500 PhotoDtos, serde-serialized the whole vector,
and shipped the full body even when nothing changed.

The handler now emits a lightweight content-derived ETag
(hash of before + limit + max(modified_at) + row count) and honours
If-None-Match, with Cache-Control: private, no-cache so the SPA's default
fetch cache mode always revalidates. An unchanged "navigate away and back"
becomes an empty 304 instead of a full rebuild + reserialize + transfer.
The DB query still runs (the cheap part); the win is skipping the DTO
build, serialization, and body bytes.

Proven end-to-end (throwaway Postgres + server, 7 images):
  1st GET (no If-None-Match)      -> 200  4586 bytes + ETag
  2nd GET (If-None-Match matches) -> 304     0 bytes
  3rd GET (If-None-Match stale)   -> 200  4586 bytes (correctly invalidated)
~655 B/photo, so a full 500-row first page saves ~320 KB + a 500-DTO
build/serialize per unchanged revalidation. Unlike a cold load this is the
common gallery-navigation path, so it hits real user-facing latency.

Regression test: tests/api/photos_etag.hurl (added to the api-test suite).
Methodology in benches/PHOTOS-ETAG.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 00:22:49 +02:00
Edouard Vanbelle 74b33744ed chore(test): disable storage_cleanup_check.sh until GC trigger lands 2026-06-19 16:06:38 +02:00
Edouard Vanbelle dd9e3b8868 feat(drive): test Tantivy lookup with drive
ensure that a user that don't havee permission to a drive cannot search elements in this drive
    note: current design: Index are associated to drive, so if a document / directory is shared,
    the shared resource will not be in index for targetted user
    design is explicitely as is to reduce complexity
2026-06-19 16:06:38 +02:00
Edouard Vanbelle e7f4826778 feat(drive): complete updated_by created_by 2026-06-19 16:06:38 +02:00
Edouard Vanbelle 16ea08b093 feat(drive): improve Drive model
now Drive is purely a metadata
    each drive has always a root folder
    this model minimize Oxicloud changes, and simplify
    the Drive name is simply the folder's root's name
    note: owner of Drive has more permission that an owner of the root folder
2026-06-19 16:06:37 +02:00
Edouard Vanbelle eab7a609b9 feat(drive): start implementation of drive
- add storage.drives
    - prepare migration phase
    - add created_by and updated_by on storage.folders
2026-06-19 16:06:37 +02:00
Edouard Vanbelle 72129af0bd refactor(role): use grant only
- remove permission centric mode
    - finalize migration drop all tables with permissions
    - ensure roles are ENUM (owner is always displayed first)
2026-06-18 02:47:06 +02:00
Edouard Vanbelle f168c4578f feat(roles): prepare migration ReBAC to roles
prepare migration of permission to roles
    this simplify drastically database (permission are now simply defined in code)
    and will permit reuse of the same ReBAC engine to define owners of drives

    mapping:

    ```
        Role::Viewer => &[Permission::Read],
        Role::Commenter => &[Permission::Read, Permission::Comment],
        Role::Contributor => &[Permission::Read, Permission::Create],
        Role::Editor => &[
            Permission::Read,
            Permission::Comment,
            Permission::Create,
            Permission::Update,
        ],
        Role::Owner => &[
            Permission::Read,
            Permission::Comment,
            Permission::Create,
            Permission::Update,
            Permission::Share,
            Permission::Delete,
            Permission::Manage,
        ],
    ```
2026-06-18 02:47:06 +02:00
Dionisio Pozo b8a0018785 Merge pull request #473 from EdouardVanbelle/fix/nextcloud+webdav
fix(nextcloud+webdav) fix bugs found via end to end tests
2026-06-17 12:42:31 +02:00
Edouard Vanbelle 619c24e1e9 test(api): unpin folder-share file fetch; assert in/out-of-scope
The KNOWN BUG pin for `GET /api/s/{folder-token}/file/{file_id}` was
stale — the route now returns 200 + body for files inside the share's
subtree and 404 for anything outside it. Replaces the sidestep
comment with two positive assertions:

  - in-share: 200 + Content-Disposition references the file name
  - out-of-share (caller-owned file in a different folder): 404,
    matching "no such file" so the response can't be used to
    enumerate foreign file ids

Coverage now exercises the actual recipient-side download path that
NC desktop and web clients use; the file-share variant (item_type=file)
moves down to test 8b.

Adds a teardown DELETE for the outsider hello.txt so the next test in
the runner (permissions.hurl) can re-upload its own hello.txt into
admin's home folder without hitting the live-name unique index.
2026-06-17 09:32:37 +02:00
Edouard Vanbelle 68abb891d7 feat(by-hash): allow /by-hash even if blob is trashed
- permit reuse of blob where file is trashed (by-hash and chunked)
    - add hurl test on /api/files/by-hash
    - add anti enumeration of blob (404 is always blob_not_owned_by_caller)
2026-06-16 22:41:55 +02:00
Edouard Vanbelle 6d65f7eb09 chore(test): add new API coverage
ensure better API test coverage on important routes

  1. tests/api/public_shares.hurl — create a share token, verify, list contents, fetch a file, fetch a folder zip, then revoke and re-verify with the token. Same pattern as grants.hurl. ~30 min, biggest security ROI.
  2. tests/api/auth_session_lifecycle.hurl — login → refresh → use new token → logout → refresh-rejected → login-again. Covers the session-family invalidation contract.
  3. tests/api/admin_user_ops.hurl — admin disables / re-enables / changes role / resets password / sets quota for a fixture user. Five POSTs.
  4. tests/api/groups_effective_members.hurl — nested groups: A contains B contains user X; effective-members returns X. Two scenarios, but it's the ReBAC contract under the Drive refactor.
  5. tests/api/search_basic.hurl — upload foo.txt, search "foo", get the result; cross-user: bob can't search alice's foo.
2026-06-13 19:03:07 +02:00
Edouard Vanbelle da65987e58 chore(test): retry storage check to prevent race condition
case: server on CI may be overloaded which can create race
    condition between file/bolb deletion andd check
2026-06-13 19:01:41 +02:00
Edouard Vanbelle ce69bbe151 feat(bin): add --version and --config parameters
--config is to prevent Oxicloud loading any other .env, usefully when you want to test with different environements
2026-06-13 19:01:41 +02:00
Edouard Vanbelle 140cbb440e chore(test): fix deprecated call or syntax 2026-06-13 19:01:40 +02:00
Edouard Vanbelle 595273277b test(e2e): webdav + nextcloud full e2e test coverage
add a full coverage of Webdav and Nextcloud
    purpose: prepare move to Drives and ensure no regression at all

    test scenarios are in docs/plan/BASELINE_TESTS_NC_WEBDAV.md

    current existing bugs identified via these tests:

      ┌──────────┬─────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┐
      │   Bug    │ Surface │                                            Pin location                                            │
      ├──────────┼─────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
      │ G4/G5/K5 │ NC      │ AlreadyExists → 500 instead of 412 (handle_move + trashbin restore)                                │
      ├──────────┼─────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
      │ G9       │ NC      │ Folder DELETE not row-recursive — orphan descendants stay live                                     │
      ├──────────┼─────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
      │ M5/M7    │ Native  │ resolve_path_for_user mismatch — PUT writes, GET reads via lenient lookup, MOVE/DELETE can't find  │
      │          │         │ via strict                                                                                         │
      ├──────────┼─────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
      │ M8       │ Native  │ COPY discards destination filename — collides with source                                          │
      ├──────────┼─────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
      │ N2       │ Native  │ LOCK creates the token, mutators don't check it — class-2 advertisement is aspirational            │
      └──────────┴─────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘
2026-06-13 19:01:40 +02:00
Edouard Vanbelle 50ea406719 feat(OXICLOUD_DIRECT_PUT_MAX_BYTES): add a security limit on direct PUT
ensure files does not exeed OXICLOUD_MAX_UPLOAD_SIZE, prefer to deny from header rather consuming bandwidth
    add OXICLOUD_DIRECT_PUT_MAX_BYTES for direct PUT (non chunked), admins can fine tune their prefered values
2026-06-09 11:06:44 +02:00
Edouard Vanbelle 964d66e7f8 featchukn-upload): client can provide full file hash completion 2026-06-09 09:55:27 +02:00
Edouard Vanbelle 41aad26702 feat(upload): cover chunk upload + add support of different digest hash
Prefer stream storage rather using buffered (in memory)

  note: on many unix like tmpfs are in-memory, sungle PUT are sized limited

  Storage map (NC stands for Nextcloud gateway)

  ┌───────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────┐
  │                   Streaming surface                   │                            Destination                             │                Configurable via                 │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST chunked PUT /api/uploads/{id} chunk              │ {storage_path}/.uploads/{upload_id}/chunk_{NNNNNN}                 │ OXICLOUD_STORAGE_PATH (the .uploads subdir is   │
  │                                                       │                                                                    │ hard-wired)                                     │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST chunked assemble (during /complete)              │ {storage_path}/.uploads/{upload_id}/assembled                      │ same                                            │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ NC chunked PUT /dav/uploads/.../{chunk}               │ {storage_path}/.uploads/nextcloud/{user}/{upload_id}/{chunk_name}  │ same                                            │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ NC chunked assemble (during MOVE)                     │ {storage_path}/.uploads/nextcloud/{user}/{upload_id}/.assembled    │ same                                            │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ NC single-file PUT /dav/files/.../{path} (via         │ OXICLOUD_UPLOAD_TMPDIR if set, else OS default temp (/tmp on       │ OXICLOUD_UPLOAD_TMPDIR                          │
  │ spool_body_to_temp)                                   │ Linux)                                                             │                                                 │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST WebDAV PUT /webdav/{path} (via                   │ same as above                                                      │ OXICLOUD_UPLOAD_TMPDIR                          │
  │ spool_body_to_temp)                                   │                                                                    │                                                 │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST multipart upload /api/files/upload               │ {storage_path}/.dedup_temp/upload-{uuid}                           │ OXICLOUD_STORAGE_PATH (hard-wired subdir)       │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ WOPI PutFile                                          │ OS default temp via NamedTempFile::new() (no override)             │ (none — bug worth tracking)                     │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ Final blob storage (after fsync + rename)             │ {storage_path}/.blobs/{ab}/{abc…}.blob                             │ OXICLOUD_STORAGE_PATH                           │
  └───────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────┘

  one caveat: a malicious user can create many chunked upload and saturate local storage
2026-06-09 09:52:09 +02:00
Edouard Vanbelle 5e638691ad security(upload): cap upload size to prevent memody/disk consumption
add OXICLOUD_CHUNK_MAX_BYTES which correspond to the max upload chunk allowed
    (differs from OXICLOUD_MAX_UPLOAD_SIZE which is the max total size of a file)
    hurl test validate the change

    Streams the request body straight to the chunk file with peak heap of
    ~one HTTP frame, regardless of chunk size or the configured cap. The
    `storage.chunk_max_bytes` config (env `OXICLOUD_CHUNK_MAX_BYTES`,
    default 100 MB) bounds a single PUT — separate from `max_upload_size`
    which governs whole-file uploads. Without this separation, a client
    could submit a chunk up to the whole-file cap (10 GB default) and
    monopolise server memory.
2026-06-09 09:26:33 +02:00
Edouard Vanbelle 5a18ec4bac test(hash_content): check that hash matches 2026-06-06 18:49:18 +02:00
Edouard Vanbelle 8cc21f17c5 feat(notify): add notif to internal users when granted
- add coalesced protection to avoid mail bombing if an invited goes many grant in a short period
    - add resentd method in share menu item (work for both internal and external users)
    - user can disable email notification via his properties
    - add env variable from admin to disable notifications
2026-06-05 11:25:06 +02:00
Edouard Vanbelle a9a2660576 feat(user edition): permit user without username to define one
- permit also edition of given_name & family_name
    - once username has been define it become immuable (due to Nextcloud implementation)
2026-06-03 00:35:25 +02:00
Edouard Vanbelle 6aba7cbbbf feat(email_verified): store email verification on a user 2026-06-03 00:35:25 +02:00
Edouard Vanbelle 8fc9a50681 feat(passwordless): add cookie challenge + low TTL
magic-link as now 2 modes:

        - invitation: long TTL (24), no challenge
        - passwordless login: short TTL (10min), cookie challenge to ensure that
        user goes back to same browser (no man in the middle capturing email)
2026-06-03 00:35:25 +02:00
Edouard Vanbelle 00af0e8a89 feat(registraton): add anti enumeration (cannot know if an account already exists)
Important: anti-enumeration is active only if SMTP is defined, welcome email can be used
    otherwise it is a classic registration with ok or conflic if account alrady exists
2026-06-03 00:35:25 +02:00
Edouard Vanbelle 9a49ab44d8 feat(passwordless): pass3: passwordless account (via emailed magic-link)
Backend
  - RegisterDto — username and password both become Option<String> with #[serde(default)] so JSON can omit them entirely.
  - AuthApplicationService::register — username uniqueness check skipped when None (multiple NULLs OK under the UNIQUE index); password hashing skipped when None; User::new called with the actual Options instead of forcing Some(...).
  - auth_handler::register — branches on dto.password.is_none(). With password → existing 201 + UserDto. Without → triggers MagicLinkInviteService::send_login_link(&email) best-effort, then returns 200 + {"message": "Check your email…"}. The
  OIDC-mode-disables-password-registration gate now only fires for the password path (email-only signup is still allowed even in OIDC-only mode, because it doesn't store a password).
  - magic_link_handler::redirect_target — new 3-way decision tree:
    - Resource target (folder invitation) → /#/files/folder/{id} (existing)
    - NULL resource + is_external = false → /#/files (the welcome path for new internal users — they have a home folder)
    - NULL resource + is_external = true → /#/sharedwithme (the existing external-user landing)

  Tests
  - New tests/api/registration.hurl with 9 requests covering: classic (with-password) register → 201 + UserDto, email-only register → 200 + uniform message + welcome magic-link captured, redemption → 302 to /#/files + cookies set, profile read → username
  absent + is_external: false, resend magic-link works (eligible while passwordless), cleanup deletes both new users.
  - Wired into tests/api/run.sh right after auth_login.hurl.

  Plan additions
  - auth-simplification.md gained PR 22 at the bottom of the PR sequence — device-bound magic-link redemption via challenge cookie + asymmetric TTLs (login: 10 min, invitation: 24 h). Full design recap, schema migration, config knobs
  (OXICLOUD_MAGIC_LINK_LOGIN_TTL_MINUTES / _INVITE_TTL_HOURS), and Hurl coverage outline are in the plan. Slots in before PR 21's docs so the architecture page describes the final state from the start.

  Checks — cargo fmt, cargo clippy --all-features --all-targets -- -D warnings, cargo test --lib (297 passed), biome, stylelint, tsc, full Hurl suite (16 files) all green.
2026-06-03 00:35:25 +02:00
Edouard Vanbelle ac24a0eda1 feat(username|email): pass2: accept login via email orusername
- login via (username or email) + password
    - hurl test to cover the feature
2026-06-03 00:35:25 +02:00
Edouard Vanbelle d57a50d056 feat(username|email): pass1: normalize auth.user data
username: now optional, if defined 2..64 chars
    password: now optional (no mode __NO_PASSWORD...__)
    oidc: now optional

    important: if need Nextcloud, username must be defined
2026-06-03 00:35:25 +02:00