Commit Graph

264 Commits

Author SHA1 Message Date
Edouard Vanbelle 41aad26702 feat(upload): cover chunk upload + add support of different digest hash
Prefer stream storage rather using buffered (in memory)

  note: on many unix like tmpfs are in-memory, sungle PUT are sized limited

  Storage map (NC stands for Nextcloud gateway)

  ┌───────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────┐
  │                   Streaming surface                   │                            Destination                             │                Configurable via                 │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST chunked PUT /api/uploads/{id} chunk              │ {storage_path}/.uploads/{upload_id}/chunk_{NNNNNN}                 │ OXICLOUD_STORAGE_PATH (the .uploads subdir is   │
  │                                                       │                                                                    │ hard-wired)                                     │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST chunked assemble (during /complete)              │ {storage_path}/.uploads/{upload_id}/assembled                      │ same                                            │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ NC chunked PUT /dav/uploads/.../{chunk}               │ {storage_path}/.uploads/nextcloud/{user}/{upload_id}/{chunk_name}  │ same                                            │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ NC chunked assemble (during MOVE)                     │ {storage_path}/.uploads/nextcloud/{user}/{upload_id}/.assembled    │ same                                            │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ NC single-file PUT /dav/files/.../{path} (via         │ OXICLOUD_UPLOAD_TMPDIR if set, else OS default temp (/tmp on       │ OXICLOUD_UPLOAD_TMPDIR                          │
  │ spool_body_to_temp)                                   │ Linux)                                                             │                                                 │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST WebDAV PUT /webdav/{path} (via                   │ same as above                                                      │ OXICLOUD_UPLOAD_TMPDIR                          │
  │ spool_body_to_temp)                                   │                                                                    │                                                 │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ REST multipart upload /api/files/upload               │ {storage_path}/.dedup_temp/upload-{uuid}                           │ OXICLOUD_STORAGE_PATH (hard-wired subdir)       │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ WOPI PutFile                                          │ OS default temp via NamedTempFile::new() (no override)             │ (none — bug worth tracking)                     │
  ├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
  │ Final blob storage (after fsync + rename)             │ {storage_path}/.blobs/{ab}/{abc…}.blob                             │ OXICLOUD_STORAGE_PATH                           │
  └───────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────┘

  one caveat: a malicious user can create many chunked upload and saturate local storage
2026-06-09 09:52:09 +02:00
Edouard Vanbelle 5e638691ad security(upload): cap upload size to prevent memody/disk consumption
add OXICLOUD_CHUNK_MAX_BYTES which correspond to the max upload chunk allowed
    (differs from OXICLOUD_MAX_UPLOAD_SIZE which is the max total size of a file)
    hurl test validate the change

    Streams the request body straight to the chunk file with peak heap of
    ~one HTTP frame, regardless of chunk size or the configured cap. The
    `storage.chunk_max_bytes` config (env `OXICLOUD_CHUNK_MAX_BYTES`,
    default 100 MB) bounds a single PUT — separate from `max_upload_size`
    which governs whole-file uploads. Without this separation, a client
    could submit a chunk up to the whole-file cap (10 GB default) and
    monopolise server memory.
2026-06-09 09:26:33 +02:00
Edouard Vanbelle a3a8fdbf12 test: activate nextcloud on tests 2026-06-07 21:40:22 +02:00
Edouard Vanbelle 3fc0a272ca fix(nextcloud): resume (PROPFIND) on upload chunk
PROPFIND on upload chunk is necessary for application to resume the upload
    it is used at least by Nextcloud Android app

    issue has been raised via #415

    unable to test, so I am using a playbook in test/webdav to simulate a
    PROPIND on remote.php/dav/uploads/{user}/{session}
2026-06-07 20:47:16 +02:00
EdouardVanbelle b25d1593d3 test(e2e): update playwright snapshots
- due to change one91d4dbab9f644feaa6216bb6bd697156272f07e
2026-06-07 19:46:42 +02:00
Edouard Vanbelle 5a18ec4bac test(hash_content): check that hash matches 2026-06-06 18:49:18 +02:00
Edouard Vanbelle 8cc21f17c5 feat(notify): add notif to internal users when granted
- add coalesced protection to avoid mail bombing if an invited goes many grant in a short period
    - add resentd method in share menu item (work for both internal and external users)
    - user can disable email notification via his properties
    - add env variable from admin to disable notifications
2026-06-05 11:25:06 +02:00
Edouard Vanbelle 45ba118c35 test(front): update screenshots 2026-06-03 00:35:25 +02:00
Edouard Vanbelle a9a2660576 feat(user edition): permit user without username to define one
- permit also edition of given_name & family_name
    - once username has been define it become immuable (due to Nextcloud implementation)
2026-06-03 00:35:25 +02:00
Edouard Vanbelle 6aba7cbbbf feat(email_verified): store email verification on a user 2026-06-03 00:35:25 +02:00
Edouard Vanbelle 8fc9a50681 feat(passwordless): add cookie challenge + low TTL
magic-link as now 2 modes:

        - invitation: long TTL (24), no challenge
        - passwordless login: short TTL (10min), cookie challenge to ensure that
        user goes back to same browser (no man in the middle capturing email)
2026-06-03 00:35:25 +02:00
Edouard Vanbelle 00af0e8a89 feat(registraton): add anti enumeration (cannot know if an account already exists)
Important: anti-enumeration is active only if SMTP is defined, welcome email can be used
    otherwise it is a classic registration with ok or conflic if account alrady exists
2026-06-03 00:35:25 +02:00
Edouard Vanbelle 9a49ab44d8 feat(passwordless): pass3: passwordless account (via emailed magic-link)
Backend
  - RegisterDto — username and password both become Option<String> with #[serde(default)] so JSON can omit them entirely.
  - AuthApplicationService::register — username uniqueness check skipped when None (multiple NULLs OK under the UNIQUE index); password hashing skipped when None; User::new called with the actual Options instead of forcing Some(...).
  - auth_handler::register — branches on dto.password.is_none(). With password → existing 201 + UserDto. Without → triggers MagicLinkInviteService::send_login_link(&email) best-effort, then returns 200 + {"message": "Check your email…"}. The
  OIDC-mode-disables-password-registration gate now only fires for the password path (email-only signup is still allowed even in OIDC-only mode, because it doesn't store a password).
  - magic_link_handler::redirect_target — new 3-way decision tree:
    - Resource target (folder invitation) → /#/files/folder/{id} (existing)
    - NULL resource + is_external = false → /#/files (the welcome path for new internal users — they have a home folder)
    - NULL resource + is_external = true → /#/sharedwithme (the existing external-user landing)

  Tests
  - New tests/api/registration.hurl with 9 requests covering: classic (with-password) register → 201 + UserDto, email-only register → 200 + uniform message + welcome magic-link captured, redemption → 302 to /#/files + cookies set, profile read → username
  absent + is_external: false, resend magic-link works (eligible while passwordless), cleanup deletes both new users.
  - Wired into tests/api/run.sh right after auth_login.hurl.

  Plan additions
  - auth-simplification.md gained PR 22 at the bottom of the PR sequence — device-bound magic-link redemption via challenge cookie + asymmetric TTLs (login: 10 min, invitation: 24 h). Full design recap, schema migration, config knobs
  (OXICLOUD_MAGIC_LINK_LOGIN_TTL_MINUTES / _INVITE_TTL_HOURS), and Hurl coverage outline are in the plan. Slots in before PR 21's docs so the architecture page describes the final state from the start.

  Checks — cargo fmt, cargo clippy --all-features --all-targets -- -D warnings, cargo test --lib (297 passed), biome, stylelint, tsc, full Hurl suite (16 files) all green.
2026-06-03 00:35:25 +02:00
Edouard Vanbelle ac24a0eda1 feat(username|email): pass2: accept login via email orusername
- login via (username or email) + password
    - hurl test to cover the feature
2026-06-03 00:35:25 +02:00
Edouard Vanbelle d57a50d056 feat(username|email): pass1: normalize auth.user data
username: now optional, if defined 2..64 chars
    password: now optional (no mode __NO_PASSWORD...__)
    oidc: now optional

    important: if need Nextcloud, username must be defined
2026-06-03 00:35:25 +02:00
EdouardVanbelle 54c1989b44 test(e2e): update playwright linux snapshots 2026-06-03 00:32:00 +02:00
Edouard Vanbelle f4362a4832 test(end to end): update tests (login page has now login via email) 2026-06-03 00:32:00 +02:00
Edouard Vanbelle feaa066303 test(integration-test): fix issue with server start 2026-06-03 00:32:00 +02:00
Edouard Vanbelle 6d70a7000e security(external_user): protect unnecessary route access to external users 2026-06-03 00:32:00 +02:00
Edouard Vanbelle 21c06da700 feat(magic-links): add rate limiting + archirecture documentation 2026-06-03 00:32:00 +02:00
Edouard Vanbelle 64d081ad0b feat(external): permit login via email (magic link) 2026-06-03 00:31:59 +02:00
Edouard Vanbelle 6763f2ca9e fix(/api/users): external users can only query themself and their granters 2026-06-03 00:31:59 +02:00
Edouard Vanbelle ec72374651 feat(api): can grant external user (via email)
- add possibility to grant an external user.
    - route /api/users/{id} added (rate limited for security)
    - security: start route limitation for external users
        ex: they must not browse /api/users/{id} nor addressbook
2026-06-03 00:31:59 +02:00
Edouard Vanbelle 03f63ad103 feat(external users): email sanity + mock SMTP
- SMTP has a mock to enable end to end test and validate the whole path
     (via OXICLOUD_SMTP_MOCK)
    - add email normalisation ( including punicode)
    - api to share to external user
2026-06-03 00:31:59 +02:00
Edouard Vanbelle d2dedcbb00 fix(integration-test): ensure integration tests are runned on a separate DB to avoid polution 2026-05-31 23:53:40 +02:00
Edouard Vanbelle 96097431c3 test(api): upgrade tests to new routes (don't use deprecated routes anymore) 2026-05-31 20:58:01 +02:00
Edouard Vanbelle 09985f8a95 feat(group): 1st implementation of Groups
this implements first version (manageable only by admin right now)

    routes:

        GET /api/groups
        List subject groups (paginated). Admin-only.

        POST /api/groups
        Create a new ReBAC subject group. Admin-only. The name must match the RFC 5321 local-part shape and be globally unique (case-insensitive).

        GET /api/groups/search
        Search non-virtual groups by name substring. Authenticated only (no admin role required) — backs the share-dialog recipient autocomplete.

        GET /api/groups/{id}
        Fetch a single group's details. Admin-only.

        DELETE /api/groups/{id}
        Delete a group. Cascades to `subject_group_members` (FK) and to `access_grants` rows referencing this group as a subject. Admin-only.

        PATCH /api/groups/{id}
        Update a group's metadata. Admin-only. v1 only persists name renames.

        GET /api/groups/{id}/effective-members
        List every user transitively reached through this group (members of members of members, etc.). Used by admin / audit tooling. Admin-only.

        GET /api/groups/{id}/members
        List the *direct* members of a group (one level only). Admin-only.

        POST /api/groups/{id}/members
        Add a member to a group. Exactly one of `user_id` / `group_id` must be provided. Adding a group-member runs a write-time cycle check and a nesting-depth check (max 8). Admin-only.

        DELETE /api/groups/{id}/members/group/{gid}
        Remove a nested group-member from a group. Admin-only.

        DELETE /api/groups/{id}/members/user/{uid}
        Remove a user-member from a group. Admin-only.

fix hurl

groups

round

groups
2026-05-31 20:57:45 +02:00
Edouard Vanbelle ca1a2bb649 feat(light-dark): normalize light/dark/like-os mode
permits user to define light, dark or like the desktop
2026-05-30 09:36:15 +02:00
Edouard Vanbelle ea83891a61 feat(trash): move trash API to normalized version (with cursor, orderBy) + normalize Trash section to existing components
normalize also component to format badges (expiry, role, etc)
2026-05-30 00:49:32 +02:00
Edouard Vanbelle 5a43b14e29 test(api): move deprecated call to new one 2026-05-29 13:13:23 +02:00
Edouard Vanbelle ec43c4f9c9 fix(playwright): raise any pageerror, purpose is to stops immediatly on a bundle issue 2026-05-28 01:54:48 +02:00
Edouard Vanbelle 7a849be3d8 chore(playwright): upgrade to version 1.60.0 2026-05-27 11:50:02 +02:00
EdouardVanbelle 2dc2b0876d test(e2e): update playwright linux snapshots 2026-05-27 09:48:25 +00:00
Edouard Vanbelle 7ddbbc94cc test(e2e): update all images and grow snapshot comparaison sensitivity 2026-05-27 11:35:42 +02:00
Edouard Vanbelle 891ff89a5d test(playwright): add playwright report to track changes + correct playwright parameters
ci: fix actions/checkout@v6 → @v4 (v6 does not exist)

GitHub started rejecting invalid action versions at parse time,
causing every CI run to fail with 0s / 'workflow file issue'
before any jobs could start.

 * test(e2e): only no retry
 * test(e2e): use static name in tests to facilitate screenshot comparaison
 * test(e2e): update linux screenshots
 * test(e2e): adapt test due to user-vignette generation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 11:29:58 +02:00
Edouard Vanbelle 12466d4b83 feat(roles): simplify roles to only Viewer, Editor, Admin
┌────────┬──────┬─────────┬────────┬────────┬───────┬────────┐
  │  Role  │ read │ comment │ create │ update │ share │ delete │
  ├────────┼──────┼─────────┼────────┼────────┼───────┼────────┤
  │ viewer │  ✓   │         │        │        │       │        │
  ├────────┼──────┼─────────┼────────┼────────┼───────┼────────┤
  │ editor │  ✓   │    ✓    │   ✓    │   ✓    │       │        │
  ├────────┼──────┼─────────┼────────┼────────┼───────┼────────┤
  │ admin  │  ✓   │    ✓    │   ✓    │   ✓    │   ✓   │   ✓    │
  └────────┴──────┴─────────┴────────┴────────┴───────┴────────┘
2026-05-25 22:47:36 +02:00
Edouard Vanbelle 1dbd56e9cc tests(load): prepare a load repository to test response time under load in the future 2026-05-25 22:47:01 +02:00
Edouard Vanbelle dd68d783e0 fix(authz): permit policiy: a user with Delete permission can delete a file/folder. Only the owner can permanently delete or restore a trashed item 2026-05-21 22:45:49 +02:00
Edouard Vanbelle a1c21ce446 refactor(authz): permet require_permission() as has_permission(), more explicit 2026-05-21 21:50:42 +02:00
Edouard Vanbelle cb35775f77 fix(dedub): correct ref count on hashes, many thanks to you api tests... 2026-05-21 21:12:38 +02:00
Edouard Vanbelle eb95567a7d feat(authz): test & cover batch cases
┌────────────────────────────────┬─────────────────────────────────┬───────────────────────┬─────────────────┬──────────────────────────────┐
  │            Endpoint            │        Phase 3A no-grant        │    Phase 3B Viewer    │ Phase 3C Editor │        Phase 3D Admin        │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ POST /api/batch/files/get      │ 400 (all failed)                │ 200 (2 successful)    │ —               │ —                            │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ POST /api/batch/files/move     │ 400                             │ 400 (no Update)       │ 200             │ —                            │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ POST /api/batch/files/copy     │ 400                             │ —                     │ 200             │ —                            │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ POST /api/batch/files/delete   │ 400                             │ 400                   │ 400 (no Delete) │ —                            │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ POST /api/batch/folders/get    │ 400                             │ 200                   │ —               │ —                            │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ POST /api/batch/folders/create │ 400                             │ —                     │ 201             │ —                            │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ POST /api/batch/folders/move   │ 400                             │ —                     │ 200             │ —                            │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ POST /api/batch/folders/copy   │ 400                             │ —                     │ 200             │ —                            │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ POST /api/batch/folders/delete │ 400                             │ —                     │ 400 (no Delete) │ 200                          │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ POST /api/batch/trash          │ 400                             │ —                     │ —               │ 400 (owner-only, documented) │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ POST /api/batch/download       │ 404 (NotFound)                  │ 200 + application/zip │ —               │ —                            │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ GET /api/batch/download?...    │ 404                             │ 200 + zip             │ —               │ —                            │
  ├────────────────────────────────┼─────────────────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────┤
  │ Phase 3E lifecycle cleanup     │ grants table empty after delete │                       │                 │                              │
  └────────────────────────────────┴─────────────────────────────────┴───────────────────────┴─────────────────┴──────────────────────────────┘
2026-05-21 20:30:58 +02:00
Edouard Vanbelle a53c09f361 feat(authz): covert and test chunked upload with permissions 2026-05-21 20:30:58 +02:00
Edouard Vanbelle bd1b17b589 test(grants): full coverate of /api/files and /api/folders 2026-05-21 20:30:58 +02:00
Edouard Vanbelle 3362e277ab feat(authz): check permission on read handlers + check create permission on folder 2026-05-21 20:30:53 +02:00
Edouard Vanbelle cba9be8c21 feat(rebac): first pass 2026-05-20 22:56:00 +02:00
Edouard Vanbelle ac42a6d3cc test(api): check right management for folder creation and folder move + check also webdsav MKCOL protection
│ Steps 1-6 │ Setup: admin's resources + create bob + bob's home folder                                                           │
  │ Step 7    │ REST: bob can't create a folder inside admin's home → 404                                                           │
  │ Step 8    │ REST: bob can't create inside admin's private folder → 404                                                          │
  │ Step 9    │ REST: parent_id: null auto-resolves to bob's home (documents the convenience)                                       │
  │ Step 10   │ REST: positive control — bob creates in his own home → 201                                                          │
  │ Step 12   │ REST: bob can't move his file into admin's folder → 404                                                             │
  │ Step 13   │ REST: bob moves file to root (null) → 200 (legitimate root state)                                                   │
  │ Step 14   │ REST: bob can't read admin's file → 404                                                                             │
  │ Step 15   │ REST: admin's tree integrity preserved                                                                              │
  │ Step 16   │ WebDAV: path-prefix isolation rewrites cross-user paths into caller's tree                                          │
  │ Step 17   │ WebDAV: positive control MKCOL in bob's own tree → 201                                                              │
  │ Step 18   │ WebDAV: bob's home contains the rewritten "My Folder - admin" sub-folder, proving the isolation rerouted the attack │
  │ Step 19   │ WebDAV: admin's tree never sees bob's WebDAV traffic                                                                │
2026-05-20 13:04:15 +02:00
Edouard Vanbelle 872abd4d27 test: add front end folder management test (creation, renaming, etc)
note: firefox test remove due to idempotence issue
2026-05-19 13:31:20 +02:00
Edouard Vanbelle edbfe2848c fix(front): fix issue with already declared constants on release mode
- ensure not loading twice the same variables, namespace to ensure it in the future
 - end to end tests where successfull, need to check it is not reusing a previous release (in cache from build CI)
 - fix: #377 #378
2026-05-19 09:36:24 +02:00
Edouard Vanbelle 739235faf0 test: check dedup of large files (requiring chunked) + test checunk upload via API 2026-05-13 16:42:28 +02:00
Edouard Vanbelle 28e25f9d16 fix+test: check hash ref count on copy-on-write (a duplicated beeing updated) 2026-05-13 15:55:41 +02:00