41aad26702
feat(upload): cover chunk upload + add support of different digest hash
...
Prefer stream storage rather using buffered (in memory)
note: on many unix like tmpfs are in-memory, sungle PUT are sized limited
Storage map (NC stands for Nextcloud gateway)
┌───────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────┐
│ Streaming surface │ Destination │ Configurable via │
├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
│ REST chunked PUT /api/uploads/{id} chunk │ {storage_path}/.uploads/{upload_id}/chunk_{NNNNNN} │ OXICLOUD_STORAGE_PATH (the .uploads subdir is │
│ │ │ hard-wired) │
├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
│ REST chunked assemble (during /complete) │ {storage_path}/.uploads/{upload_id}/assembled │ same │
├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
│ NC chunked PUT /dav/uploads/.../{chunk} │ {storage_path}/.uploads/nextcloud/{user}/{upload_id}/{chunk_name} │ same │
├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
│ NC chunked assemble (during MOVE) │ {storage_path}/.uploads/nextcloud/{user}/{upload_id}/.assembled │ same │
├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
│ NC single-file PUT /dav/files/.../{path} (via │ OXICLOUD_UPLOAD_TMPDIR if set, else OS default temp (/tmp on │ OXICLOUD_UPLOAD_TMPDIR │
│ spool_body_to_temp) │ Linux) │ │
├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
│ REST WebDAV PUT /webdav/{path} (via │ same as above │ OXICLOUD_UPLOAD_TMPDIR │
│ spool_body_to_temp) │ │ │
├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
│ REST multipart upload /api/files/upload │ {storage_path}/.dedup_temp/upload-{uuid} │ OXICLOUD_STORAGE_PATH (hard-wired subdir) │
├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
│ WOPI PutFile │ OS default temp via NamedTempFile::new() (no override) │ (none — bug worth tracking) │
├───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────┤
│ Final blob storage (after fsync + rename) │ {storage_path}/.blobs/{ab}/{abc…}.blob │ OXICLOUD_STORAGE_PATH │
└───────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────┘
one caveat: a malicious user can create many chunked upload and saturate local storage
2026-06-09 09:52:09 +02:00
3fc0a272ca
fix(nextcloud): resume (PROPFIND) on upload chunk
...
PROPFIND on upload chunk is necessary for application to resume the upload
it is used at least by Nextcloud Android app
issue has been raised via #415
unable to test, so I am using a playbook in test/webdav to simulate a
PROPIND on remote.php/dav/uploads/{user}/{session}
2026-06-07 20:47:16 +02:00
5a18ec4bac
test(hash_content): check that hash matches
2026-06-06 18:49:18 +02:00
3362e277ab
feat(authz): check permission on read handlers + check create permission on folder
2026-05-21 20:30:53 +02:00
739235faf0
test: check dedup of large files (requiring chunked) + test checunk upload via API
2026-05-13 16:42:28 +02:00
28e25f9d16
fix+test: check hash ref count on copy-on-write (a duplicated beeing updated)
2026-05-13 15:55:41 +02:00
f7946028f4
tests: add webdav tests
2026-05-13 15:55:34 +02:00
Edouard Vanbelle