diff --git a/docs/config/env.md b/docs/config/env.md index 62b50f9b..f131ccd4 100644 --- a/docs/config/env.md +++ b/docs/config/env.md @@ -11,8 +11,9 @@ Most runtime variables use the `OXICLOUD_` prefix. A few build-time or allocator | `OXICLOUD_SERVER_PORT` | `8086` | Server port | | `OXICLOUD_SERVER_HOST` | `127.0.0.1` | Server bind address (IPv4 or IPv6 allowed) | | `OXICLOUD_BASE_URL` | (auto) | Public base URL for share links; defaults to `http://{host}:{port}` | -| `OXICLOUD_MAX_UPLOAD_SIZE` | `10737418240` | Maximum upload size in bytes (10 GB on 64-bit, 1 GB on 32-bit) | -| `OXICLOUD_CHUNK_MAX_BYTES` | `104857600` | Maximum size of a single chunked-upload PUT in bytes (100 MB). Distinct from `OXICLOUD_MAX_UPLOAD_SIZE`, which is the whole-file cap. See [Storage Fine Tuning](./storage-fine-tuning.md). | +| `OXICLOUD_MAX_UPLOAD_SIZE` | `10737418240` | Whole-file size ceiling, in bytes (10 GB on 64-bit, 1 GB on 32-bit). Applies to BOTH direct PUTs (per-request body) and chunked uploads (declared `total_size`, checked upfront at session creation). | +| `OXICLOUD_DIRECT_PUT_MAX_BYTES` | `1073741824` | Per-request cap for non-chunked PUT bodies, in bytes (1 GiB). Set below `OXICLOUD_MAX_UPLOAD_SIZE` so larger files are pushed onto the chunked protocol (resumable on failure). See [Storage Fine Tuning](./storage-fine-tuning.md). | +| `OXICLOUD_CHUNK_MAX_BYTES` | `104857600` | Maximum size of a single chunked-upload PUT in bytes (100 MB). Per-chunk cap, independent of `OXICLOUD_MAX_UPLOAD_SIZE` (whole-file cap). See [Storage Fine Tuning](./storage-fine-tuning.md). | | `OXICLOUD_UPLOAD_TMPDIR` | (OS temp dir) | Spool directory for non-chunked PUT bodies (`/api/files/upload`, WebDAV PUT). Point at a real-disk path on the same FS as `.blobs/` to avoid tmpfs OOMKill and make blob promotion an atomic rename. See [Storage Fine Tuning](./storage-fine-tuning.md). | | `OXICLOUD_CHUNK_DIR` | `{STORAGE_PATH}/.uploads` | Root directory for chunked-upload sessions (REST + NextCloud). Same-FS / NVMe placement guidance: see [Storage Fine Tuning](./storage-fine-tuning.md). | | `OXICLOUD_REUSE_PORT` | `false` | Enable `SO_REUSEPORT` so multiple processes can share the same port. **Disabled by default** — a second accidental instance will fail with "address already in use". Enable only for deliberate multi-worker setups (process supervisor, rolling restart). Not supported on Windows. | diff --git a/docs/config/storage-fine-tuning.md b/docs/config/storage-fine-tuning.md index a42d029d..ea51d3e1 100644 --- a/docs/config/storage-fine-tuning.md +++ b/docs/config/storage-fine-tuning.md @@ -50,60 +50,85 @@ Two practical consequences: Three independent caps control how large an upload OxiCloud will accept. Pick them with disk and tmpfs sizing in mind: the spool/chunk -directories must be able to hold the worst case (max cap × concurrent +directories must be able to hold the worst case (cap × concurrent uploads). -| Variable | Default | What it caps | -|---|---|---| -| `OXICLOUD_MAX_UPLOAD_SIZE` | 10 GB | Total file size — whether uploaded as one `PUT` or assembled from many chunks. The hard ceiling on any single file. | -| `OXICLOUD_CHUNK_MAX_BYTES` | 100 MB | A single chunked-PUT request body (`PATCH /api/uploads/{id}` or `PUT /dav/uploads/.../chunk`). Independent of the whole-file cap — a 5 GB file uploaded in 100 MB chunks needs ~50 PUTs, each bounded by this. | -| `OXICLOUD_MAX_UPLOAD_SIZE` (again, for the PUT path) | 10 GB | A single non-chunked PUT body. Same env var as the whole-file cap because for non-chunked uploads they're equivalent. | +| Variable | Default | What it caps | When it fires | +|---|---|---|---| +| `OXICLOUD_MAX_UPLOAD_SIZE` | 10 GB | **Whole-file ceiling.** Applies to both direct PUT (per-body) and chunked uploads (declared `total_size`). The absolute upper bound on any single file in OxiCloud. | Chunked: at `POST /api/uploads` against the JSON-declared `total_size`, before any chunk is uploaded. Direct PUT: indirectly via `OXICLOUD_DIRECT_PUT_MAX_BYTES`, which is expected to be ≤ `OXICLOUD_MAX_UPLOAD_SIZE`. | +| `OXICLOUD_DIRECT_PUT_MAX_BYTES` | 1 GiB | **Non-chunked PUT body.** Per-request cap for `POST /api/files/upload`, `PUT /webdav/...`, and `PUT /remote.php/dav/files/.../...`. Set below `OXICLOUD_MAX_UPLOAD_SIZE` so larger files are pushed onto the chunked protocol — which is resumable on failure. | During body streaming, as a per-frame accumulator. Excess → 413 with a "use chunked upload" hint. | +| `OXICLOUD_CHUNK_MAX_BYTES` | 100 MB | **Per-chunk body** in a chunked-upload session (`PATCH /api/uploads/{id}` or `PUT /remote.php/dav/uploads/.../chunk`). Independent of the whole-file cap — a 5 GB file in 100 MB chunks is 50 PATCHes each bounded by this. | During chunk-body streaming. Excess → 413. | -### Why these matter for tmpfs sizing +### Recommendation: prefer chunked uploads for large files + +The defaults (`OXICLOUD_DIRECT_PUT_MAX_BYTES` = 1 GiB, well below +`OXICLOUD_MAX_UPLOAD_SIZE` = 10 GB) are deliberately asymmetric. +Files between those two caps can only succeed via the chunked +protocol. Three reasons to keep them that way: + +- **Resilience.** A direct PUT at 95 % of 5 GB that drops loses + everything. The same drop on a chunked upload loses one ~5 MB + chunk; the client retries that chunk and continues. +- **Memory + disk pressure.** Direct PUT spools the full body to + disk per request. Ten concurrent 5 GB direct PUTs use up to 50 GB + of transient spool disk. Chunked spreads each upload across many + small PATCHes; per-request resource use stays bounded by + `OXICLOUD_CHUNK_MAX_BYTES`. +- **Convention.** NextCloud desktop and the OxiCloud web UI already + switch to chunked at ~10 MB (`CHUNKED_UPLOAD_THRESHOLD`). + +### Why caps matter for tmpfs sizing OxiCloud streams bodies frame-by-frame, so **RAM** is bounded to one HTTP frame (~64 KB) per request regardless of the caps. **Disk space**, however, scales with the caps: -- **Direct PUT** (single-file): each in-flight upload spools the full - body to disk under `OXICLOUD_UPLOAD_TMPDIR` until promotion. Worst - case disk = `OXICLOUD_MAX_UPLOAD_SIZE × concurrent_PUTs`. -- **Chunked upload**: each in-flight session accumulates chunks under - `OXICLOUD_CHUNK_DIR`, then assembles them into a single temp file - before promotion. Worst case disk per session = **2 × file_size** - (chunks + assembled file); total disk = - `2 × OXICLOUD_MAX_UPLOAD_SIZE × concurrent_sessions`. +- **Direct PUT**: each in-flight upload spools the full body to disk + under `OXICLOUD_UPLOAD_TMPDIR` until promotion. Worst case disk = + `OXICLOUD_DIRECT_PUT_MAX_BYTES × concurrent_direct_PUTs`. +- **Chunked upload**: each in-flight session accumulates chunks + under `OXICLOUD_CHUNK_DIR`, then assembles them into a single temp + file before promotion. Worst case disk per session = **2 × + file_size** (chunks + assembled file); total disk = + `2 × OXICLOUD_MAX_UPLOAD_SIZE × concurrent_chunked_sessions`. + +The chunked formula uses `OXICLOUD_MAX_UPLOAD_SIZE` because that's +what bounds the declared `total_size` at session creation. The +direct-PUT formula uses the smaller `OXICLOUD_DIRECT_PUT_MAX_BYTES` +since that's what bounds each direct PUT body. ### Sizing examples -A 4 GB tmpfs serving a small team (~5 concurrent uploads): +A 4 GB tmpfs serving a small team (5 concurrent direct PUTs OR 5 +concurrent chunked sessions): -| Setting | Disk worst case | Safe on 4 GB tmpfs? | -|---|---|---| -| `MAX_UPLOAD=10 GB`, no `CHUNK_MAX` tweak | 50 GB direct, 100 GB chunked | ❌ no — single upload OOMs the tmpfs | -| `MAX_UPLOAD=500 MB`, `CHUNK_MAX=50 MB` | 2.5 GB direct, 5 GB chunked | ⚠ direct fits, chunked overflows | -| `MAX_UPLOAD=300 MB`, `CHUNK_MAX=30 MB` | 1.5 GB direct, 3 GB chunked | ✅ both fit | +| Settings | Direct-PUT worst case | Chunked worst case | Safe on 4 GB tmpfs? | +|---|---|---|---| +| Defaults: `OXICLOUD_MAX_UPLOAD_SIZE`=10 GB, `OXICLOUD_DIRECT_PUT_MAX_BYTES`=1 GiB, `OXICLOUD_CHUNK_MAX_BYTES`=100 MB | 5 GiB (5 × 1 GiB) | 100 GB (5 × 2 × 10 GB) | ❌ chunked overflows | +| `OXICLOUD_MAX_UPLOAD_SIZE`=500 MB, `OXICLOUD_DIRECT_PUT_MAX_BYTES`=100 MB, `OXICLOUD_CHUNK_MAX_BYTES`=20 MB | 500 MB | 5 GB | ⚠ direct PUT fits, chunked still overflows | +| `OXICLOUD_MAX_UPLOAD_SIZE`=300 MB, `OXICLOUD_DIRECT_PUT_MAX_BYTES`=50 MB, `OXICLOUD_CHUNK_MAX_BYTES`=10 MB | 250 MB | 3 GB | ✅ both fit | A real-disk volume (cheap, large): -| Setting | Disk worst case | Comment | -|---|---|---| -| `MAX_UPLOAD=10 GB`, `CHUNK_MAX=100 MB` (defaults) | 100 GB chunked worst case | Fine on a 200+ GB volume; almost any real-disk setup | -| `MAX_UPLOAD=100 GB`, `CHUNK_MAX=500 MB` | 1 TB chunked worst case | Plausible for video archives; needs a dedicated upload volume | +| Settings | Direct-PUT worst case | Chunked worst case | Comment | +|---|---|---|---| +| Defaults (see row above) | 5 GiB | 100 GB | Fine on a 200+ GB volume; almost any real-disk setup | +| `OXICLOUD_MAX_UPLOAD_SIZE`=100 GB, `OXICLOUD_DIRECT_PUT_MAX_BYTES`=5 GiB, `OXICLOUD_CHUNK_MAX_BYTES`=500 MB | 25 GiB | 1 TB | Plausible for video archives; needs a dedicated upload volume | ### Choosing tmpfs vs real disk | Constraint | Choice | |---|---| -| Upload caps × concurrent users ≤ free RAM × 0.5 | tmpfs OK (fast, atomic with `.blobs/` if also tmpfs) | -| Upload caps × concurrent users > free RAM × 0.5 | **real disk** — same FS as `.blobs/` ideal | +| `OXICLOUD_DIRECT_PUT_MAX_BYTES × concurrent_direct_PUTs + 2 × OXICLOUD_MAX_UPLOAD_SIZE × concurrent_chunked_sessions ≤ free RAM × 0.5` | tmpfs OK (fast, atomic with `.blobs/` if also tmpfs) | +| Worst case exceeds half free RAM | **real disk** — same filesystem as `.blobs/` ideal | | Container with cgroup memory limit | **real disk** — tmpfs spool counts against the cgroup limit and triggers OOMKill | | Multi-GB uploads expected | **real disk** — even small concurrency on tmpfs runs out of space | | Small-file workload only (≤ 50 MB), high concurrency | tmpfs gives a noticeable intake speedup | -The defaults (`MAX_UPLOAD=10 GB`, `CHUNK_MAX=100 MB`) assume **real -disk**. Don't run the defaults against tmpfs unless you've sized it -for the worst case. +The defaults (`OXICLOUD_MAX_UPLOAD_SIZE`=10 GB, +`OXICLOUD_DIRECT_PUT_MAX_BYTES`=1 GiB, +`OXICLOUD_CHUNK_MAX_BYTES`=100 MB) assume **real disk**. Don't run +the defaults against tmpfs unless you've sized it for the worst case. ## TL;DR diff --git a/example.env b/example.env index a83f6876..1c46e18e 100644 --- a/example.env +++ b/example.env @@ -31,18 +31,48 @@ OXICLOUD_SERVER_HOST=127.0.0.1 # Example: https://cloud.example.com #OXICLOUD_BASE_URL=https://cloud.example.com -# Maximum upload size in bytes (default: 10 GB on 64-bit) +# ── Upload size caps ────────────────────────────────────────────────── +# See docs/config/storage-fine-tuning.md for sizing guidance. + +# Whole-file size ceiling. Applies to BOTH direct PUTs (per-request body) +# and chunked uploads (declared `total_size` at session creation — checked +# upfront so oversized requests never accumulate chunks on disk). +# Default: 10 GB on 64-bit, 1 GB on 32-bit. #OXICLOUD_MAX_UPLOAD_SIZE=10737418240 -# Directory for upload spool temp files. Uploads are streamed to a temp file -# before deduplication. By default this uses the OS temp dir ($TMPDIR / /tmp), -# which in many containers is tmpfs (RAM) — writing a large upload there fills -# page-cache that counts against the cgroup memory limit and can OOMKill the -# process. Point this at a real-disk path (same filesystem as the storage -# backend is ideal) to keep the upload footprint off RAM. Leave unset to use -# the OS default. +# Per-request cap for non-chunked PUT bodies (`POST /api/files/upload`, +# `PUT /webdav/...`, `PUT /remote.php/dav/files/...`). Set below +# MAX_UPLOAD_SIZE so files larger than this are pushed onto the chunked +# protocol (resumable on failure, bounded per-request by CHUNK_MAX_BYTES). +# Default: 1 GiB. +#OXICLOUD_DIRECT_PUT_MAX_BYTES=1073741824 + +# Per-chunk cap for a single chunked-upload PUT (PATCH /api/uploads/{id} +# or PUT /dav/uploads/.../chunk). NC desktop and the OxiCloud frontend +# split large files into chunks of this size or smaller, so this knob +# tightly bounds the worst-case per-request memory/disk footprint +# independently of the whole-file cap. Default: 100 MB. +#OXICLOUD_CHUNK_MAX_BYTES=104857600 + +# ── Upload spool directories ───────────────────────────────────────── +# Where in-flight uploads land BEFORE being promoted into final blob +# storage. Same-filesystem placement (with `.blobs/`) makes the +# promotion an atomic rename(2); NVMe placement speeds up intake. +# See docs/config/storage-fine-tuning.md for layout examples. + +# Directory for non-chunked PUT spool tempfiles. Default: OS temp dir +# ($TMPDIR / /tmp), often tmpfs (RAM) in containers — writing a large +# upload there fills page-cache that counts against the cgroup memory +# limit and can OOMKill the process. Point this at a real-disk path +# (same FS as the storage backend is ideal). #OXICLOUD_UPLOAD_TMPDIR=/var/lib/oxicloud/tmp +# Root directory for chunked-upload sessions (REST + NextCloud chunked +# share this root). Default: {STORAGE_PATH}/.uploads. Pointing this at +# NVMe accelerates the chunk-write + assembly loop; pointing it at the +# same FS as `.blobs/` makes blob promotion atomic. +#OXICLOUD_CHUNK_DIR=/var/lib/oxicloud/.uploads + # How often (seconds) the background sweep reconciles each user's cached # storage usage with the real sum of their files (default: 600 = 10 min). # GET /api/auth/me serves the cached value instead of recomputing per request; diff --git a/src/common/config.rs b/src/common/config.rs index 94e2f699..f3c89561 100644 --- a/src/common/config.rs +++ b/src/common/config.rs @@ -218,6 +218,16 @@ pub struct StorageConfig { /// be far tighter than the whole-file cap and prevents one HTTP request /// from monopolising server memory or disk. Env: `OXICLOUD_CHUNK_MAX_BYTES`. pub chunk_max_bytes: usize, + /// Maximum size of a single non-chunked PUT body, in bytes (default: + /// 1 GiB). Set below `max_upload_size` so files larger than this are + /// pushed onto the chunked-upload protocol (`/api/uploads/…` or + /// `/dav/uploads/…`) — which is resilient to mid-transfer failures, + /// resumable, and bounded per-request by `chunk_max_bytes`. Without + /// this cap a 10 GB direct PUT spools 10 GB to disk in a single + /// request; a connection drop at 95 % loses everything. The server + /// returns 413 with a "use chunked upload" hint when a direct PUT + /// exceeds this cap. Env: `OXICLOUD_DIRECT_PUT_MAX_BYTES`. + pub direct_put_max_bytes: usize, /// Directory for upload spool temp files. When `Some`, large uploads are /// spooled here instead of the OS default temp dir (often tmpfs/RAM in /// containers, where the spool's page-cache counts against the cgroup @@ -376,6 +386,7 @@ impl Default for StorageConfig { trash_retention_days: 30, // 30 days max_upload_size: MAX_UPLOAD_SIZE, chunk_max_bytes: 100 * 1024 * 1024, // 100 MB — sane upper bound for a single chunked-upload PUT + direct_put_max_bytes: 1024 * 1024 * 1024, // 1 GiB — pushes larger uploads onto the chunked protocol upload_temp_dir: None, chunk_dir: None, usage_reconcile_secs: 600, // 10 minutes @@ -1247,6 +1258,12 @@ impl AppConfig { { config.storage.chunk_max_bytes = val; } + if let Ok(direct_max) = + env::var("OXICLOUD_DIRECT_PUT_MAX_BYTES").map(|v| v.parse::()) + && let Ok(val) = direct_max + { + config.storage.direct_put_max_bytes = val; + } // Upload spool directory — keep large upload temp files off tmpfs/RAM // (otherwise their page-cache counts against the cgroup memory limit). diff --git a/src/interfaces/api/handlers/chunked_upload_handler.rs b/src/interfaces/api/handlers/chunked_upload_handler.rs index 5a0b05fd..f2276e50 100644 --- a/src/interfaces/api/handlers/chunked_upload_handler.rs +++ b/src/interfaces/api/handlers/chunked_upload_handler.rs @@ -162,6 +162,33 @@ impl ChunkedUploadHandler { .into_response(); } + // ── Whole-file cap ────────────────────────────────────────── + // Reject upfront, before any chunk is uploaded — wasting + // bandwidth + server disk on an upload that's going to be + // rejected at /complete is the worst-of-both-worlds outcome. + // `max_upload_size` is the same ceiling that bounds direct + // PUTs (per-byte during streaming there; declared per-session + // here). When quotas are disabled, this is the only whole-file + // limit for chunked uploads — without it a hostile client + // could declare `total_size: 1 TB` and accumulate chunks + // until disk fills. + let max_upload = state.core.config.storage.max_upload_size as u64; + if request.total_size > max_upload { + tracing::warn!( + "⛔ CHUNKED UPLOAD REJECTED (total_size cap): user={}, file={}, declared={}, max={}", + auth_user.username, + request.filename, + request.total_size, + max_upload + ); + return AppError::payload_too_large(format!( + "Declared total_size {} exceeds the server's `max_upload_size` cap ({} bytes). \ + Raise OXICLOUD_MAX_UPLOAD_SIZE on the server if larger uploads are expected.", + request.total_size, max_upload + )) + .into_response(); + } + // ── Permission pre-check: caller must have Create on the target // folder BEFORE we allocate a session and accept chunks. The // upload service re-checks at finalize time, but failing here diff --git a/src/interfaces/api/handlers/webdav_handler.rs b/src/interfaces/api/handlers/webdav_handler.rs index ef339dff..152a3b9a 100644 --- a/src/interfaces/api/handlers/webdav_handler.rs +++ b/src/interfaces/api/handlers/webdav_handler.rs @@ -927,8 +927,10 @@ async fn handle_put( // another user — acceptable risk since PathResolver should always // be enabled in production) - // Hard upload size limit from config - let max_upload = state.core.config.storage.max_upload_size; + // Direct PUT cap — see `nextcloud/webdav_handler::handle_put` for + // the reasoning. Files above `direct_put_max_bytes` must go through + // the chunked-upload protocol (`/api/uploads/…`) which is resumable. + let max_upload = state.core.config.storage.direct_put_max_bytes; // Extract content type before consuming the request let content_type = req diff --git a/src/interfaces/nextcloud/webdav_handler.rs b/src/interfaces/nextcloud/webdav_handler.rs index 8be9577b..5a82baa7 100644 --- a/src/interfaces/nextcloud/webdav_handler.rs +++ b/src/interfaces/nextcloud/webdav_handler.rs @@ -596,7 +596,14 @@ async fn handle_put( .and_then(|v| v.to_str().ok()) .and_then(|v| v.parse::().ok()); - let max_upload = state.core.config.storage.max_upload_size; + // ── Direct PUT cap ─────────────────────────────────────────────── + // We use `direct_put_max_bytes` (default 1 GiB), not `max_upload_size` + // (default 10 GB). Larger files must come through the chunked upload + // protocol (`/dav/uploads/...`) which is resumable on failure and + // bounded per-request by `chunk_max_bytes`. Trying to stream a + // multi-GB body through a single PUT is a footgun: a connection drop + // at 95 % loses everything. + let max_upload = state.core.config.storage.direct_put_max_bytes; // Stream the body to a temp file + incremental hash — never buffer the // full upload in RAM. The old `body::to_bytes` path loaded the entire diff --git a/src/interfaces/upload_spool.rs b/src/interfaces/upload_spool.rs index 25878019..7ac2a56a 100644 --- a/src/interfaces/upload_spool.rs +++ b/src/interfaces/upload_spool.rs @@ -70,7 +70,10 @@ pub async fn spool_body_to_temp( drop(file); let _ = tokio::fs::remove_file(&temp_path).await; return Err(AppError::payload_too_large(format!( - "Upload exceeds maximum size of {max_upload} bytes" + "Upload body exceeds the direct-PUT cap ({max_upload} bytes). \ + Use the chunked-upload protocol (REST: `/api/uploads/...`, \ + NextCloud: `/remote.php/dav/uploads/...`) for files larger than this. \ + Chunked uploads are resumable on transient failure." ))); } hasher.update(chunk); diff --git a/src/main.rs b/src/main.rs index 8f3463aa..f6bd27af 100644 --- a/src/main.rs +++ b/src/main.rs @@ -150,6 +150,7 @@ async fn main() -> Result<(), Box> { // hardest to spot from chunked-upload tests. tracing::info!( max_upload_size_mb = config.storage.max_upload_size / (1024 * 1024), + direct_put_max_bytes_mb = config.storage.direct_put_max_bytes / (1024 * 1024), chunk_max_bytes_mb = config.storage.chunk_max_bytes / (1024 * 1024), "Upload limits loaded from config" ); diff --git a/tests/api/chunked_upload_cap.hurl b/tests/api/chunked_upload_cap.hurl index 12fd9038..992968fa 100644 --- a/tests/api/chunked_upload_cap.hurl +++ b/tests/api/chunked_upload_cap.hurl @@ -563,3 +563,120 @@ POST {{base_url}}/api/uploads/{{upload_id_nobody}}/complete Authorization: Bearer {{token}} HTTP 201 + + +# ═════════════════════════════════════════════════════════════ +# Whole-file size cap (OXICLOUD_MAX_UPLOAD_SIZE) +# ═════════════════════════════════════════════════════════════ +# The whole-file cap is checked at session creation against the +# JSON-declared `total_size` — no upload body required to trip +# it. Test default `OXICLOUD_MAX_UPLOAD_SIZE` is 10 GiB; we +# declare 100 GiB to be safely above without bothering with a +# Hurl `--variables` override. +# +# This guards the case where chunks would otherwise accumulate +# disk space against an oversized declared upload — the reject +# fires BEFORE any chunk is PATCHed. + + +# ───────────────────────────────────────────────────────────── +# Step 21 — POST /api/uploads with `total_size` above +# OXICLOUD_MAX_UPLOAD_SIZE → 413 Payload Too Large. +# No body bytes are sent; the check is purely +# against the declared JSON field. +# ───────────────────────────────────────────────────────────── +POST {{base_url}}/api/uploads +Authorization: Bearer {{token}} +Content-Type: application/json +{ + "filename": "way-too-big.bin", + "folder_id": "{{home_folder_id}}", + "content_type": "application/octet-stream", + "total_size": 107374182400, + "chunk_size": 5242880 +} + +HTTP 413 + + +# ───────────────────────────────────────────────────────────── +# Step 22 — Sanity check: a session JUST BELOW the cap is +# accepted. Uses a sub-cap value (32 bytes — same +# pattern as earlier steps so no extra fixture is +# needed). Confirms that the reject in step 21 was +# the size cap, not an unrelated regression. +# ───────────────────────────────────────────────────────────── +POST {{base_url}}/api/uploads +Authorization: Bearer {{token}} +Content-Type: application/json +{ + "filename": "small-and-fine.txt", + "folder_id": "{{home_folder_id}}", + "content_type": "text/plain", + "total_size": 32, + "chunk_size": 1048576 +} + +HTTP 201 +[Captures] +upload_id_sanity: jsonpath "$.upload_id" + +# Cleanup — cancel without uploading. +DELETE {{base_url}}/api/uploads/{{upload_id_sanity}} +Authorization: Bearer {{token}} + +HTTP 204 + + +# ═════════════════════════════════════════════════════════════ +# Direct-PUT body cap (OXICLOUD_DIRECT_PUT_MAX_BYTES) +# ═════════════════════════════════════════════════════════════ +# `OXICLOUD_DIRECT_PUT_MAX_BYTES` (4 MiB in the test env) bounds +# a single non-chunked PUT body. Larger uploads must come through +# the chunked protocol — the server returns 413 with a hint +# pointing at `/api/uploads/...` / `/dav/uploads/...`. The cap +# fires mid-stream via the same accumulator that bounds chunks. +# +# Tested via the native REST WebDAV PUT endpoint (`/webdav/...`) +# because it's JWT-authed (no app-password mint dance) and +# straightforwardly path-mapped. The cap is wired into the same +# `spool_body_to_temp` helper from the NextCloud single-file PUT +# path, so this exercise covers both wirings. + + +# ───────────────────────────────────────────────────────────── +# Step 23 — Direct PUT of the 5 MiB fixture → 413. The same +# fixture used in step 8 for the chunked-PATCH cap; +# here it lands against the direct-PUT cap. +# ───────────────────────────────────────────────────────────── +PUT {{base_url}}/webdav/test-direct-put-over-cap.bin +Authorization: Bearer {{token}} +Content-Type: application/octet-stream +file,fixtures/chunk-over-cap-5mb.bin; + +HTTP 413 + + +# ───────────────────────────────────────────────────────────── +# Step 24 — Sanity: direct PUT WELL under the cap → 201 or 204. +# Confirms step 23's reject was the cap, not a route / +# auth / handler regression. +# ───────────────────────────────────────────────────────────── +PUT {{base_url}}/webdav/test-direct-put-under-cap.txt +Authorization: Bearer {{token}} +Content-Type: text/plain +file,fixtures/hello.txt; + +# WebDAV PUT returns 201 on new file, 204 on overwrite. Either +# value confirms a successful body acceptance. +HTTP * +[Asserts] +status >= 200 +status < 300 + +# Cleanup so a re-run finds a clean slate (DELETE is idempotent +# enough for this — 204 on success, 404 if nothing left). +DELETE {{base_url}}/webdav/test-direct-put-under-cap.txt +Authorization: Bearer {{token}} + +HTTP * diff --git a/tests/common/server.env b/tests/common/server.env index d36eed35..8d5d36f8 100644 --- a/tests/common/server.env +++ b/tests/common/server.env @@ -27,6 +27,14 @@ RUST_LOG="warn,audit=info" # under the cap, while the cap test sends a 5 MiB fixture to trigger 413. OXICLOUD_CHUNK_MAX_BYTES=4194304 +# Direct-PUT (non-chunked) cap, exercised by chunked_upload_cap.hurl. +# 4 MiB: same threshold as the chunked cap so the existing 5 MiB +# fixture (chunk-over-cap-5mb.bin) can prove BOTH caps with one +# generated file. All existing direct-PUT tests +# (test_dedup_webdav_multichunk.sh = 2.76 MB, _ref_count = ~66 KB, +# _nextcloud_put_blake3 = 32 B) stay safely under this cap. +OXICLOUD_DIRECT_PUT_MAX_BYTES=4194304 + # grow up limits for tests OXICLOUD_RATE_LIMIT_REFRESH_MAX=360 OXICLOUD_RATE_LIMIT_LOGIN_MAX=360