feat(uploads): add OXICLOUD_CHUNK_DIR + documentation for admins

explain OXICLOUD_CHUNK_DIR and OXICLOUD_UPLOAD_TMPDIR
    and also the OXICLOUD_CHUNK_MAX_BYTES & OXICLOUD_UPLOAD_TMPDIR
    to help administratorrs to defined correctly their storage architecture
This commit is contained in:
Edouard Vanbelle
2026-06-09 10:19:21 +02:00
parent 2843b3351b
commit 4e36de49eb
5 changed files with 411 additions and 22 deletions
+3
View File
@@ -12,6 +12,9 @@ Most runtime variables use the `OXICLOUD_` prefix. A few build-time or allocator
| `OXICLOUD_SERVER_HOST` | `127.0.0.1` | Server bind address (IPv4 or IPv6 allowed) | | `OXICLOUD_SERVER_HOST` | `127.0.0.1` | Server bind address (IPv4 or IPv6 allowed) |
| `OXICLOUD_BASE_URL` | (auto) | Public base URL for share links; defaults to `http://{host}:{port}` | | `OXICLOUD_BASE_URL` | (auto) | Public base URL for share links; defaults to `http://{host}:{port}` |
| `OXICLOUD_MAX_UPLOAD_SIZE` | `10737418240` | Maximum upload size in bytes (10 GB on 64-bit, 1 GB on 32-bit) | | `OXICLOUD_MAX_UPLOAD_SIZE` | `10737418240` | Maximum upload size in bytes (10 GB on 64-bit, 1 GB on 32-bit) |
| `OXICLOUD_CHUNK_MAX_BYTES` | `104857600` | Maximum size of a single chunked-upload PUT in bytes (100 MB). Distinct from `OXICLOUD_MAX_UPLOAD_SIZE`, which is the whole-file cap. See [Storage Fine Tuning](./storage-fine-tuning.md). |
| `OXICLOUD_UPLOAD_TMPDIR` | (OS temp dir) | Spool directory for non-chunked PUT bodies (`/api/files/upload`, WebDAV PUT). Point at a real-disk path on the same FS as `.blobs/` to avoid tmpfs OOMKill and make blob promotion an atomic rename. See [Storage Fine Tuning](./storage-fine-tuning.md). |
| `OXICLOUD_CHUNK_DIR` | `{STORAGE_PATH}/.uploads` | Root directory for chunked-upload sessions (REST + NextCloud). Same-FS / NVMe placement guidance: see [Storage Fine Tuning](./storage-fine-tuning.md). |
| `OXICLOUD_REUSE_PORT` | `false` | Enable `SO_REUSEPORT` so multiple processes can share the same port. **Disabled by default** — a second accidental instance will fail with "address already in use". Enable only for deliberate multi-worker setups (process supervisor, rolling restart). Not supported on Windows. | | `OXICLOUD_REUSE_PORT` | `false` | Enable `SO_REUSEPORT` so multiple processes can share the same port. **Disabled by default** — a second accidental instance will fail with "address already in use". Enable only for deliberate multi-worker setups (process supervisor, rolling restart). Not supported on Windows. |
## Database ## Database
+294
View File
@@ -0,0 +1,294 @@
# Storage Fine Tuning
This page is for sysadmins who want to tune **where** OxiCloud spools
upload bodies and **why** the placement matters for throughput and
memory. The defaults work; the gains from a tuned layout are
significant on busy instances or constrained containers.
## The upload lifecycle in 30 seconds
Every upload moves through two stages:
```
┌─── direct (single-PUT) upload ────────┐
client ─┤ ├──► OxiCloud accepts the
└─── multi-chunk upload │ bytes into a SPOOL on
(`/api/uploads` / │ local disk.
`/dav/uploads/...`) │
│ Direct upload → OXICLOUD_UPLOAD_TMPDIR
│ Chunked upload → OXICLOUD_CHUNK_DIR
│
▼
┌─────────────────────────┐
│ Once the upload is │
│ complete (and verified │
│ if a checksum was │
│ supplied), OxiCloud │
│ MOVES the assembled │
│ blob into the configured│
│ STORAGE BACKEND: │
│ │
│ • local FS (.blobs/) │
│ • S3-compatible │
│ • Azure Blob │
└─────────────────────────┘
```
Two practical consequences:
- **The spool/chunk directories see write-heavy churn** during uploads —
fast disk (NVMe) and sufficient free space matter more here than on
the final storage backend.
- **The promotion from spool → storage is a `rename(2)` whenever
source and destination share a filesystem** (i.e. when the backend
is `local` and the spool dir is on the same FS as `.blobs/`). On
remote backends (S3, Azure) the promotion is always a network
upload from the local spool; placement of the spool still matters
for intake throughput but the "same FS" rule doesn't apply.
## Upload size caps — what each one bounds
Three independent caps control how large an upload OxiCloud will
accept. Pick them with disk and tmpfs sizing in mind: the spool/chunk
directories must be able to hold the worst case (max cap × concurrent
uploads).
| Variable | Default | What it caps |
|---|---|---|
| `OXICLOUD_MAX_UPLOAD_SIZE` | 10 GB | Total file size — whether uploaded as one `PUT` or assembled from many chunks. The hard ceiling on any single file. |
| `OXICLOUD_CHUNK_MAX_BYTES` | 100 MB | A single chunked-PUT request body (`PATCH /api/uploads/{id}` or `PUT /dav/uploads/.../chunk`). Independent of the whole-file cap — a 5 GB file uploaded in 100 MB chunks needs ~50 PUTs, each bounded by this. |
| `OXICLOUD_MAX_UPLOAD_SIZE` (again, for the PUT path) | 10 GB | A single non-chunked PUT body. Same env var as the whole-file cap because for non-chunked uploads they're equivalent. |
### Why these matter for tmpfs sizing
OxiCloud streams bodies frame-by-frame, so **RAM** is bounded to one
HTTP frame (~64 KB) per request regardless of the caps. **Disk space**,
however, scales with the caps:
- **Direct PUT** (single-file): each in-flight upload spools the full
body to disk under `OXICLOUD_UPLOAD_TMPDIR` until promotion. Worst
case disk = `OXICLOUD_MAX_UPLOAD_SIZE × concurrent_PUTs`.
- **Chunked upload**: each in-flight session accumulates chunks under
`OXICLOUD_CHUNK_DIR`, then assembles them into a single temp file
before promotion. Worst case disk per session = **2 × file_size**
(chunks + assembled file); total disk =
`2 × OXICLOUD_MAX_UPLOAD_SIZE × concurrent_sessions`.
### Sizing examples
A 4 GB tmpfs serving a small team (~5 concurrent uploads):
| Setting | Disk worst case | Safe on 4 GB tmpfs? |
|---|---|---|
| `MAX_UPLOAD=10 GB`, no `CHUNK_MAX` tweak | 50 GB direct, 100 GB chunked | ❌ no — single upload OOMs the tmpfs |
| `MAX_UPLOAD=500 MB`, `CHUNK_MAX=50 MB` | 2.5 GB direct, 5 GB chunked | ⚠ direct fits, chunked overflows |
| `MAX_UPLOAD=300 MB`, `CHUNK_MAX=30 MB` | 1.5 GB direct, 3 GB chunked | ✅ both fit |
A real-disk volume (cheap, large):
| Setting | Disk worst case | Comment |
|---|---|---|
| `MAX_UPLOAD=10 GB`, `CHUNK_MAX=100 MB` (defaults) | 100 GB chunked worst case | Fine on a 200+ GB volume; almost any real-disk setup |
| `MAX_UPLOAD=100 GB`, `CHUNK_MAX=500 MB` | 1 TB chunked worst case | Plausible for video archives; needs a dedicated upload volume |
### Choosing tmpfs vs real disk
| Constraint | Choice |
|---|---|
| Upload caps × concurrent users ≤ free RAM × 0.5 | tmpfs OK (fast, atomic with `.blobs/` if also tmpfs) |
| Upload caps × concurrent users > free RAM × 0.5 | **real disk** — same FS as `.blobs/` ideal |
| Container with cgroup memory limit | **real disk** — tmpfs spool counts against the cgroup limit and triggers OOMKill |
| Multi-GB uploads expected | **real disk** — even small concurrency on tmpfs runs out of space |
| Small-file workload only (≤ 50 MB), high concurrency | tmpfs gives a noticeable intake speedup |
The defaults (`MAX_UPLOAD=10 GB`, `CHUNK_MAX=100 MB`) assume **real
disk**. Don't run the defaults against tmpfs unless you've sized it
for the worst case.
## TL;DR
| Variable | Default | Purpose |
|---|---|---|
| `OXICLOUD_STORAGE_PATH` | `./storage` | Where `.blobs/` lives (the canonical content store) |
| `OXICLOUD_UPLOAD_TMPDIR` | OS temp dir | Where non-chunked PUT bodies are spooled |
| `OXICLOUD_CHUNK_DIR` | `{STORAGE_PATH}/.uploads` | Where chunked-upload sessions accumulate |
The two rules that matter most:
1. **Put all three on the same filesystem.** Blob promotion is an
atomic `rename(2)` when source and destination share an FS — cheap
and crash-safe. Across filesystems it becomes a full `read + write +
unlink`, multiplying the IO and widening the durability window.
2. **Don't leave the spool dir on tmpfs** (the default in many
containers). Spool bodies count against the cgroup memory limit
and can trigger OOMKill on multi-GB uploads.
## Where each upload surface spools
OxiCloud has several entry points that accept request bodies. They
land in different places by default:
| Surface | Default destination | Configurable via |
|---|---|---|
| REST chunked PUT (`PATCH /api/uploads/{id}`) | `{STORAGE_PATH}/.uploads/{upload_id}/chunk_NNNNNN` | `OXICLOUD_CHUNK_DIR` |
| REST chunked assemble (during `/complete`) | `{STORAGE_PATH}/.uploads/{upload_id}/assembled` | `OXICLOUD_CHUNK_DIR` |
| NextCloud chunked PUT (`PUT /dav/uploads/.../chunk`) | `{STORAGE_PATH}/.uploads/nextcloud/{user}/{upload_id}/{chunk_name}` | `OXICLOUD_CHUNK_DIR` |
| NextCloud chunked assemble (during `MOVE`) | `{STORAGE_PATH}/.uploads/nextcloud/{user}/{upload_id}/.assembled` | `OXICLOUD_CHUNK_DIR` |
| Native WebDAV PUT (`PUT /webdav/{path}`) | OS temp dir (`/tmp`) | `OXICLOUD_UPLOAD_TMPDIR` |
| NextCloud single-file PUT (`PUT /dav/files/.../{path}`) | OS temp dir | `OXICLOUD_UPLOAD_TMPDIR` |
| REST multipart upload (`POST /api/files/upload`) | `{STORAGE_PATH}/.dedup_temp/upload-{uuid}` | `OXICLOUD_STORAGE_PATH` (subdir is hard-wired) |
| Final blob storage (after fsync + rename) | `{STORAGE_PATH}/.blobs/{ab}/{abc…}.blob` | `OXICLOUD_STORAGE_PATH` |
## Why placement matters
### 1. Same filesystem ⇒ promotion is a rename
OxiCloud uses **content-addressable storage**: the final blob path is
derived from the file's BLAKE3 hash, which can only be known after the
last byte arrives. So every upload writes to a temp location first,
then **promotes** the temp file to `.blobs/{ab}/{abc…}.blob` by way of
a `rename(2)` call.
- **Same FS:** `rename` is atomic, O(1), no data copy. Total upload
cost = body bytes received + one rename syscall. Crash-safe — the
blob either exists at the final path or doesn't.
- **Cross-FS:** the kernel can't `rename(2)` across filesystems. The
blob backend falls back to `fs::copy + fs::remove_file` (visible in
`local_blob_backend.rs` as the EXDEV handler). Total cost = body
bytes received + one full file copy. Doubles the IO bandwidth used
per upload and widens the durability window.
### 2. Spool off tmpfs
`tempfile::NamedTempFile::new()` (used when `OXICLOUD_UPLOAD_TMPDIR`
is unset) honors `$TMPDIR`, which in many container setups points at
**tmpfs** — RAM-backed storage. A 2 GB upload spool then consumes 2 GB
of memory until the rename promotes it to disk.
In a Kubernetes pod with a 4 GB memory limit, the OOMKiller wakes up
long before the upload finishes. With `OXICLOUD_UPLOAD_TMPDIR` pointed
at a real-disk directory, the spool's memory footprint stays at ~one
HTTP frame regardless of file size.
### 3. NVMe for the hot path
The chunked-upload session directory sees a LOT of small writes —
each chunk PUT writes a file, the progress bitmap is rewritten after
each PUT, the assemble step reads them all back in order. Pointing
`OXICLOUD_CHUNK_DIR` at an NVMe device is a substantial win on
deployments that handle large file uploads, even if the final blob
storage is on slower disk.
The same applies to `OXICLOUD_UPLOAD_TMPDIR` (single-file PUTs).
A common high-throughput layout:
- **NVMe** (small, fast): `OXICLOUD_CHUNK_DIR`, `OXICLOUD_UPLOAD_TMPDIR`
- **HDD or NAS** (large, cheap): `OXICLOUD_STORAGE_PATH`/`.blobs/`
Trade-off: the rename optimization (rule 1) DOESN'T apply across
filesystems. If you split the hot path off the blob filesystem, every
upload pays a full file copy on promotion. You have to choose
between **fast intake** and **zero-copy promotion**.
| Goal | Layout | Cost per upload |
|---|---|---|
| Fastest possible intake | NVMe chunk dir + HDD blobs | 1× write to NVMe + 1× read NVMe + 1× write to HDD (copy) |
| Lowest IO + crash safety | NVMe everything OR HDD everything | 1× write to disk + 1 rename (~0 cost) |
| Default (do nothing) | Everything under `STORAGE_PATH` on whatever FS that is | Depends on `STORAGE_PATH` placement |
For most deployments **"same FS everywhere"** wins. The NVMe-split is
useful when intake latency dominates the user experience and you can
afford the doubled IO.
## Recommended layouts
### Single-disk box (most common)
Defaults are fine. Optionally set `OXICLOUD_UPLOAD_TMPDIR` to keep
the PUT spool off `/tmp`:
```bash
OXICLOUD_STORAGE_PATH=/var/lib/oxicloud
OXICLOUD_UPLOAD_TMPDIR=/var/lib/oxicloud/.spool
# OXICLOUD_CHUNK_DIR unset → /var/lib/oxicloud/.uploads
```
All three on the same filesystem → rename promotion → atomic and fast.
### Container with constrained memory
Critical: make sure neither spool sits on tmpfs.
```bash
OXICLOUD_STORAGE_PATH=/data
OXICLOUD_UPLOAD_TMPDIR=/data/.spool
OXICLOUD_CHUNK_DIR=/data/.uploads
```
If you can't mount a writable `/data`, at minimum bind-mount a real
volume at the spool dirs.
### Split-disk (NVMe intake + HDD blobs)
```bash
OXICLOUD_STORAGE_PATH=/mnt/hdd/oxicloud # .blobs/ + .dedup_temp/
OXICLOUD_UPLOAD_TMPDIR=/mnt/nvme/oxi-spool
OXICLOUD_CHUNK_DIR=/mnt/nvme/oxi-chunks
```
Faster intake; pays a copy on promotion. Worth it when uploads are
many small files (NVMe IOPS dominates) or when intake latency directly
hits user-visible UX.
## Sharing the spool and chunk directories
Pointing `OXICLOUD_UPLOAD_TMPDIR` and `OXICLOUD_CHUNK_DIR` at the
**same directory** is supported by design. Each writer tags its
output so the surfaces never interfere with each other:
| Writer | On-disk name pattern |
|---|---|
| PUT spool (single-file uploads) | `.tmpXXXXXXXX` — files (not directories), random suffix |
| REST chunked sessions | `oxi-chunk-{uuid}/` — directories with a well-known prefix |
| NC chunked subtree | `nextcloud/{user}/{uuid}/` — under its own root subdir |
The 24-hour orphan-session cleanup loop filters strictly on the
`oxi-chunk-` prefix, so it can NEVER delete a non-OxiCloud directory
that happens to live alongside chunked sessions. The PUT spool's
`.tmpXXXX` files are files (not directories) and the NC subtree's
`nextcloud/` root has its own name — both are invisible to the
cleanup loop.
**Recommendation:** for new deployments, use separate directories
anyway (the defaults `.spool/` and `.uploads/` already do this) —
it makes disk-usage attribution clearer and keeps IOPS isolated when
both are busy. Shared directories are safe to use when disk layout
forces it.
## What's NOT yet configurable
- **REST multipart upload directory** (`POST /api/files/upload`) is
hard-wired to `{STORAGE_PATH}/.dedup_temp/`. It can't be moved
separately. Same-FS placement is automatic.
- **WOPI PutFile spool** (Office editor saves) uses the bare OS temp
dir without honoring `OXICLOUD_UPLOAD_TMPDIR`. This is a known
inconsistency and on the hardening backlog.
- **Per-user / per-drive spool directories** — all users share the
same `OXICLOUD_CHUNK_DIR` root today. Multi-tenant isolation
through separate spool dirs isn't supported.
## Quick verification
Boot the server with `RUST_LOG=info` and the first lines after the
banner include:
```
oxicloud: Upload limits loaded from config max_upload_size_mb=10240 chunk_max_bytes_mb=100
```
That confirms the upload-cap env vars were read. To confirm
directory placement, watch for chunk file creation under your
`OXICLOUD_CHUNK_DIR` (or its default `{STORAGE_PATH}/.uploads/`)
during a chunked upload — `ls` while a sync is in progress shows the
`{uuid}/chunk_NNNNNN` files appearing in real time.
+20
View File
@@ -224,6 +224,15 @@ pub struct StorageConfig {
/// memory limit and can trigger OOMKill on large files). Env: /// memory limit and can trigger OOMKill on large files). Env:
/// `OXICLOUD_UPLOAD_TMPDIR`. /// `OXICLOUD_UPLOAD_TMPDIR`.
pub upload_temp_dir: Option<PathBuf>, pub upload_temp_dir: Option<PathBuf>,
/// Root directory for chunked-upload sessions. When `Some`, chunks land
/// under `{chunk_dir}/{upload_id}/` (REST) and
/// `{chunk_dir}/nextcloud/{user}/{upload_id}/` (NC). When `None`, falls
/// back to `{root_dir}/.uploads/`. Pointing this at the **same
/// filesystem** as `.blobs/` keeps the final assembled-to-blob promotion
/// an atomic `rename(2)` rather than a full cross-FS copy; pointing it
/// at fast storage (NVMe) accelerates the chunk-write + assembly loop
/// independently of where final blobs live. Env: `OXICLOUD_CHUNK_DIR`.
pub chunk_dir: Option<PathBuf>,
/// Interval (seconds) of the background sweep that reconciles every user's /// Interval (seconds) of the background sweep that reconciles every user's
/// cached `storage_used_bytes` with the real sum of their files. Keeps the /// cached `storage_used_bytes` with the real sum of their files. Keeps the
/// quota fresh for all mutations without recomputing on the request path. /// quota fresh for all mutations without recomputing on the request path.
@@ -368,6 +377,7 @@ impl Default for StorageConfig {
max_upload_size: MAX_UPLOAD_SIZE, max_upload_size: MAX_UPLOAD_SIZE,
chunk_max_bytes: 100 * 1024 * 1024, // 100 MB — sane upper bound for a single chunked-upload PUT chunk_max_bytes: 100 * 1024 * 1024, // 100 MB — sane upper bound for a single chunked-upload PUT
upload_temp_dir: None, upload_temp_dir: None,
chunk_dir: None,
usage_reconcile_secs: 600, // 10 minutes usage_reconcile_secs: 600, // 10 minutes
backend: StorageBackendType::Local, backend: StorageBackendType::Local,
s3: None, s3: None,
@@ -1245,6 +1255,16 @@ impl AppConfig {
{ {
config.storage.upload_temp_dir = Some(PathBuf::from(dir.trim())); config.storage.upload_temp_dir = Some(PathBuf::from(dir.trim()));
} }
// Chunked-upload session root — separate from the PUT spool because
// chunked sessions accumulate disk on long uploads (multi-chunk
// resumable transfers) while PUT spool is short-lived. Sysadmins
// commonly want one of them on fast/local storage (NVMe) and the
// other on bulk storage; this knob lets that be expressed.
if let Ok(dir) = env::var("OXICLOUD_CHUNK_DIR")
&& !dir.trim().is_empty()
{
config.storage.chunk_dir = Some(PathBuf::from(dir.trim()));
}
// Background storage-usage reconciliation interval // Background storage-usage reconciliation interval
if let Ok(secs) = if let Ok(secs) =
+27 -4
View File
@@ -171,11 +171,23 @@ impl AppServiceFactory {
// Initialize thumbnail directories // Initialize thumbnail directories
thumbnail_service.initialize().await?; thumbnail_service.initialize().await?;
// Chunked upload service for large files (>10MB) // Chunked upload service for large files (>10MB).
let chunked_temp_dir = std::path::PathBuf::from(&self.storage_path).join(".uploads"); // Root for both REST (`/api/uploads/...`) and NC (`/dav/uploads/...`)
// chunked sessions: honour `OXICLOUD_CHUNK_DIR` when set so sysadmins
// can put session directories on fast storage (NVMe) or on the same
// filesystem as `.blobs/` (turns the final blob promotion into an
// atomic rename instead of a cross-FS copy). Falls back to
// `{storage_path}/.uploads/` when unset — backwards-compatible with
// every existing deployment.
let chunk_root = self
.config
.storage
.chunk_dir
.clone()
.unwrap_or_else(|| std::path::PathBuf::from(&self.storage_path).join(".uploads"));
let chunked_upload_service = Arc::new( let chunked_upload_service = Arc::new(
crate::infrastructure::services::chunked_upload_service::ChunkedUploadService::new( crate::infrastructure::services::chunked_upload_service::ChunkedUploadService::new(
chunked_temp_dir, chunk_root.clone(),
) )
.await, .await,
); );
@@ -870,7 +882,18 @@ impl AppServiceFactory {
); );
} }
let chunk_base = self.storage_path.join(".uploads/nextcloud"); // NC chunked-upload sessions root. Honour `OXICLOUD_CHUNK_DIR`
// (same env var that the REST chunked service uses) so a single
// value covers both surfaces and they stay co-located on one
// filesystem; fall back to `{storage_path}/.uploads/` to match
// the legacy layout.
let chunk_root = self
.config
.storage
.chunk_dir
.clone()
.unwrap_or_else(|| self.storage_path.join(".uploads"));
let chunk_base = chunk_root.join("nextcloud");
let chunked_uploads = Arc::new(NextcloudChunkedUploadService::new(chunk_base)); let chunked_uploads = Arc::new(NextcloudChunkedUploadService::new(chunk_base));
let file_id_repo = Arc::new( let file_id_repo = Arc::new(
@@ -44,10 +44,32 @@ pub const MAX_PARALLEL_CHUNKS: usize = 6;
/// Upload session expiration time (24 h) /// Upload session expiration time (24 h)
const SESSION_EXPIRATION: Duration = Duration::from_secs(24 * 60 * 60); const SESSION_EXPIRATION: Duration = Duration::from_secs(24 * 60 * 60);
/// Prefix every session directory name with this string so the cleanup
/// loop can be safely co-located with unrelated writers (PUT spool
/// tempfiles, the NC chunked subtree, anything else a sysadmin places
/// under the same `OXICLOUD_CHUNK_DIR`). The orphan-cleanup scan
/// filters by this prefix, so non-OxiCloud directories sharing the
/// root are never touched.
const SESSION_DIR_PREFIX: &str = "oxi-chunk-";
/// Sentinel file names inside each session directory /// Sentinel file names inside each session directory
const SESSION_META_FILE: &str = "session.json"; const SESSION_META_FILE: &str = "session.json";
const PROGRESS_FILE: &str = "progress.bin"; const PROGRESS_FILE: &str = "progress.bin";
/// Build a session directory name from an upload_id by attaching the
/// well-known prefix. Symmetric with [`strip_session_prefix`].
fn session_dir_name(upload_id: &str) -> String {
format!("{}{}", SESSION_DIR_PREFIX, upload_id)
}
/// Extract the upload_id from a session directory name. Returns
/// `None` when the directory wasn't created by this service (no
/// `oxi-chunk-` prefix) — the recovery and cleanup paths use this to
/// skip foreign directories cohabiting under `OXICLOUD_CHUNK_DIR`.
fn strip_session_prefix(dir_name: &str) -> Option<&str> {
dir_name.strip_prefix(SESSION_DIR_PREFIX)
}
// ─── Serialisable types ────────────────────────────────────────────────────── // ─── Serialisable types ──────────────────────────────────────────────────────
/// Chunk status /// Chunk status
@@ -252,6 +274,17 @@ impl ChunkedUploadService {
if !dir.is_dir() { if !dir.is_dir() {
continue; continue;
} }
// Only consider directories WE created — anything without the
// `oxi-chunk-` prefix belongs to a sibling writer (NC subtree,
// PUT spool tempfiles, sysadmin-placed dirs) and must be left
// strictly alone. See `SESSION_DIR_PREFIX`.
let dir_name = match dir.file_name().and_then(|n| n.to_str()) {
Some(n) => n,
None => continue,
};
if strip_session_prefix(dir_name).is_none() {
continue;
}
let meta_path = dir.join(SESSION_META_FILE); let meta_path = dir.join(SESSION_META_FILE);
let meta_bytes = match fs::read(&meta_path).await { let meta_bytes = match fs::read(&meta_path).await {
@@ -346,21 +379,32 @@ impl ChunkedUploadService {
} }
} }
// Also clean orphaned temp directories (no session.json or very old) // Also clean orphaned temp directories (no session.json or very old).
// Filter strictly on the `oxi-chunk-` prefix so we never touch
// sibling directories sharing `OXICLOUD_CHUNK_DIR` (NC subtree
// `nextcloud/`, PUT spool tempfiles which are files anyway,
// operator-placed dirs). Without the prefix filter this loop
// would silently delete anything older than 24 h sitting at the
// root of the chunked-upload dir.
if let Ok(mut entries) = fs::read_dir(&temp_base_dir).await { if let Ok(mut entries) = fs::read_dir(&temp_base_dir).await {
while let Ok(Some(entry)) = entries.next_entry().await { while let Ok(Some(entry)) = entries.next_entry().await {
let path = entry.path(); let path = entry.path();
if path.is_dir() { if !path.is_dir() {
let dir_name = path.file_name().and_then(|n| n.to_str()).unwrap_or(""); continue;
}
let dir_name = path.file_name().and_then(|n| n.to_str()).unwrap_or("");
let upload_id = match strip_session_prefix(dir_name) {
Some(id) => id,
None => continue, // not ours — never touch
};
if !sessions.contains_key(dir_name) if !sessions.contains_key(upload_id)
&& let Ok(metadata) = fs::metadata(&path).await && let Ok(metadata) = fs::metadata(&path).await
&& let Ok(modified) = metadata.modified() && let Ok(modified) = metadata.modified()
&& modified.elapsed().unwrap_or_default() > SESSION_EXPIRATION && modified.elapsed().unwrap_or_default() > SESSION_EXPIRATION
{ {
let _ = fs::remove_dir_all(&path).await; let _ = fs::remove_dir_all(&path).await;
tracing::info!("🧹 Cleaned orphaned upload dir: {:?}", path); tracing::info!("🧹 Cleaned orphaned upload dir: {:?}", path);
}
} }
} }
} }
@@ -396,8 +440,11 @@ impl ChunkedUploadService {
let chunk_size = chunk_size.unwrap_or(DEFAULT_CHUNK_SIZE); let chunk_size = chunk_size.unwrap_or(DEFAULT_CHUNK_SIZE);
let chunk_count = UploadSession::calculate_chunk_count(total_size, chunk_size); let chunk_count = UploadSession::calculate_chunk_count(total_size, chunk_size);
// Create temp directory for chunks // Create temp directory for chunks. The `oxi-chunk-` prefix
let temp_dir = self.temp_base_dir.join(&upload_id); // tags the directory as belonging to this service so the
// shared-`OXICLOUD_CHUNK_DIR` story holds — see
// `SESSION_DIR_PREFIX` for the full rationale.
let temp_dir = self.temp_base_dir.join(session_dir_name(&upload_id));
fs::create_dir_all(&temp_dir) fs::create_dir_all(&temp_dir)
.await .await
.map_err(|e| format!("Failed to create temp directory: {e}"))?; .map_err(|e| format!("Failed to create temp directory: {e}"))?;
@@ -1254,7 +1301,7 @@ mod tests {
.expect("upload_chunk 0"); .expect("upload_chunk 0");
// Verify files exist on disk // Verify files exist on disk
let session_dir = base.join(&upload_id); let session_dir = base.join(session_dir_name(&upload_id));
assert!(session_dir.join(SESSION_META_FILE).exists()); assert!(session_dir.join(SESSION_META_FILE).exists());
assert!(session_dir.join(PROGRESS_FILE).exists()); assert!(session_dir.join(PROGRESS_FILE).exists());
assert!(session_dir.join("chunk_000000").exists()); assert!(session_dir.join("chunk_000000").exists());
@@ -1366,7 +1413,7 @@ mod tests {
.await .await
.expect("create"); .expect("create");
let session_dir = base.join(&resp.upload_id); let session_dir = base.join(session_dir_name(&resp.upload_id));
assert!(session_dir.exists()); assert!(session_dir.exists());
service service
@@ -1385,8 +1432,10 @@ mod tests {
let base = std::env::temp_dir().join(format!("oxicloud_test_{}", Uuid::new_v4())); let base = std::env::temp_dir().join(format!("oxicloud_test_{}", Uuid::new_v4()));
let _ = fs::create_dir_all(&base).await; let _ = fs::create_dir_all(&base).await;
// Manually create an expired session on disk // Manually create an expired session on disk. The dir name MUST
let session_dir = base.join("expired-session"); // carry the `oxi-chunk-` prefix or recovery will (correctly) skip
// it as belonging to another writer co-located in chunk_dir.
let session_dir = base.join(session_dir_name("expired-session"));
let _ = fs::create_dir_all(&session_dir).await; let _ = fs::create_dir_all(&session_dir).await;
let expired_session = UploadSession { let expired_session = UploadSession {
@@ -1429,7 +1478,7 @@ mod tests {
let base = std::env::temp_dir().join(format!("oxicloud_test_{}", Uuid::new_v4())); let base = std::env::temp_dir().join(format!("oxicloud_test_{}", Uuid::new_v4()));
let _ = fs::create_dir_all(&base).await; let _ = fs::create_dir_all(&base).await;
let session_dir = base.join("partial-session"); let session_dir = base.join(session_dir_name("partial-session"));
let _ = fs::create_dir_all(&session_dir).await; let _ = fs::create_dir_all(&session_dir).await;
let session = UploadSession { let session = UploadSession {