Stream uploads directly into the CDC chunk store (no spool, single write)

Every upload surface previously wrote each byte to disk twice: the HTTP
body was spooled to a temp file (or assembled from chunk parts), then
mmap-re-read for FastCDC analysis, and finally the new chunks were
written to the blob backend. CDC could not start until the last byte
arrived, so large uploads paid receive + reread + rewrite latency.

The dedup engine now chunks, hashes and settles the stream WHILE it
arrives (fastcdc AsyncStreamCDC + incremental BLAKE3):

- Each batch of distinct chunks is pinned-or-classified by ONE
  `UPDATE … RETURNING` (no check-then-bump TOCTOU; pinned chunks can't
  be reclaimed mid-upload), and only chunks the store doesn't have are
  written — a full dedup hit performs zero content writes.
- Durability before visibility is preserved: one batched fsync sweep,
  then one batched INSERT, then the manifest. Identical concurrent
  uploads are resolved at the manifest INSERT via ON CONFLICT (the
  loser releases its references and becomes a dedup hit).
- A drop guard rolls back pins and surfaces written-but-unregistered
  chunks to GC if the request future is cancelled mid-stream.
- MIME sniffing now peeks the first bytes in-flight; client-requested
  MD5/SHA-256 checksums are computed by a stream tee — the post-upload
  re-read of the assembled file is gone.

All surfaces converge on the new interfaces::upload_ingest helper:
REST multipart, WebDAV PUT, NextCloud PUT, WOPI PutFile, the dedup
endpoint, and both chunked-upload completions (which now stream their
ordered parts straight into the store instead of writing an assembled
file — chunk parts persist until finalize, so completion is genuinely
retryable). The legacy blob re-chunk migration streams from the
backend with no spool file either.

Legacy removed: store_from_file + mmap CDC analysers + temp-path
plumbing through every port (pre_computed_hash, save_file_from_temp,
update_file_content_from_temp), upload_spool + assembled-file
assembly in both chunked services, create_file/update_file byte-slice
variants (no callers), common::temp, the OXICLOUD_UPLOAD_TMPDIR
config, and the memmap2 dependency.

Verified end-to-end against PostgreSQL 16: 8 MB upload (26 chunks),
identical re-upload (dedup hit, zero writes), 3-byte edit re-upload
(26 chunks, 1 written), byte-identical downloads, Range across chunk
boundaries, concurrent identical-upload race (manifest ref 2), and
trash-empty reclaiming exactly the unshared chunk while the shared 25
survive for the edited file. The empty/sub-8KB multipart path found a
post-EOF re-poll panic in the MIME peek (fixed with fuse + regression
test).

https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS

src/application/ports/chunked_upload_ports.rs

@@ -98,6 +98,21 @@ pub struct UploadStatusResponseDto {
     pub is_complete: bool,
 }
 
+/// A completed upload session, ready to be streamed into the blob store.
+///
+/// `chunk_paths` lists every chunk file in assembly order — the caller
+/// concatenates them as one byte stream (CDC chunking + hashing happen in
+/// that single pass). The files stay on disk until `finalize_upload`, so a
+/// failed completion (e.g. checksum mismatch) remains retryable.
+#[derive(Debug, Clone)]
+pub struct CompletedUploadParts {
+    pub chunk_paths: Vec<PathBuf>,
+    pub filename: String,
+    pub folder_id: Option<String>,
+    pub content_type: String,
+    pub total_size: u64,
+}
+
 /// Port for chunked/resumable upload operations.
 ///
 /// Implementations manage upload sessions, chunk storage, reassembly,
@@ -137,16 +152,15 @@ pub trait ChunkedUploadPort: Send + Sync + 'static {
         user_id: Uuid,
     ) -> Result<UploadStatusResponseDto, DomainError>;
 
-    /// Assemble all chunks into the final file.
+    /// Validate completion and hand back the ordered chunk parts.
     ///
-    /// Returns `(assembled_file_path, filename, folder_id, content_type, total_size, blake3_hash)`.
-    /// The hash is computed during assembly (hash-on-write), eliminating a
-    /// second sequential read of the assembled file.
+    /// No assembled file is written — the caller streams the parts straight
+    /// into the content-addressable store.
     async fn complete_upload(
         &self,
         upload_id: &str,
         user_id: Uuid,
-    ) -> Result<(PathBuf, String, Option<String>, String, u64, String), DomainError>;
+    ) -> Result<CompletedUploadParts, DomainError>;
 
     /// Finalize upload: clean up the session and temporary files.
     async fn finalize_upload(&self, upload_id: &str, user_id: Uuid) -> Result<(), DomainError>;

src/application/ports/dedup_ports.rs

@@ -80,18 +80,6 @@ pub struct DedupStatsDto {
 /// duplicate storage automatically. Multiple file references can
 /// point to the same physical blob.
 pub trait DedupPort: Send + Sync + 'static {
-    /// Store content with deduplication (streaming from file).
-    ///
-    /// If `pre_computed_hash` is provided (e.g. hash-on-write from the handler),
-    /// the file will NOT be re-read to calculate the hash — saving one full
-    /// sequential read of the file.
-    async fn store_from_file(
-        &self,
-        source_path: &Path,
-        content_type: Option<String>,
-        pre_computed_hash: Option<String>,
-    ) -> Result<DedupResultDto, DomainError>;
-
     /// Check if a blob with the given hash exists.
     async fn blob_exists(&self, hash: &str) -> bool;
 

src/application/ports/file_ports.rs

@@ -1,6 +1,5 @@
 use bytes::Bytes;
 use futures::Stream;
-use std::path::Path;
 use std::pin::Pin;
 use std::sync::Arc;
 use uuid::Uuid;
@@ -17,74 +16,51 @@ use crate::domain::services::authorization::Permission;
 // Upload port
 // ─────────────────────────────────────────────────────
 
+/// A blob already stored in the content-addressable chunk store, carrying
+/// exactly ONE reference that the receiving method takes ownership of.
+///
+/// Produced by the upload-ingest layer (`interfaces::upload_ingest`), which
+/// streams the request body straight into the CDC dedup store. Methods that
+/// accept a `StoredBlob` either attach the reference to a file row or
+/// release it on failure — callers never need to compensate themselves.
+#[derive(Debug, Clone)]
+pub struct StoredBlob {
+    /// BLAKE3 of the full content (manifest / blob key).
+    pub hash: String,
+    /// Content size in bytes.
+    pub size: u64,
+    /// `false` when the content already existed (dedup hit) — forwarded to
+    /// lifecycle hooks so e.g. thumbnails aren't regenerated for known blobs.
+    pub is_new_blob: bool,
+}
+
 /// Primary port for file upload operations.
 ///
-/// **All upload paths converge on streaming-to-disk** — no method accepts
-/// `Vec<u8>` for content.  Even `create_file` / `update_file` (WebDAV
-/// helpers that receive `&[u8]`) spool to a temp file internally so that
-/// peak RAM stays at ~256 KB regardless of file size.
-///
-/// - Normal uploads: handler spools multipart to temp file → `upload_file_streaming`
-/// - Chunked uploads: chunks already on disk → `upload_file_from_path`
-/// - WebDAV PUT (new): handler streams to temp file → `update_file_streaming`
-/// - WebDAV PUT (small/compat): `create_file` / `update_file` spool internally
+/// **All upload paths converge on streaming-into-the-chunk-store** — content
+/// never passes through this port; it is ingested by the interface layer
+/// (CDC chunking + hashing while the body arrives, no spool file) and only
+/// the resulting [`StoredBlob`] reference travels through here.
 pub trait FileUploadUseCase: Send + Sync + 'static {
-    /// Upload from a temp file already on disk (true streaming, ~256 KB RAM).
+    /// Register a new file row pointing at an already-ingested blob.
     ///
-    /// When `pre_computed_hash` is `Some`, the blob store skips the hash
-    /// re-read — the handler already computed it during the multipart spool.
+    /// Takes ownership of the blob's reference (released on failure).
     async fn upload_file_streaming(
         &self,
         name: String,
         folder_id: Option<String>,
         content_type: String,
-        temp_path: &Path,
-        size: u64,
-        pre_computed_hash: Option<String>,
+        blob: StoredBlob,
     ) -> Result<FileDto, DomainError>;
 
-    /// Upload from a file already assembled on disk (chunked uploads).
+    /// Replace the content of the file at `path` with an already-ingested
+    /// blob, or create the file when it doesn't exist (WebDAV/WOPI PUT).
     ///
-    /// Same as `upload_file_streaming` but with a separate name for clarity.
-    async fn upload_file_from_path(
-        &self,
-        name: String,
-        folder_id: Option<String>,
-        content_type: String,
-        file_path: &Path,
-        pre_computed_hash: Option<String>,
-    ) -> Result<FileDto, DomainError>;
-
-    /// Creates a new file at the specified path (for WebDAV)
-    async fn create_file(
-        &self,
-        parent_path: &str,
-        filename: &str,
-        content: &[u8],
-        content_type: &str,
-    ) -> Result<FileDto, DomainError>;
-
-    /// Updates the content of an existing file (for WebDAV)
-    async fn update_file(
-        &self,
-        path: &str,
-        content: &[u8],
-        content_type: &str,
-        modified_at: Option<i64>,
-    ) -> Result<FileDto, DomainError>;
-
-    /// Streaming update — spools body to a temp file with incremental hash,
-    /// then atomically replaces the file content via dedup store.
-    ///
-    /// Peak RAM: ~256 KB regardless of file size.
-    /// Used by WebDAV PUT for large files.
+    /// Takes ownership of the blob's reference (released on failure).
     async fn update_file_streaming(
         &self,
         path: &str,
-        temp_path: &Path,
-        size: u64,
+        blob: StoredBlob,
         content_type: &str,
-        pre_computed_hash: Option<String>,
         modified_at: Option<i64>,
     ) -> Result<FileDto, DomainError>;
 }

src/application/ports/storage_ports.rs

@@ -251,21 +251,21 @@ pub struct CopyFolderTreeResult {
 
 /// Secondary port for file **writing**.
 ///
-/// Covers: upload (buffered + streaming), move, delete, update,
-/// and deferred registration for the write-behind cache.
+/// Covers: upload registration, move, delete, update, and deferred
+/// registration for the write-behind cache.
 pub trait FileWritePort: Send + Sync + 'static {
-    /// Streaming upload — saves a file from a temp file already on disk.
+    /// Register a file row pointing at a blob already stored in the
+    /// content-addressable chunk store.
     ///
-    /// When `pre_computed_hash` is provided, the dedup service skips the
-    /// hash re-read — zero extra I/O beyond the initial spool.
-    async fn save_file_from_temp(
+    /// Takes ownership of one blob reference: on any failure the reference
+    /// is released before the error is returned.
+    async fn save_file_with_blob(
         &self,
         name: String,
         folder_id: Option<String>,
         content_type: String,
-        temp_path: &std::path::Path,
+        blob_hash: &str,
         size: u64,
-        pre_computed_hash: Option<String>,
     ) -> Result<File, DomainError>;
 
     /// Moves a file to another folder.
@@ -281,22 +281,20 @@ pub trait FileWritePort: Send + Sync + 'static {
     /// Deletes a file.
     async fn delete_file(&self, id: &str) -> Result<(), DomainError>;
 
-    /// Streaming update — replaces file content from a temp file on disk.
+    /// Atomically swap a file's content to a blob already stored in the
+    /// content-addressable chunk store.
     ///
-    /// When `pre_computed_hash` is provided, the dedup service skips the
-    /// hash re-read — zero extra I/O beyond the initial spool.
-    /// Peak RAM: ~256 KB regardless of file size.
+    /// Takes ownership of one blob reference (released on failure); the
+    /// previous content's reference is dropped after the swap.
     ///
     /// Returns `(new_blob_hash, updated_at_epoch)` — everything a caller
     /// needs to rebuild the fresh entity/ETag from a `File` it already
     /// holds, without re-reading the row it just updated.
-    async fn update_file_content_from_temp(
+    async fn update_file_content_with_blob(
         &self,
         file_id: &str,
-        temp_path: &std::path::Path,
+        blob_hash: &str,
         size: u64,
-        content_type: Option<String>,
-        pre_computed_hash: Option<String>,
         modified_at: Option<i64>,
     ) -> Result<(String, i64), DomainError>;