Two fixes in dedup_service.rs:
1. Rename error path (L234): std::fs::remove_file → tokio::fs::remove_file
Restructured from map_err closure to match block since .await
cannot be used inside a sync closure.
2. Integrity verify (L691): fused blocking .exists() + async metadata()
into a single fs::metadata().await call. Eliminates one stat()
syscall per blob AND removes the only remaining blocking I/O from
the verify_integrity hot loop (buffer_unordered × VERIFY_CONCURRENCY).
- Replace fetch_all() with fetch() streaming cursor in verify_integrity
so memory stays O(batch=16) instead of O(total_blobs)
- Replace fetch_all() with fetch() streaming cursor in garbage_collect
so memory stays O(1) instead of O(orphans)
- Add TryStreamExt import for try_next() on cursors
Eliminates OOM risk with millions of blobs — RAM usage is now constant
regardless of table size.
Replace sequential blob-by-blob SHA-256 verification with
futures::stream::buffer_unordered(16) to hash up to 16 blobs
concurrently.
Each hash_file() already runs on spawn_blocking, so 16 concurrent
verifications saturate both disk I/O queue and CPU cores.
Before: 10K blobs × 13ms = ~130s (NVMe) — 1 core, 1 I/O in flight
After: 10K blobs / 16 concurrency = ~8s — 16 cores, 16 I/O in flight
Standardize code formatting across all 173 Rust source files
using rustfmt. No functional changes - purely cosmetic.
This establishes a consistent code style baseline for the
project going forward.
Dionisio