perf(blob): one syscall per new chunk in write_blob_bytes (create_new)
Replace the try_exists(stat) + File::create(O_CREAT|O_TRUNC) pair with a single OpenOptions::create_new (O_CREAT|O_EXCL), treating AlreadyExists as the existing idempotent skip. One metadata syscall per new chunk instead of two (each a spawn_blocking round-trip), and O_EXCL closes the check-then-create TOCTOU the old pair left open (a racing writer could be truncated). Honest measurement caveat (benches/BLOB-WRITE.md): the wall-clock throughput effect is BELOW the noise floor of the test environment — three 9-rep interleaved runs on the same ext4 device swing −12%..+21% at the 256 KiB CDC size, because a negative stat on a warm dentry cache is ~µs, dwarfed by the chunk's create+write+flush. So this is justified as a code-quality / correctness change (canonical idiom, strictly fewer syscalls, closes a TOCTOU, zero downside), NOT as a benchmarked perf win. The sibling idea — reusing the File handle for the fsync sweep — is deliberately NOT done: sync_blobs is a single end-of-stream sweep over all the upload's new hashes, so retaining handles would hold thousands of FDs open (>ulimit) on a large upload. The re-open sweep is a deliberate FD-frugal design. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5yYZ9s868mJwqT2Qz7ez
This commit is contained in:
@@ -128,12 +128,29 @@ async fn fsync_paths_parallel(paths: Vec<PathBuf>, strict: bool) -> Result<(), D
|
||||
/// (fsync now vs. deferred batch sync), or `None` when the blob already
|
||||
/// existed (idempotent skip — content-addressed, so identical by definition).
|
||||
async fn write_blob_bytes(blob_path: &Path, data: &Bytes) -> Result<Option<File>, DomainError> {
|
||||
if fs::try_exists(blob_path).await.unwrap_or(false) {
|
||||
return Ok(None);
|
||||
}
|
||||
let mut file = fs::File::create(blob_path).await.map_err(|e| {
|
||||
DomainError::internal_error("Blob", format!("Failed to create blob file: {}", e))
|
||||
})?;
|
||||
// `create_new` = `open(O_CREAT | O_EXCL)`: ONE syscall that both tests for
|
||||
// existence and creates, replacing the old `try_exists` (stat) + `File::create`
|
||||
// (two metadata syscalls, each its own `spawn_blocking` round-trip on Tokio's
|
||||
// blocking pool). On a large upload of unique data that's thousands of stats
|
||||
// saved. The blob is content-addressed, so an already-present file is
|
||||
// byte-identical by definition → `AlreadyExists` is exactly the idempotent
|
||||
// skip the old `try_exists` branch performed — and O_EXCL closes the TOCTOU
|
||||
// window the check-then-create pair left open (no truncate-over-a-racing-writer).
|
||||
let mut file = match fs::OpenOptions::new()
|
||||
.write(true)
|
||||
.create_new(true)
|
||||
.open(blob_path)
|
||||
.await
|
||||
{
|
||||
Ok(file) => file,
|
||||
Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => return Ok(None),
|
||||
Err(e) => {
|
||||
return Err(DomainError::internal_error(
|
||||
"Blob",
|
||||
format!("Failed to create blob file: {}", e),
|
||||
));
|
||||
}
|
||||
};
|
||||
file.write_all(data).await.map_err(|e| {
|
||||
DomainError::internal_error("Blob", format!("Failed to write blob from bytes: {}", e))
|
||||
})?;
|
||||
|
||||
Reference in New Issue
Block a user