Instant upload: register already-owned content by hash, zero bytes on the wire

Phase 0 of the delta-sync plan. Re-uploading a file the user already has
(another device, a restore, a duplicate) used to transfer every byte just
for the server to discard them as a dedup hit. The frontend now computes
the file's BLAKE3 locally and, on a hit, registers the file with a single
~150-byte metadata call.

Server — POST /api/files/by-hash:
- All checks live in the application service per the AuthZ rule:
  Create permission on the target folder via the authorization engine,
  hash ownership via the existing user-scoped query (a non-owned hash
  returns 404 — same shape as "no such blob" — and emits an
  instant_upload.rejected audit event), quota on the logical size.
- On success: one ref_count bump + the existing save_file_with_blob row
  registration (compensation included); is_new_blob=false so lifecycle
  hooks skip thumbnail regeneration. ~10 ms warm.
- The storage-usage service is now built before the application services
  and injected, instead of only living on AppState.

Client — WASM BLAKE3 + worker:
- wasm/oxicloud-hash: the exact same blake3 crate the server uses,
  compiled with WASM SIMD128 (~660 MB/s measured) so browser hashes match
  server content addresses bit for bit. Built by scripts/build-wasm.sh;
  the artifacts (45 KB wasm + 8 KB glue) are vendored like pdf.js — no
  npm dependencies, no wasm toolchain needed for regular builds.
- static/js/workers/hashWorker.js streams the File in 8 MiB slices off
  the main thread (constant RAM at any file size).
- features/files/instantUpload.js orchestrates: threshold (8 MiB — below
  it the round-trips cost more than the bytes), user-scoped
  /api/dedup/check, by-hash registration, and silent fallback to the
  normal byte upload on any miss, race or unsupported environment.
  Wired into both uploadFiles and uploadFolderEntries.
- biome.json vendors exclusion fixed to cover nested directories
  (previous vendors were .mjs and never matched the *.js include).

Verified end-to-end against PostgreSQL 16: node-driven WASM hash equals
the server's content_hash for a 20 MB file; by-hash returns 201 in ~10 ms
warm with a 151-byte request (vs 20,971,873 bytes for the byte upload);
the copy downloads byte-identical and the manifest ref_count goes 1→2;
a second user probing the same hash gets exists:false and 404 plus the
audit line; duplicate name → 409, malformed hash → 400; worker and wasm
are served with correct MIME (application/wasm).

https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
This commit is contained in:
Claude
2026-06-11 13:54:32 +00:00
parent 944c833787
commit 0fab4ce17d
17 changed files with 1209 additions and 61 deletions
+98
View File
@@ -0,0 +1,98 @@
//! BLAKE3 for the OxiCloud web frontend.
//!
//! Compiled from the same `blake3` crate the server uses, so a hash
//! computed in the browser equals the server's content address bit for
//! bit — the property the instant-upload path depends on.
//!
//! The API is incremental on purpose: the worker feeds the file in
//! slices (`Blob.slice().arrayBuffer()`), keeping RAM constant no matter
//! how large the file is.
use wasm_bindgen::prelude::*;
/// Incremental BLAKE3 hasher.
///
/// ```js
/// const h = new Blake3Hasher();
/// h.update(chunkBytes); // repeat per slice
/// const hex = h.finalizeHex();
/// ```
#[wasm_bindgen]
pub struct Blake3Hasher {
inner: blake3::Hasher,
}
#[wasm_bindgen]
impl Blake3Hasher {
/// Create a fresh hasher.
#[wasm_bindgen(constructor)]
pub fn new() -> Blake3Hasher {
Blake3Hasher {
inner: blake3::Hasher::new(),
}
}
/// Feed one slice of the file.
pub fn update(&mut self, data: &[u8]) {
self.inner.update(data);
}
/// Finish and return the lowercase hex digest (64 chars). The hasher
/// can keep receiving `update` calls afterwards (BLAKE3 finalization
/// is non-destructive), but the frontend treats it as terminal.
#[wasm_bindgen(js_name = finalizeHex)]
pub fn finalize_hex(&self) -> String {
self.inner.finalize().to_hex().to_string()
}
/// Bytes hashed so far — lets the worker report progress without
/// tracking its own counter.
pub fn count(&self) -> f64 {
self.inner.count() as f64
}
}
impl Default for Blake3Hasher {
fn default() -> Self {
Self::new()
}
}
/// One-shot convenience for small buffers.
#[wasm_bindgen(js_name = blake3Hex)]
pub fn blake3_hex(data: &[u8]) -> String {
blake3::hash(data).to_hex().to_string()
}
#[cfg(test)]
mod tests {
use super::*;
/// The vector the frontend smoke test uses — also proves the wasm
/// build hashes identically to the server (same crate, same output).
#[test]
fn hello_world_vector() {
let hasher = {
let mut h = Blake3Hasher::new();
h.update(b"Hello, ");
h.update(b"World!");
h
};
assert_eq!(
hasher.finalize_hex(),
"288a86a79f20a3d6dccdca7713beaed178798296bdfa7913fa2a62d9727bf8f8"
);
assert_eq!(
blake3_hex(b"Hello, World!"),
"288a86a79f20a3d6dccdca7713beaed178798296bdfa7913fa2a62d9727bf8f8"
);
}
#[test]
fn empty_input_vector() {
assert_eq!(
blake3_hex(b""),
"af1349b9f5f9a1a6a0404dea36dcc9499bcb25c9adc112b7cc9a93cae41f3262"
);
}
}