Instant upload: register already-owned content by hash, zero bytes on the wire

Phase 0 of the delta-sync plan. Re-uploading a file the user already has
(another device, a restore, a duplicate) used to transfer every byte just
for the server to discard them as a dedup hit. The frontend now computes
the file's BLAKE3 locally and, on a hit, registers the file with a single
~150-byte metadata call.

Server — POST /api/files/by-hash:
- All checks live in the application service per the AuthZ rule:
  Create permission on the target folder via the authorization engine,
  hash ownership via the existing user-scoped query (a non-owned hash
  returns 404 — same shape as "no such blob" — and emits an
  instant_upload.rejected audit event), quota on the logical size.
- On success: one ref_count bump + the existing save_file_with_blob row
  registration (compensation included); is_new_blob=false so lifecycle
  hooks skip thumbnail regeneration. ~10 ms warm.
- The storage-usage service is now built before the application services
  and injected, instead of only living on AppState.

Client — WASM BLAKE3 + worker:
- wasm/oxicloud-hash: the exact same blake3 crate the server uses,
  compiled with WASM SIMD128 (~660 MB/s measured) so browser hashes match
  server content addresses bit for bit. Built by scripts/build-wasm.sh;
  the artifacts (45 KB wasm + 8 KB glue) are vendored like pdf.js — no
  npm dependencies, no wasm toolchain needed for regular builds.
- static/js/workers/hashWorker.js streams the File in 8 MiB slices off
  the main thread (constant RAM at any file size).
- features/files/instantUpload.js orchestrates: threshold (8 MiB — below
  it the round-trips cost more than the bytes), user-scoped
  /api/dedup/check, by-hash registration, and silent fallback to the
  normal byte upload on any miss, race or unsupported environment.
  Wired into both uploadFiles and uploadFolderEntries.
- biome.json vendors exclusion fixed to cover nested directories
  (previous vendors were .mjs and never matched the *.js include).

Verified end-to-end against PostgreSQL 16: node-driven WASM hash equals
the server's content_hash for a 20 MB file; by-hash returns 201 in ~10 ms
warm with a 151-byte request (vs 20,971,873 bytes for the byte upload);
the copy downloads byte-identical and the manifest ref_count goes 1→2;
a second user probing the same hash gets exists:false and 404 plus the
audit line; duplicate name → 409, malformed hash → 400; worker and wasm
are served with correct MIME (application/wasm).

https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
This commit is contained in:
Claude
2026-06-11 13:54:32 +00:00
parent 944c833787
commit 0fab4ce17d
17 changed files with 1209 additions and 61 deletions
@@ -69,6 +69,57 @@ impl FileHandler {
}
}
/// Instant upload: create a file from a blob the caller already owns.
///
/// Zero content bytes travel — the client proved possession of the
/// content by hash (it computed BLAKE3 locally and confirmed via
/// `GET /api/dedup/check/{hash}`), so the server only bumps the blob's
/// reference count and registers the metadata row.
///
/// All authorization (folder Create permission, hash ownership with
/// anti-enumeration, quota) lives in the application service.
pub(super) async fn create_file_by_hash_impl(
State(state): State<GlobalState>,
auth_user: AuthUser,
Json(request): Json<CreateFileByHashRequest>,
) -> impl IntoResponse {
// Hash shape check — same contract as /api/dedup/check/{hash}.
if request.hash.len() != 64 || !request.hash.chars().all(|c| c.is_ascii_hexdigit()) {
return AppError::bad_request(
"Invalid hash format. Expected BLAKE3 (64 hex characters)",
)
.into_response();
}
// Basename only — same path-traversal guard as the multipart upload.
let filename = request
.name
.rsplit('/')
.next()
.unwrap_or(&request.name)
.rsplit('\\')
.next()
.unwrap_or(&request.name)
.to_string();
if filename.is_empty() {
return AppError::bad_request("File name must not be empty").into_response();
}
match state
.applications
.file_upload_service
.create_file_from_owned_blob_with_perms(
auth_user.id,
filename,
request.folder_id,
&request.hash,
)
.await
{
Ok(file) => Self::created_json_response(&file).into_response(),
Err(err) => Self::domain_error_response(err).into_response(),
}
}
/// Core upload logic shared by [`Self::upload_file`] and
/// [`Self::upload_file_with_thumbnails`].
///
@@ -1039,6 +1090,39 @@ pub async fn upload_file_with_thumbnails(
FileHandler::upload_file_with_thumbnails_impl(state, auth_user, multipart).await
}
/// Request body for the instant-upload endpoint.
#[derive(Debug, Deserialize, ToSchema)]
pub struct CreateFileByHashRequest {
/// File name to create (path components are stripped).
pub name: String,
/// Target folder ID (the caller needs Create permission on it).
pub folder_id: String,
/// BLAKE3 hash (64 hex chars) of content the caller already owns.
pub hash: String,
}
#[utoipa::path(
post,
path = "/api/files/by-hash",
request_body = CreateFileByHashRequest,
responses(
(status = 201, description = "File created from an already-owned blob — zero bytes transferred", body = FileDto),
(status = 400, description = "Invalid hash format or empty name"),
(status = 404, description = "No owned blob with this hash (anti-enumeration: same shape as unknown hash)"),
(status = 409, description = "A file with this name already exists in the folder"),
(status = 507, description = "Storage quota exceeded"),
),
security(("bearerAuth" = [])),
tag = "files"
)]
pub async fn create_file_by_hash(
state: State<GlobalState>,
auth_user: AuthUser,
request: Json<CreateFileByHashRequest>,
) -> impl IntoResponse {
FileHandler::create_file_by_hash_impl(state, auth_user, request).await
}
#[utoipa::path(
get,
path = "/api/files/{id}",