Claude
0fab4ce17d
Phase 0 of the delta-sync plan. Re-uploading a file the user already has (another device, a restore, a duplicate) used to transfer every byte just for the server to discard them as a dedup hit. The frontend now computes the file's BLAKE3 locally and, on a hit, registers the file with a single ~150-byte metadata call. Server — POST /api/files/by-hash: - All checks live in the application service per the AuthZ rule: Create permission on the target folder via the authorization engine, hash ownership via the existing user-scoped query (a non-owned hash returns 404 — same shape as "no such blob" — and emits an instant_upload.rejected audit event), quota on the logical size. - On success: one ref_count bump + the existing save_file_with_blob row registration (compensation included); is_new_blob=false so lifecycle hooks skip thumbnail regeneration. ~10 ms warm. - The storage-usage service is now built before the application services and injected, instead of only living on AppState. Client — WASM BLAKE3 + worker: - wasm/oxicloud-hash: the exact same blake3 crate the server uses, compiled with WASM SIMD128 (~660 MB/s measured) so browser hashes match server content addresses bit for bit. Built by scripts/build-wasm.sh; the artifacts (45 KB wasm + 8 KB glue) are vendored like pdf.js — no npm dependencies, no wasm toolchain needed for regular builds. - static/js/workers/hashWorker.js streams the File in 8 MiB slices off the main thread (constant RAM at any file size). - features/files/instantUpload.js orchestrates: threshold (8 MiB — below it the round-trips cost more than the bytes), user-scoped /api/dedup/check, by-hash registration, and silent fallback to the normal byte upload on any miss, race or unsupported environment. Wired into both uploadFiles and uploadFolderEntries. - biome.json vendors exclusion fixed to cover nested directories (previous vendors were .mjs and never matched the *.js include). Verified end-to-end against PostgreSQL 16: node-driven WASM hash equals the server's content_hash for a 20 MB file; by-hash returns 201 in ~10 ms warm with a 151-byte request (vs 20,971,873 bytes for the byte upload); the copy downloads byte-identical and the manifest ref_count goes 1→2; a second user probing the same hash gets exists:false and 404 plus the audit line; duplicate name → 409, malformed hash → 400; worker and wasm are served with correct MIME (application/wasm). https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
40 lines
1.4 KiB
Bash
Executable File
40 lines
1.4 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
# Rebuild the vendored BLAKE3 WASM module (static/js/vendors/hash-wasm/).
|
||
#
|
||
# The generated artifacts ARE committed — like the other vendored modules
|
||
# (pdf.js) — so regular frontend/backend builds never need the wasm
|
||
# toolchain. Re-run this script only when wasm/oxicloud-hash/ changes
|
||
# (e.g. bumping the blake3 crate, or adding FastCDC for the delta-sync
|
||
# client) and commit the regenerated files.
|
||
#
|
||
# Requirements (one-time):
|
||
# rustup target add wasm32-unknown-unknown
|
||
# cargo install wasm-bindgen-cli --locked
|
||
#
|
||
# wasm-bindgen-cli's version must match the crate's `wasm-bindgen`
|
||
# dependency; cargo prints a clear error when they drift.
|
||
|
||
set -euo pipefail
|
||
cd "$(dirname "$0")/.."
|
||
|
||
CRATE=wasm/oxicloud-hash
|
||
OUT=static/js/vendors/hash-wasm
|
||
|
||
# SIMD128 is baseline in every evergreen browser (Chrome 91+, Firefox 89+,
|
||
# Safari 16.4+) and is worth ~3-4× in hashing throughput. Browsers without
|
||
# it fail instantiation; the frontend detects that and falls back to a
|
||
# plain byte upload.
|
||
RUSTFLAGS="-C target-feature=+simd128" \
|
||
cargo build \
|
||
--manifest-path "$CRATE/Cargo.toml" \
|
||
--target wasm32-unknown-unknown \
|
||
--release
|
||
|
||
wasm-bindgen \
|
||
--target web \
|
||
--no-typescript \
|
||
--out-dir "$OUT" \
|
||
"$CRATE/target/wasm32-unknown-unknown/release/oxicloud_hash_wasm.wasm"
|
||
|
||
echo "Vendored: $(ls -la "$OUT" | tail -n +2 | awk '{print $9, "("$5" bytes)"}' | xargs)"
|