fix(upload): stop browser connection exhaustion that froze folder uploads

With WASM finally enabled, large files (e.g. 32 MB logs) started running the
delta worker, which opens SEVERAL concurrent requests each (overlapping
negotiate batches + chunk PUTs). A few of those running at once blew past the
browser's ~6 connections-per-host limit, so plain uploads of the small files
queued with zero bytes sent until the 30 s stall watchdog cancelled them — the
upload "stuck at 4% / 94%" with N (pending) XHRs in the Network panel. The
session-refresh request got starved too (the spurious 401s).

- Raise the delta-worker threshold to 64 MB (new DELTA_WORKER_MIN_SIZE) so
  typical large files take a single-connection plain upload. Delta's payoff is
  sub-file dedup on RE-upload; on a first upload it is pure connection overhead.
  Client-side instant-hashing still only reads files < 8 MB into memory.
- Lower upload concurrency 3 -> 2, leaving headroom under the 6-connection
  budget for session refresh/poll and the occasional delta worker.

npm run check: clean, 58 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
DioCrafts
2026-06-21 02:33:16 +02:00
parent 9534dfa103
commit 13b59aabd8
2 changed files with 21 additions and 7 deletions
+15 -2
View File
@@ -11,9 +11,22 @@ import { getCsrfToken } from '$lib/api/csrf';
import { createFileByHash, dedupCheckBatch } from '$lib/api/endpoints/files';
import { blake3HexOfFile } from '$lib/vendor/hashWasm';
/** Files smaller than this skip delta: the round-trips cost more than the bytes. */
/** Files smaller than this skip delta: the round-trips cost more than the bytes.
* Also the upper bound for client-side whole-file hashing (instant by-hash
* uploads) — we never read a file larger than this fully into memory. */
export const DELTA_UPLOAD_MIN_SIZE = 8 * 1024 * 1024;
/** Only files at least this large actually run the delta worker. A delta worker
* opens SEVERAL concurrent requests (overlapping `negotiate` batches + chunk
* PUTs); a few running at once exhaust the browser's ~6 connections-per-host
* budget and starve plain uploads (they queue, then the upload watchdog cancels
* them — the "stuck at N%" folder upload). Typical large files (e.g. tens of MB)
* therefore go through a single-connection plain upload; delta is reserved for
* genuinely huge files, where chunked, resumable transfer earns its keep and few
* run concurrently. (Delta's real payoff — sub-file dedup — only helps on
* re-upload anyway, not the first upload that dominates these batches.) */
const DELTA_WORKER_MIN_SIZE = 64 * 1024 * 1024;
const DELTA_WORKER_URL = '/workers/deltaWorker.js';
const DELTA_TIMEOUT_BASE_MS = 120_000;
const DELTA_TIMEOUT_PER_GB_MS = 90_000;
@@ -59,7 +72,7 @@ export function tryDeltaUpload(
): Promise<DeltaUploadAnswer | null> {
if (
!folderId ||
file.size < DELTA_UPLOAD_MIN_SIZE ||
file.size < DELTA_WORKER_MIN_SIZE ||
usable === false ||
typeof Worker === 'undefined'
) {