cjw/Oxicloud
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

feat(upload): idempotent re-upload + auto-retry so partial folders self-complete

Browse Source 
Re-uploading a partially-uploaded folder used to surface hundreds of spurious
"already exists" failures, and a file the watchdog aborted (or one the server
committed just before the client gave up) was lost.

Backend — save_file_with_blob_impl (the shared write path for both plain and
by-hash uploads): on a name conflict (23505), if the existing non-trashed file
holds byte-identical content (same folder, same name, same blob hash), return
that file as success instead of erroring. A different-content clash still
conflicts. Re-upload / re-sync becomes a clean no-op for everything already
stored — only the genuinely missing files transfer.

Frontend — uploadWithRetry: each file gets one automatic retry on a transient
failure (quota is never retried). With backend idempotency, retrying an
already-stored file is an instant no-op and a stalled/aborted file gets a real
second chance, so a folder upload self-completes instead of leaving gaps.

cargo test: 448 passed. npm run check: clean, 58 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
DioCrafts
2026-06-20 18:56:05 +02:00
parent b58b2d8f95
commit e6ee5988ab
2 changed files with 140 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
frontend/src/routes/files/[...path]/+page.svelte
+31 -15
View File
@@ -348,12 +348,35 @@
return 0;
}
/**
* Upload one file, retrying once on a transient failure. A connection the
* watchdog aborted, or an upload the server actually committed before the
* client gave up, both recover here: the backend treats a re-upload of
* byte-identical content as success (idempotent), so the retry is a clean
* no-op for anything that already landed and a real second chance for the
* rest. A quota error is never retried — the disk won't free up mid-batch.
*/
async function uploadWithRetry(
folderId: string | null,
file: File,
report: (frac: number) => void,
ownedHash: string | null
): Promise<number> {
try {
return await withTimeout(uploadOneFile(folderId, file, report, ownedHash), FILE_BACKSTOP_MS);
} catch (e) {
if ((e as { isQuota?: boolean } | null)?.isQuota) throw e;
report(0); // reset this file's progress for the second attempt
return await withTimeout(uploadOneFile(folderId, file, report, ownedHash), FILE_BACKSTOP_MS);
}
}
/**
* Upload `items` ({file, folderId}) with bounded concurrency, a per-file
* deadline and live aggregate progress. A stuck or failing file no longer
* freezes the batch: it blocks only its own lane (the rest keep going) and
* eventually times out / is skipped. Quota exhaustion stops the run early.
* Returns the bytes deduplicated and the count of files that failed.
* freezes the batch: it blocks only its own lane (the rest keep going), is
* retried once, and only then counted as failed. Quota exhaustion stops the
* run early. Returns the bytes deduplicated and the count of files that failed.
*/
async function uploadAll(
items: { file: File; folderId: string | null }[],
@@ -377,19 +400,12 @@
while (next < total) {
const i = next++;
const { file, folderId } = items[i];
const report = (f: number) => {
if (!Number.isNaN(f)) frac[i] = Math.min(1, f);
refresh();
};
try {
savedBytes += await withTimeout(
uploadOneFile(
folderId,
file,
(f) => {
if (!Number.isNaN(f)) frac[i] = Math.min(1, f);
refresh();
},
owned.get(file) ?? null
),
FILE_BACKSTOP_MS
);
savedBytes += await uploadWithRetry(folderId, file, report, owned.get(file) ?? null);
} catch (e) {
failures++;
// A full disk won't recover within this batch — stop pulling new