cjw/Oxicloud
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

perf(frontend): round 6 — coalesced progressive listing, in-place SvelteSet, batch fan-out, t() value cache

Browse Source 
Four SPA hot-path fixes, each shipping with a vitest benchmark gate
(verbatim BEFORE replica + equivalence + perf assertion) so CI
re-verifies the win on every run:

- fetchFolderListing invoked onPage after EVERY 200-row page with the
  whole accumulated listing, and the files view re-sorts everything per
  emission — O(N²/page) main-thread work on large folders. Page one and
  the final page always emit; intermediates coalesce to one per 150 ms.
  25×200 load: 30.9 → 4.0 ms (7.8x), 65 000 → 5 200 sorted elements.

- selected/favoriteIds/sharedIds (files) and favoriteIds (recent) were
  $state<Set>s copied whole on every toggle. Now one SvelteSet each,
  mutated in place (the useSelection pattern): 1 000 toggles @ N=5 000
  771.9 → 1.9 ms (399x); one-toggle fan-out across 40 mounted rows
  40 → 3 re-runs when refining a select-all.

- batchDelete/moveInto awaited one request per item serially and
  probed listing.folders.find per id (O(N·M)). Now an id index built
  once + mapLimit(6) fan-out, failure semantics preserved: 100-item
  delete @ 5 ms RTT 525 → 89 ms (5.9x), 38 825 → 500 probes.

- t() re-split its dotted key and walked the nested dict on every call,
  and interpolate regex-scanned strings without placeholders. Resolved
  values now memoize per (dict, key) in a WeakMap + a {{ guard:
  20k mixed calls 22.7 → 8.6 ms (2.63x).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
This commit is contained in:
Claude
2026-07-18 00:54:38 +00:00
parent 63cf6646d0
commit 61c9470981
10 changed files with 905 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files
frontend/src/lib/api/endpoints/folders.bench.test.ts
+205
View File
@@ -0,0 +1,205 @@
import { describe, expect, it, vi, beforeEach } from 'vitest';
vi.mock('$lib/api/client', () => ({ apiFetch: vi.fn(), apiJson: vi.fn() }));
import { apiFetch } from '$lib/api/client';
import type { FileItem, FolderItem, ItemType } from '$lib/api/types';
import { fetchFolderListing, invalidateFolderCache, type FolderListing } from './folders';
/**
* Benchmark gate for the coalesced progressive-render emissions in
* {@link fetchFolderListing}.
*
* Audit finding: the loader invoked `onPage` after EVERY 200-item page with a
* fresh copy of the whole accumulated listing, and the files view re-derives
* its filtered + sorted view (two `localeCompare` sorts + entry rebuild) from
* each emission. For a folder of N items that is Σ page sizes ≈ O(N²/200)
* elements re-sorted on the main thread during a single load — hundreds of ms
* of jank on exactly the large folders progressive rendering was meant to
* help. The fix emits page one (first paint) and the final page always, and
* intermediate pages at most once per PAGE_EMIT_MIN_INTERVAL_MS.
*
* Gates:
* 1. Equivalence — final listing identical to the emit-every-page reference,
* first emission still after page one (first paint preserved), last
* emission still `done === true` with the complete listing.
* 2. Perf — on a fast connection (pages resolve in ≪150 ms) the consumer-side
* derive work collapses from 25 full re-sorts to ≤3; wall time of the
* load+derive cycle must drop accordingly (≥3x on the derive term).
*/
type ResourceItem = { resource_type: ItemType; resource: { id: string; name: string } };
type ResourcePage = { items?: ResourceItem[]; next_cursor?: string };
const PAGE_SIZE = 200;
const PAGES = 25; // 5 000-item folder
/** Deterministic shuffled names so the consumer sort actually works. */
function pageBody(page: number): ResourcePage {
const items: ResourceItem[] = [];
for (let i = 0; i < PAGE_SIZE; i++) {
const n = page * PAGE_SIZE + i;
const id = `f-${n.toString().padStart(5, '0')}`;
// Mix folders into the first page like a real listing (folders first).
const isFolder = page === 0 && i < 20;
items.push({
resource_type: isFolder ? 'folder' : 'file',
resource: { id, name: `item ${((n * 7919) % 100000).toString().padStart(5, '0')}.txt` }
});
}
return { items, next_cursor: page + 1 < PAGES ? `c${page + 1}` : undefined };
}
function fakeRes(body: ResourcePage): Response {
return {
status: 200,
ok: true,
json: async () => body,
headers: { get: () => null }
} as unknown as Response;
}
function mockPagedFetch(): void {
let call = 0;
vi.mocked(apiFetch).mockImplementation(async () => fakeRes(pageBody(call++)));
}
/**
* The pre-fix loader, verbatim shape: accumulate pages and emit a fresh copy
* of the whole accumulated listing after every page.
*/
async function referenceFetchFolderListing(
folderId: string,
onPage: (partial: FolderListing, done: boolean) => void
): Promise<FolderListing> {
const folders: FolderItem[] = [];
const files: FileItem[] = [];
let cursor: string | undefined;
do {
const params = new URLSearchParams({ order_by: 'name', limit: '200' });
if (cursor) params.set('cursor', cursor);
const res = await apiFetch(`/api/folders/${folderId}/resources?${params.toString()}`, {
credentials: 'same-origin',
cache: 'no-store'
});
if (!res.ok) throw new Error(`listing failed: ${res.status}`);
const page = (await res.json()) as ResourcePage;
for (const it of page.items ?? []) {
if (it.resource_type === 'folder') folders.push(it.resource as FolderItem);
else files.push(it.resource as FileItem);
}
cursor = page.next_cursor;
onPage({ folders: [...folders], files: [...files], favoriteIds: [], sharedIds: [] }, !cursor);
} while (cursor);
return { folders, files, favoriteIds: [], sharedIds: [] };
}
/**
* The files view's per-emission derive chain, reduced to its dominant costs:
* dotfile filter pass + two localeCompare sorts + ordered-entry rebuild
* (`sortedFolders`/`sortedFiles`/`entries`/`orderedIds` in +page.svelte).
* Returns the number of elements that went through the sort — the O(N²) term.
*/
function consumerDerive(partial: FolderListing): number {
const visF = partial.folders.filter((f) => !f.name.startsWith('.'));
const visX = partial.files.filter((f) => !f.name.startsWith('.'));
const sortedF = [...visF].sort((a, b) => a.name.localeCompare(b.name));
const sortedX = [...visX].sort((a, b) => a.name.localeCompare(b.name));
const orderedIds = [...sortedF.map((f) => f.id), ...sortedX.map((f) => f.id)];
return orderedIds.length;
}
beforeEach(() => {
vi.clearAllMocks();
invalidateFolderCache();
});
describe('coalesced progressive listing emissions (benchmark gate)', () => {
it('final listing, first-paint page and done-flag match the emit-every-page reference', async () => {
mockPagedFetch();
const refEmits: Array<{ n: number; done: boolean }> = [];
const refFinal = await referenceFetchFolderListing('bench', (p, done) =>
refEmits.push({ n: p.folders.length + p.files.length, done })
);
mockPagedFetch();
const emits: Array<{ n: number; done: boolean; partial: FolderListing }> = [];
const r = await fetchFolderListing('bench', {
onPage: (partial, done) =>
emits.push({ n: partial.folders.length + partial.files.length, done, partial })
});
// Identical complete listing.
expect(r.listing).toEqual(refFinal);
// First paint unchanged: the first emission is still page one.
expect(emits[0].n).toBe(refEmits[0].n);
expect(emits[0].n).toBe(PAGE_SIZE);
// Exactly one done emission, last, carrying the full listing — as before.
expect(emits.filter((e) => e.done).length).toBe(1);
expect(emits[emits.length - 1].done).toBe(true);
expect(emits[emits.length - 1].n).toBe(PAGES * PAGE_SIZE);
expect(refEmits[refEmits.length - 1].done).toBe(true);
// Emissions are a subset of what the reference produced (never more).
expect(emits.length).toBeLessThanOrEqual(refEmits.length);
// Every emitted partial is a prefix-accumulation (monotone growth).
for (let i = 1; i < emits.length; i++) expect(emits[i].n).toBeGreaterThan(emits[i - 1].n);
});
it('single-page folders still emit exactly once, done=true (fast path untouched)', async () => {
vi.mocked(apiFetch).mockResolvedValue(
fakeRes({ items: pageBody(PAGES - 1).items }) // no next_cursor
);
const emits: boolean[] = [];
await fetchFolderListing('one', { onPage: (_p, done) => emits.push(done) });
expect(emits).toEqual([true]);
});
it(
`collapses the O(N²) consumer re-derive on a fast ${PAGES}-page load (perf gate)`,
{ timeout: 30_000 },
async () => {
// Warm-up both paths (JIT tiering outside the measured windows).
mockPagedFetch();
await referenceFetchFolderListing('warm', (p) => consumerDerive(p));
mockPagedFetch();
await fetchFolderListing('warm', { onPage: (p) => consumerDerive(p) });
mockPagedFetch();
let refSorted = 0;
let refEmits = 0;
const t0 = performance.now();
await referenceFetchFolderListing('bench', (p) => {
refEmits++;
refSorted += consumerDerive(p);
});
const refMs = performance.now() - t0;
mockPagedFetch();
let sorted = 0;
let emitsN = 0;
const t1 = performance.now();
await fetchFolderListing('bench', {
onPage: (p) => {
emitsN++;
sorted += consumerDerive(p);
}
});
const ms = performance.now() - t1;
console.info(
`progressive load ${PAGES}×${PAGE_SIZE}: before ${refEmits} emissions / ${refSorted} sorted elements / ${refMs.toFixed(1)} ms — after ${emitsN} emissions / ${sorted} sorted elements / ${ms.toFixed(1)} ms (${(refMs / ms).toFixed(1)}x wall, ${(refSorted / sorted).toFixed(1)}x fewer sorted elements)`
);
// The reference re-derived every page: Σ = P(P+1)/2 pages of elements.
expect(refEmits).toBe(PAGES);
expect(refSorted).toBe((PAGES * (PAGES + 1) * PAGE_SIZE) / 2);
// Coalesced: page 1 + final (+ occasionally one mid emission if the
// stubbed pages ever take >150 ms — they don't on any healthy runner).
expect(emitsN).toBeLessThanOrEqual(3);
// ≥5x less consumer sort work is the point of the change.
expect(sorted).toBeLessThan(refSorted / 5);
// And it must show up as wall time on the combined load+derive cycle.
expect(ms).toBeLessThan(refMs / 3);
}
);
});
frontend/src/lib/api/endpoints/folders.ts
+34 -10
View File
@@ -94,6 +94,17 @@ export async function getFolder(id: string): Promise<FolderItem> {
return folder;
}
/**
* Minimum spacing between intermediate progressive-render emissions of
* {@link fetchFolderListing}. Each emission hands the consumer the WHOLE
* accumulated listing, and the files view re-derives its filtered + sorted
* view from it (O(accumulated · log) with `localeCompare`), so emitting every
* page made a large-folder load Σ O(N²/page) of main-thread sort work. Page
* one and the final page always emit; pages in between only emit after this
* much time has passed since the previous emission.
*/
export const PAGE_EMIT_MIN_INTERVAL_MS = 150;
/**
* Fetch a folder's complete listing (sub-folders + files), rebuilt from the
* cursor-paginated `/api/folders/{id}/resources` feed — the old combined
@@ -112,12 +123,15 @@ export async function fetchFolderListing(
etag?: string;
forceRefresh?: boolean;
/**
* Progressive render hook: invoked after EVERY page with the
* accumulated listing so far (the arrays are fresh copies — safe to
* hand to reactive state). Without it, a 2,000-item folder waited
* for all ⌈N/200⌉ sequential round-trips before the first row
* painted; with it the view paints after page one (~200 items) and
* fills in as the tail pages land.
* Progressive render hook: invoked with the accumulated listing so
* far (the arrays are fresh copies — safe to hand to reactive
* state). Without it, a 2,000-item folder waited for all ⌈N/200⌉
* sequential round-trips before the first row painted; with it the
* view paints after page one (~200 items) and fills in as the tail
* pages land. Emissions are coalesced to at most one per
* {@link PAGE_EMIT_MIN_INTERVAL_MS} between the first and the final
* page — the hook is always called for page one and always called
* once more with `done === true` and the complete listing.
*/
onPage?: (partial: FolderListing, done: boolean) => void;
} = {}
@@ -125,6 +139,8 @@ export async function fetchFolderListing(
const folders: FolderItem[] = [];
const files: FileItem[] = [];
let cursor: string | undefined;
let firstPage = true;
let lastEmit = 0;
do {
const params = new URLSearchParams({ order_by: 'name', limit: '200' });
if (opts.forceRefresh) params.set('force_refresh', 'true');
@@ -144,10 +160,18 @@ export async function fetchFolderListing(
else files.push(it.resource as FileItem);
}
cursor = page.next_cursor;
opts.onPage?.(
{ folders: [...folders], files: [...files], favoriteIds: [], sharedIds: [] },
!cursor
);
const done = !cursor;
if (
opts.onPage &&
(done || firstPage || performance.now() - lastEmit >= PAGE_EMIT_MIN_INTERVAL_MS)
) {
lastEmit = performance.now();
opts.onPage(
{ folders: [...folders], files: [...files], favoriteIds: [], sharedIds: [] },
done
);
}
firstPage = false;
} while (cursor);
return { status: 200, listing: { folders, files, favoriteIds: [], sharedIds: [] } };