diff --git a/Cargo.toml b/Cargo.toml index 50376e8a..c03a0171 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -350,6 +350,23 @@ name = "bench_micro_allocs" path = "examples/bench_micro_allocs.rs" required-features = ["bench"] +# Round-7 battery ───────────────────────────────────────────────────────────── + +# Range-seek per-request authz duplication — the per-seek require the range +# branch used to run (warm CPU + cold drive-resolve query) vs 0 after routing +# through the non-perms range read (needs the dev Postgres up). +[[example]] +name = "bench_range_seek_authz" +path = "examples/bench_range_seek_authz.rs" +required-features = ["bench"] + +# `/api/folders/{id}/resources` row→DTO mapping — per-row name clone vs move +# (pure CPU; counting allocator). +[[example]] +name = "bench_resource_row_map" +path = "examples/bench_resource_row_map.rs" +required-features = ["bench"] + # Round-6 battery ───────────────────────────────────────────────────────────── # CardDAV whole-book REPORT/PROPFIND — buffered double-residency vs cursor diff --git a/benches/ROUND7.md b/benches/ROUND7.md new file mode 100644 index 00000000..61d5f163 --- /dev/null +++ b/benches/ROUND7.md @@ -0,0 +1,146 @@ +# Round 7 — photo timeline O(N²) → incremental, range-seek authz duplication, row-map clone + +Benchmark-gated changes, same rule as ROUND2-6: every change ships with a +BEFORE/AFTER benchmark; an AFTER that doesn't beat its BEFORE gets rolled +back. Equivalence gates (identical output / byte-identical responses) guard +every behavior-preserving rewrite. Frontend changes carry vitest benchmark +gates (verbatim BEFORE replica + equivalence + perf assertion) committed +beside the code so CI re-verifies the win on every run. + +Measured on 4 cores / 15 GiB, local PostgreSQL 16 (fsync off), release +profile; frontend on Node 22 / vitest 4 (jsdom). Reproduce any row with the +command in its section. + +## Summary + +| # | change | key metric | before → after | +|--:|---|---|---| +| 1 | Photos timeline incremental grouping/layout | 50-page (3k-photo) scroll drain | 76 500 → 3 000 group ops (**25.5x**) / 23.0 → 2.2 ms (**10.6x**) | +| 2 | Range-seek per-request authz duplication removed | per-seek authz on a shared-drive scrub | WARM 0.67 → 0 µs/seek; **COLD 1362.66 → 0 µs/seek** (a drive-resolve query per seek) | +| 3 | `/resources` row→DTO name clone → move | allocs/row (500-row page) | 10.004 → 9.004 (**500 allocs saved**, 1.00/row) | + +## [1] Photos timeline — O(N²) re-group + re-layout per page → incremental builder + +The photos view appended each 60-item page with `items = [...items, ...page]` +and re-derived both `groups` (O(N), a `new Date()` per photo) and `photoRows` +(O(N) row layout) over the whole accumulated list on every page — so paging to +photo N re-grouped + re-laid-out everything loaded so far, Σ ≈ O(N²/60) of +main-thread work during the scroll (the exact class ROUND6 fixed for the files +listing). The DOM was already windowed (`VirtualRows`); this was the derivation +feeding it. + +Because photos arrive newest-first (`media_sort_date DESC`), grouping is +append-only: a page only ever extends the last date bucket or adds buckets +after it, never mutates an earlier group. The new `PhotoTimeline` +(`lib/utils/photoTimeline.ts`) exploits that — an append re-buckets only the +fresh page and re-lays-out only the groups that changed, reusing every +untouched group's cached rows; any other change (config, deletion, filter +toggle, non-append) falls back to a full rebuild. The pure `buildPhotoRows` is +the verbatim reference the gate holds it equal to. + +Gates: the incremental output is deep-equal to `buildPhotoRows` at EVERY page +of the drain (both square + justified layouts); config-change / deletion / +width=0 fall back to a correct full rebuild; grouping work collapses ≥5x and +wall ≥3x. + +``` +cd frontend && npx vitest run src/lib/utils/photoTimeline.bench.test.ts --disable-console-intercept +# photo timeline 50×60: before 76500 timestamp reads / 23.0 ms +# after 3000 timestamp reads / 2.2 ms +# (25.5x fewer grouping ops, 10.6x wall) +``` + +## [2] Range downloads — duplicate per-seek authz + access-notify removed + +`download_file_impl` resolves the file once via `get_file_with_perms` (authz + +access-notify + metadata), then the Range branch called +`get_file_range_preloaded_with_perms`, which re-ran `require_file` (authz) + +`notify_file_accessed` per request. Media players and PDF viewers fetch a file +*exclusively* through Range requests — a `bytes=0-` probe then one request per +seek — so every seek in a scrub re-authorized a file the request-level gate had +already cleared. The share-landing and WebDAV range paths already authorize +once then read via the non-perms `get_file_range_preloaded`; the REST handler +now does the same (and the now-unused `_with_perms` range method is deleted). + +Safety: the request-level `get_file_with_perms` still gates every request +(denies before the Range branch runs), so the removed per-seek re-check +bypasses nothing — the bench asserts the member is granted and a non-member +denied. + +``` +cargo run --release --features bench --example bench_range_seek_authz +# seeks/scrub=200 (member of a shared drive, viewer grant) +# arm wall ms µs/seek +# BEFORE per-seek (WARM) 0.13 0.67 <- moka hit + uuid parse, removed +# BEFORE per-seek (COLD) 272.53 1362.66 <- a grant-cascade drive-resolve +# QUERY per seek, removed +# AFTER per-seek (removed) 0.00 0.00 +# A 200-seek scrub of a shared video stops paying ~272 ms of authz queries +# when the drive-role cache is cold (cross-drive recipient, or 30 s TTL expiry +# mid-scrub). notify_file_accessed (a throttled hook call) is likewise removed +# per seek. +``` + +## [3] `/api/folders/{id}/resources` row→DTO mapping — clone name → move name + +The listing maps each owned `FolderResourceRow` into a DTO but cloned +`row.name` into it (`name: row.name.clone()`) — one avoidable `String` heap +alloc per listed folder/file. The folder branch uses fixed icon classes, so +`row.name` is simply moved; the file branch computes its name-derived icon / +category classes first (they borrow `&row.name`), then moves `row.name` in. One +fewer alloc per row, identical output. + +``` +cargo run --release --features bench --example bench_resource_row_map +# rows=500 +# arm allocs wall ms allocs/row +# BEFORE (clone) 5002 0.841 10.004 +# AFTER (move) 4502 0.810 9.004 +# Saved 500 allocs (1.00/row) — the per-row name clone removed; output identical. +``` + +## Deferred / flagged (not shipped this round) + +- **Thumbnail ACL-before-304 (security posture — needs maintainer decision).** + `get_thumbnail_impl` runs `require_permission(Read)` before the ETag-304 and + moka/disk short-circuits, so a shared-album recipient pays a grant-cascade + query per thumbnail revalidation. Moving authz *after* the cache would make + thumbnails "authorized at creation time only" — a user whose access was + revoked could still fetch cached thumbnails of files they once could see. + That is a deliberate security-posture change, not a perf tweak; left for a + security review. The safe alternative (back the non-owner authz with the + existing `drive_role_cache`, or a `Borrow` cache key that removes the + per-request `to_string`) is queued for round 8 with an alloc/query bench. +- **`batch_operations` `Arc` → `String` per item.** `copy_file_with_perms` + / `move_file_with_perms` take `Option`, so the batch path's + `target_folder: Arc` is re-`to_string()`-ed per item, defeating the + Arc. Widening those `_with_perms` signatures to `Option<&str>` touches the + trait + impl + stub + ~7 call sites — a contained refactor better done + deliberately with its own alloc bench; queued for round 8. +- **List-view O(N²) re-derive (favorites / recent / trash / shared-with-me / + shared swimlanes).** Same class as [1] but on typically-smaller lists; + each infinite-scroll page re-derives `entries` / `byId` / `sections` / + `lanes` over the full accumulated set. Deferred — the incremental-builder + cost isn't yet justified at those sizes; revisit if any surface reaches + thousands of rows. +- **Serial independent DB pairs → `join!` (token refresh, login, cross-drive + move, CardDAV discovery, NC PROPFIND enrichment).** Overlapping independent + round-trips saves 1 RTT *under real PG latency*, but the ROUND6 authz-fan-out + rejection showed the overhead can wash the win out on local-socket PG. These + need a decide-by-bench with an injected-latency arm (like the ROUND6 `::text` + A/B) before adoption — queued for round 8, not guessed at here. + +## Correctness-adjacent (surfaced by the round-7 hunt — not perf, flagged for follow-up) + +- **`fetchFolderListing` returns empty `favoriteIds`/`sharedIds`** + (`frontend/src/lib/api/endpoints/folders.ts`) since the combined `/listing` + route was removed — the files-grid star/shared badges are seeded empty on + every navigation. The same removal also dropped the 304 conditional + fast-path, so a folder navigation now pages the full body (`cache: no-store`) + instead of a bodiless 304 on unchanged folders (mitigated only by the + in-memory `folderCache`). Functional regression, not perf. +- **Search page lacks a stale-response guard** + (`frontend/src/routes/search/+page.svelte`): the query `$effect` awaits + `searchFiles` with no `seq`/AbortController, so a slow stale query can + resolve after and clobber a newer one. The files view's `loadSeq` is the + pattern to mirror. diff --git a/examples/bench_range_seek_authz.rs b/examples/bench_range_seek_authz.rs new file mode 100644 index 00000000..50d5b700 --- /dev/null +++ b/examples/bench_range_seek_authz.rs @@ -0,0 +1,283 @@ +//! Range-seek per-request authz duplication benchmark. +//! +//! `download_file_impl` calls `get_file_with_perms` once (authz + access +//! notify + metadata) and THEN, in the Range branch, called +//! `get_file_range_preloaded_with_perms` — which re-ran `require_file` +//! (authz) + `notify_file_accessed` per request. Media players and PDF +//! viewers fetch a file *exclusively* through Range requests: a `bytes=0-` +//! probe then one request per seek. So every seek in a scrub re-authorized a +//! file the request-level gate had already cleared. +//! +//! Round 7 drops the range branch to the non-perms `get_file_range_preloaded` +//! (the share-landing and WebDAV range paths already do exactly this). This +//! bench isolates the per-seek `require` that AFTER eliminates, driving the +//! REAL `PgAclEngine`: +//! - WARM: the cache the initial `get_file_with_perms` warmed — each removed +//! seek-check was a moka hit + uuid parse (pure CPU/alloc). +//! - COLD: a shared-drive recipient whose drive-role cache expired mid-scrub +//! (30 s TTL) — each removed seek-check was a full drive-resolve query. +//! +//! Safety gate: the surviving request-level gate still authorizes correctly — +//! the member is granted, a non-member is denied — so removing the per-seek +//! re-check bypasses nothing. +//! +//! Run (needs Postgres up; reads DATABASE_URL from .env): +//! cargo run --release --features bench --example bench_range_seek_authz +//! Tunables (env): BENCH_SEEKS (200), BENCH_POOL (8). + +use std::env; +use std::sync::Arc; +use std::time::{Duration, Instant}; + +use oxicloud::application::ports::authorization_ports::AuthorizationEngine; +use oxicloud::domain::services::authorization::{Permission, Resource, Subject}; +use oxicloud::infrastructure::repositories::pg::{ + FileBlobReadRepository, FolderDbRepository, SubjectGroupPgRepository, +}; +use oxicloud::infrastructure::services::dedup_service::DedupService; +use oxicloud::infrastructure::services::local_blob_backend::LocalBlobBackend; +use oxicloud::infrastructure::services::pg_acl_engine::PgAclEngine; +use sqlx::PgPool; +use sqlx::postgres::PgPoolOptions; +use uuid::Uuid; + +fn env_or(key: &str, default: T) -> T { + env::var(key) + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(default) +} + +struct Seeded { + member: Uuid, + outsider: Uuid, + drive_id: Uuid, + root_folder: Uuid, + blob_hash: String, + file_id: Uuid, +} + +async fn seed(pool: &PgPool) -> Seeded { + let mut tx = pool.begin().await.expect("begin"); + let member: Uuid = sqlx::query_scalar( + "INSERT INTO auth.users (username, email, role) + VALUES ('bench_rangeseek', 'bench_rangeseek@bench.invalid', 'user') RETURNING id", + ) + .fetch_one(&mut *tx) + .await + .expect("seed member"); + let outsider: Uuid = sqlx::query_scalar( + "INSERT INTO auth.users (username, email, role) + VALUES ('bench_rangeseek_out', 'bench_rangeseek_out@bench.invalid', 'user') RETURNING id", + ) + .fetch_one(&mut *tx) + .await + .expect("seed outsider"); + + let drive_id: Uuid = + sqlx::query_scalar("INSERT INTO storage.drives (kind) VALUES ('shared') RETURNING id") + .fetch_one(&mut *tx) + .await + .expect("seed drive"); + let root_folder: Uuid = sqlx::query_scalar( + "INSERT INTO storage.folders (name, path, lpath, drive_id) + VALUES ('Bench Seek', '/Bench Seek', 'x', $1) RETURNING id", + ) + .bind(drive_id) + .fetch_one(&mut *tx) + .await + .expect("seed folder"); + sqlx::query("UPDATE storage.drives SET root_folder_id = $1 WHERE id = $2") + .bind(root_folder) + .bind(drive_id) + .execute(&mut *tx) + .await + .expect("stamp root"); + sqlx::query( + "INSERT INTO storage.role_grants + (subject_type, subject_id, resource_type, resource_id, role, granted_by) + VALUES ('user', $1, 'drive', $2, 'viewer'::storage.grant_role, $1)", + ) + .bind(member) + .bind(drive_id) + .execute(&mut *tx) + .await + .expect("seed grant"); + + let blob_hash = "benchrangeseek00000000000000000000000000000000000000000000000b3".to_string(); + sqlx::query("INSERT INTO storage.blobs (hash, size, ref_count) VALUES ($1, 1048576, 1)") + .bind(&blob_hash) + .execute(&mut *tx) + .await + .expect("seed blob"); + let file_id: Uuid = sqlx::query_scalar( + "INSERT INTO storage.files (name, folder_id, blob_hash, size, mime_type, drive_id) + VALUES ('clip.mp4', $1, $2, 1048576, 'video/mp4', $3) RETURNING id", + ) + .bind(root_folder) + .bind(&blob_hash) + .bind(drive_id) + .fetch_one(&mut *tx) + .await + .expect("seed file"); + tx.commit().await.expect("commit"); + Seeded { + member, + outsider, + drive_id, + root_folder, + blob_hash, + file_id, + } +} + +async fn cleanup(pool: &PgPool, s: &Seeded) { + let _ = sqlx::query("DELETE FROM storage.role_grants WHERE resource_id = $1") + .bind(s.drive_id) + .execute(pool) + .await; + let _ = sqlx::query("DELETE FROM storage.files WHERE drive_id = $1") + .bind(s.drive_id) + .execute(pool) + .await; + let _ = sqlx::query("DELETE FROM storage.drives WHERE id = $1") + .bind(s.drive_id) + .execute(pool) + .await; + let _ = sqlx::query("DELETE FROM storage.folders WHERE id = $1") + .bind(s.root_folder) + .execute(pool) + .await; + let _ = sqlx::query("DELETE FROM storage.blobs WHERE hash = $1") + .bind(&s.blob_hash) + .execute(pool) + .await; + let _ = sqlx::query("DELETE FROM auth.users WHERE id IN ($1, $2)") + .bind(s.member) + .bind(s.outsider) + .execute(pool) + .await; +} + +fn fresh_engine(pool: &Arc) -> Arc { + let folder_repo = Arc::new(FolderDbRepository::new(pool.clone())); + let backend = Arc::new(LocalBlobBackend::new(std::path::Path::new( + "/tmp/bench-rangeseek-blobs", + ))); + let dedup = Arc::new(DedupService::new(backend, pool.clone(), pool.clone())); + let file_repo = Arc::new(FileBlobReadRepository::new( + pool.clone(), + dedup, + folder_repo.clone(), + )); + let group_repo = Arc::new(SubjectGroupPgRepository::new(pool.clone())); + Arc::new(PgAclEngine::new( + pool.clone(), + folder_repo, + file_repo, + group_repo, + )) +} + +/// The per-seek check the range branch used to run (verbatim: uuid parse + +/// `authz.require`, exactly `require_file`'s body). +async fn seek_require(engine: &Arc, caller: Uuid, file_id: Uuid) -> bool { + engine + .require( + Subject::User(caller), + Permission::Read, + Resource::File(file_id), + ) + .await + .is_ok() +} + +#[tokio::main(flavor = "multi_thread")] +async fn main() { + dotenvy::dotenv().ok(); + let url = env::var("DATABASE_URL") + .or_else(|_| env::var("OXICLOUD_DB_CONNECTION_STRING")) + .expect("set DATABASE_URL — the dev Postgres URL"); + let seeks: usize = env_or("BENCH_SEEKS", 200); + let pool_size: u32 = env_or("BENCH_POOL", 8); + + let pool = Arc::new( + PgPoolOptions::new() + .max_connections(pool_size) + .min_connections(pool_size) + .acquire_timeout(Duration::from_secs(10)) + .connect(&url) + .await + .expect("connect Postgres"), + ); + + let s = seed(&pool).await; + + // ── Safety gate: the surviving request-level gate authorizes correctly ── + let gate = fresh_engine(&pool); + let member_ok = seek_require(&gate, s.member, s.file_id).await; + let outsider_denied = !seek_require(&gate, s.outsider, s.file_id).await; + if !member_ok || !outsider_denied { + eprintln!( + "SAFETY GATE FAILED: member_ok={member_ok} outsider_denied={outsider_denied} \ + (the single request-level authz must still grant the member and deny the outsider)" + ); + cleanup(&pool, &s).await; + std::process::exit(1); + } + + println!("\n#################################################################"); + println!("# range-seek authz duplication: per-seek require (BEFORE) vs 0 (AFTER)"); + println!("# seeks/scrub={seeks} (member of a shared drive, viewer grant)"); + println!("#################################################################\n"); + println!("| {:<26} | {:>10} | {:>12} |", "arm", "wall ms", "µs/seek"); + + // WARM: one require warms owner_cache + drive_role_cache (as the handler's + // get_file_with_perms does), then the scrub's per-seek re-checks are moka + // hits — pure CPU/alloc the AFTER path removes. + { + let engine = fresh_engine(&pool); + seek_require(&engine, s.member, s.file_id).await; // warm + let t = Instant::now(); + for _ in 0..seeks { + std::hint::black_box(seek_require(&engine, s.member, s.file_id).await); + } + let el = t.elapsed(); + println!( + "| {:<26} | {:>10.2} | {:>12.2} |", + "BEFORE per-seek (WARM)", + el.as_secs_f64() * 1e3, + el.as_secs_f64() * 1e6 / seeks as f64 + ); + } + + // COLD: a fresh engine per seek models a cross-drive recipient or a + // drive-role-cache entry that expired mid-scrub (30 s TTL) — each removed + // re-check was a full grant-cascade drive-resolve query. + { + let t = Instant::now(); + for _ in 0..seeks { + let engine = fresh_engine(&pool); + std::hint::black_box(seek_require(&engine, s.member, s.file_id).await); + } + let el = t.elapsed(); + println!( + "| {:<26} | {:>10.2} | {:>12.2} |", + "BEFORE per-seek (COLD)", + el.as_secs_f64() * 1e3, + el.as_secs_f64() * 1e6 / seeks as f64 + ); + } + + println!( + "| {:<26} | {:>10.2} | {:>12.2} |", + "AFTER per-seek (removed)", 0.0, 0.0 + ); + + cleanup(&pool, &s).await; + println!("\n(AFTER runs zero per-seek authz: the request-level get_file_with_perms"); + println!(" already authorized + recorded the access. WARM = the moka/CPU cost removed"); + println!(" per seek; COLD = the drive-resolve query removed per seek when the cache"); + println!(" isn't warm. notify_file_accessed (a throttled hook call) is likewise"); + println!(" removed per seek. Safety gate: member granted, outsider denied.)"); +} diff --git a/examples/bench_resource_row_map.rs b/examples/bench_resource_row_map.rs new file mode 100644 index 00000000..0f85c960 --- /dev/null +++ b/examples/bench_resource_row_map.rs @@ -0,0 +1,282 @@ +//! `/api/folders/{id}/resources` row→DTO mapping micro-alloc benchmark. +//! +//! The listing maps each `FolderResourceRow` into a `FolderResourceItemDto`. +//! BEFORE cloned `row.name` into the DTO (`name: row.name.clone()`) even +//! though the row is owned by the mapping closure — one avoidable `String` +//! heap alloc per listed folder/file. AFTER computes the name-derived icon / +//! category classes first (they borrow `&row.name`), then MOVES `row.name` +//! into the DTO — the same output, one fewer alloc per row. +//! +//! Run: +//! cargo run --release --features bench --example bench_resource_row_map +//! Tunables (env): BENCH_ROWS (500). + +use std::alloc::{GlobalAlloc, Layout, System}; +use std::env; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::time::Instant; + +use chrono::{DateTime, TimeZone, Utc}; +use oxicloud::application::dtos::display_helpers::{ + category_for, format_file_size, icon_class_for, icon_special_class_for, intern_display, + intern_mime, +}; +use oxicloud::application::dtos::file_dto::FileDto; +use oxicloud::application::dtos::folder_dto::{FolderDto, FolderResourceRow}; +use oxicloud::domain::entities::file::File; +use uuid::Uuid; + +static ALLOC_CALLS: AtomicU64 = AtomicU64::new(0); + +struct CountingAlloc; + +unsafe impl GlobalAlloc for CountingAlloc { + unsafe fn alloc(&self, layout: Layout) -> *mut u8 { + ALLOC_CALLS.fetch_add(1, Ordering::Relaxed); + unsafe { System.alloc(layout) } + } + unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) { + unsafe { System.dealloc(ptr, layout) } + } + unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 { + ALLOC_CALLS.fetch_add(1, Ordering::Relaxed); + unsafe { System.realloc(ptr, layout, new_size) } + } + unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { + ALLOC_CALLS.fetch_add(1, Ordering::Relaxed); + unsafe { System.alloc_zeroed(layout) } + } +} + +#[global_allocator] +static GLOBAL: CountingAlloc = CountingAlloc; + +fn env_or(key: &str, default: T) -> T { + env::var(key) + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(default) +} + +fn rows(n: usize) -> Vec { + let ts: DateTime = Utc.timestamp_opt(1_700_000_000, 0).unwrap(); + (0..n) + .map(|i| { + let is_folder = i % 4 == 0; + FolderResourceRow { + resource_type: if is_folder { "folder" } else { "file" }.to_string(), + id: Uuid::new_v4(), + name: if is_folder { + format!("Folder {i:05}") + } else { + format!("document-{i:05}.pdf") + }, + parent_id: Some(Uuid::new_v4()), + mime_type: if is_folder { + None + } else { + Some("application/pdf".to_string()) + }, + size: if is_folder { -1 } else { 4096 }, + created_at: ts, + modified_at: ts, + drive_id: Uuid::new_v4(), + blob_hash: if is_folder { + None + } else { + Some("a".repeat(64)) + }, + sort_str: format!("row {i}"), + type_order: 0, + folder_first: if is_folder { 0 } else { 1 }, + } + }) + .collect() +} + +/// (name, icon_class, category) triple extracted from each produced DTO — the +/// fields the move-vs-clone touches. Used for the equivalence gate. +type Probe = (String, std::sync::Arc, std::sync::Arc); + +/// BEFORE — verbatim: `name: row.name.clone()` in both branches. +fn map_before(rows: Vec) -> Vec { + rows.into_iter() + .map(|row| { + if row.resource_type == "folder" { + let resource_id = row.id.to_string(); + let dto = FolderDto { + etag: resource_id.clone(), + id: resource_id, + name: row.name.clone(), + path: String::new(), + parent_id: row.parent_id.map(|u| u.to_string()), + drive_id: row.drive_id, + created_at: row.created_at.timestamp() as u64, + modified_at: row.modified_at.timestamp() as u64, + is_root: false, + icon_class: intern_display("fas fa-folder"), + icon_special_class: intern_display("folder-icon"), + category: intern_display("Folder"), + created_by: None, + updated_by: None, + }; + (dto.name, dto.icon_class, dto.category) + } else { + let mime = row + .mime_type + .as_deref() + .unwrap_or("application/octet-stream"); + let size_bytes = row.size.max(0) as u64; + let modified_at_u = row.modified_at.timestamp() as u64; + let content_hash = row.blob_hash.clone().unwrap_or_default(); + let etag = if content_hash.is_empty() { + String::new() + } else { + File::compute_etag(&content_hash, modified_at_u) + }; + let dto = FileDto { + id: row.id.to_string(), + name: row.name.clone(), + path: String::new(), + size: size_bytes, + mime_type: intern_mime(mime), + folder_id: row.parent_id.map(|u| u.to_string()), + created_at: row.created_at.timestamp() as u64, + modified_at: row.modified_at.timestamp() as u64, + icon_class: intern_display(icon_class_for(&row.name, mime)), + icon_special_class: intern_display(icon_special_class_for(&row.name, mime)), + category: intern_display(category_for(&row.name, mime)), + size_formatted: format_file_size(size_bytes), + sort_date: None, + content_hash, + etag, + created_by: None, + updated_by: None, + }; + (dto.name, dto.icon_class, dto.category) + } + }) + .collect() +} + +/// AFTER — icons/category first (borrow `&row.name`), then move `row.name`. +fn map_after(rows: Vec) -> Vec { + rows.into_iter() + .map(|row| { + if row.resource_type == "folder" { + let resource_id = row.id.to_string(); + let dto = FolderDto { + etag: resource_id.clone(), + id: resource_id, + name: row.name, + path: String::new(), + parent_id: row.parent_id.map(|u| u.to_string()), + drive_id: row.drive_id, + created_at: row.created_at.timestamp() as u64, + modified_at: row.modified_at.timestamp() as u64, + is_root: false, + icon_class: intern_display("fas fa-folder"), + icon_special_class: intern_display("folder-icon"), + category: intern_display("Folder"), + created_by: None, + updated_by: None, + }; + (dto.name, dto.icon_class, dto.category) + } else { + let mime = row + .mime_type + .as_deref() + .unwrap_or("application/octet-stream"); + let size_bytes = row.size.max(0) as u64; + let modified_at_u = row.modified_at.timestamp() as u64; + let content_hash = row.blob_hash.clone().unwrap_or_default(); + let etag = if content_hash.is_empty() { + String::new() + } else { + File::compute_etag(&content_hash, modified_at_u) + }; + let icon_class = intern_display(icon_class_for(&row.name, mime)); + let icon_special_class = intern_display(icon_special_class_for(&row.name, mime)); + let category = intern_display(category_for(&row.name, mime)); + let dto = FileDto { + id: row.id.to_string(), + name: row.name, + path: String::new(), + size: size_bytes, + mime_type: intern_mime(mime), + folder_id: row.parent_id.map(|u| u.to_string()), + created_at: row.created_at.timestamp() as u64, + modified_at: row.modified_at.timestamp() as u64, + icon_class, + icon_special_class, + category, + size_formatted: format_file_size(size_bytes), + sort_date: None, + content_hash, + etag, + created_by: None, + updated_by: None, + }; + (dto.name, dto.icon_class, dto.category) + } + }) + .collect() +} + +fn main() { + let n: usize = env_or("BENCH_ROWS", 500); + + // Equivalence gate: identical (name, icon_class, category) for every row. + if map_before(rows(n)) != map_after(rows(n)) { + eprintln!("EQUIVALENCE GATE FAILED: mapping output differs"); + std::process::exit(1); + } + + // Warm the string interner so its first-sight allocs sit outside the + // measured windows (they're identical for both arms anyway). + std::hint::black_box(map_before(rows(n))); + std::hint::black_box(map_after(rows(n))); + + let a0 = ALLOC_CALLS.load(Ordering::Relaxed); + let t = Instant::now(); + std::hint::black_box(map_before(rows(n))); + let before_ms = t.elapsed().as_secs_f64() * 1e3; + let before_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0; + + let a1 = ALLOC_CALLS.load(Ordering::Relaxed); + let t = Instant::now(); + std::hint::black_box(map_after(rows(n))); + let after_ms = t.elapsed().as_secs_f64() * 1e3; + let after_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1; + + // Both arms build the same `rows(n)` input inside the timed window, so the + // input allocs are equal and cancel in the delta; the difference is the + // per-row name clone the AFTER path avoids. + println!("\n#################################################################"); + println!("# resources row→DTO mapping: clone name vs move name"); + println!("# rows={n}"); + println!("#################################################################\n"); + println!( + "| {:<20} | {:>12} | {:>10} | {:>14} |", + "arm", "allocs", "wall ms", "allocs/row" + ); + println!( + "| {:<20} | {:>12} | {:>10.3} | {:>14.3} |", + "BEFORE (clone)", + before_allocs, + before_ms, + before_allocs as f64 / n as f64 + ); + println!( + "| {:<20} | {:>12} | {:>10.3} | {:>14.3} |", + "AFTER (move)", + after_allocs, + after_ms, + after_allocs as f64 / n as f64 + ); + println!( + "\nSaved {} allocs ({:.2}/row) — the per-row name clone removed.", + before_allocs.saturating_sub(after_allocs), + (before_allocs.saturating_sub(after_allocs)) as f64 / n as f64 + ); +} diff --git a/frontend/src/lib/utils/photoTimeline.bench.test.ts b/frontend/src/lib/utils/photoTimeline.bench.test.ts new file mode 100644 index 00000000..bedf1e32 --- /dev/null +++ b/frontend/src/lib/utils/photoTimeline.bench.test.ts @@ -0,0 +1,163 @@ +import { describe, expect, it } from 'vitest'; +import type { PhotoItem } from '$lib/api/endpoints/photos'; +import { + PhotoTimeline, + buildPhotoRows, + type GroupMode, + type LayoutMode, + type TimelineConfig +} from './photoTimeline'; + +/** + * Benchmark gate for the incremental photo timeline (PhotoTimeline) that + * replaced the photos view's `groups`→`photoRows` derive chain. + * + * Audit finding: `loadMore` does `items = [...items, ...page]` (60/page), and + * both `groups` (O(N), a `new Date()` per photo) and `photoRows` (O(N) row + * layout) are `$derived` over the whole accumulated list — so paging to photo + * N re-groups + re-lays-out everything loaded so far, Σ ≈ O(N²/60) main-thread + * work during the scroll (the same class ROUND6 fixed for the files listing). + * Since pages arrive newest-first, grouping is append-only; PhotoTimeline + * re-buckets only the fresh page and re-lays-out only the groups that changed. + * + * Gates: + * 1. Equivalence — at EVERY page of the drain, the incremental output is + * deep-equal to the verbatim full-rebuild reference (buildPhotoRows), for + * both layouts; plus config-change, deletion and width=0 fall back to a + * correct full rebuild. + * 2. Perf — grouping work (timestamp reads) collapses from Σ O(N²/60) to O(N) + * across the drain (deterministic count), and wall drops ≥3x. + */ + +const DAY = 86_400; // seconds + +/** A photo with a descending sort_date and a deterministic aspect ratio. */ +function photo(i: number): PhotoItem { + // Newest-first: photo 0 is most recent; ~half a day apart spans ~4 years + // over 3k photos, so month/day buckets are bounded (realistic library). + const sortDate = 1_700_000_000 - i * (DAY / 2); + const w = 200 + ((i * 37) % 400); + const h = 200 + ((i * 53) % 300); + return { + category: 'image', + created_at: sortDate, + icon_class: '', + icon_special_class: '', + id: `p-${i.toString().padStart(6, '0')}`, + mime_type: 'image/jpeg', + modified_at: sortDate, + name: `photo ${i}.jpg`, + created_by: null, + updated_by: null, + folder_id: 'f', + path: `/photo ${i}.jpg`, + size: 1000, + size_formatted: '1 KB', + sort_date: sortDate, + etag: `e${i}`, + content_hash: `h${i}`, + width: w, + height: h + } as PhotoItem; +} + +/** Instrumented config: counts every timestamp read (the grouping hot op). */ +function makeConfig( + groupMode: GroupMode, + layoutMode: LayoutMode, + width: number, + counter?: { n: number } +): TimelineConfig { + const timestampOf = (p: PhotoItem) => { + if (counter) counter.n++; + const v = p.sort_date || p.created_at || 0; + return v < 1e12 ? v * 1000 : v; + }; + // Stable label fn (reference identity matters for the config-unchanged path). + const labelOf = (d: Date, mode: GroupMode) => + mode === 'year' + ? `${d.getFullYear()}` + : mode === 'month' + ? `${d.getFullYear()}-${d.getMonth() + 1}` + : `${d.getFullYear()}-${d.getMonth() + 1}-${d.getDate()}`; + return { groupMode, layoutMode, width, mobile: false, timestampOf, labelOf }; +} + +const PAGE = 60; +const PAGES = 50; // 3 000-photo drain +const WIDTH = 1200; + +describe('incremental photo timeline (benchmark gate)', () => { + for (const layout of ['square', 'justified'] as LayoutMode[]) { + it(`stays deep-equal to the full rebuild at every page — ${layout}`, () => { + const all = Array.from({ length: PAGE * PAGES }, (_, i) => photo(i)); + const cfg = makeConfig('month', layout, WIDTH); + const timeline = new PhotoTimeline(); + for (let p = 1; p <= PAGES; p++) { + const cumulative = all.slice(0, p * PAGE); + const incremental = timeline.sync(cumulative, cfg); + const reference = buildPhotoRows(cumulative, cfg); + expect(incremental, `page ${p}`).toEqual(reference); + } + }); + } + + it('falls back to a correct full rebuild on config change, deletion and width=0', () => { + const all = Array.from({ length: 600 }, (_, i) => photo(i)); + const timeline = new PhotoTimeline(); + const monthSquare = makeConfig('month', 'square', WIDTH); + + // Drain a few pages, then flip layout — must equal a fresh full rebuild. + timeline.sync(all.slice(0, 300), monthSquare); + const justified = makeConfig('month', 'justified', WIDTH); + expect(timeline.sync(all.slice(0, 300), justified)).toEqual( + buildPhotoRows(all.slice(0, 300), justified) + ); + + // Change group mode. + const yearJust = makeConfig('year', 'justified', WIDTH); + expect(timeline.sync(all.slice(0, 300), yearJust)).toEqual( + buildPhotoRows(all.slice(0, 300), yearJust) + ); + + // Deletion (list shrinks / prefix changes) → rebuild. + const shrunk = all.slice(0, 300).filter((_, i) => i % 7 !== 0); + expect(timeline.sync(shrunk, yearJust)).toEqual(buildPhotoRows(shrunk, yearJust)); + + // width=0 yields [] and doesn't wedge the next positive-width sync. + const zero = makeConfig('year', 'justified', 0); + expect(timeline.sync(shrunk, zero)).toEqual([]); + expect(timeline.sync(shrunk, yearJust)).toEqual(buildPhotoRows(shrunk, yearJust)); + }); + + it('collapses grouping work from Σ O(N²/page) to O(N) and runs ≥3x faster', () => { + const N = PAGE * PAGES; + const all = Array.from({ length: N }, (_, i) => photo(i)); + + // AFTER: incremental — each photo is bucketed exactly once across the drain. + const afterCounter = { n: 0 }; + const afterCfg = makeConfig('month', 'square', WIDTH, afterCounter); + const timeline = new PhotoTimeline(); + const t1 = performance.now(); + for (let p = 1; p <= PAGES; p++) timeline.sync(all.slice(0, p * PAGE), afterCfg); + const afterMs = performance.now() - t1; + + // BEFORE: full rebuild per page — re-buckets the whole cumulative list. + const beforeCounter = { n: 0 }; + const beforeCfg = makeConfig('month', 'square', WIDTH, beforeCounter); + const t0 = performance.now(); + for (let p = 1; p <= PAGES; p++) buildPhotoRows(all.slice(0, p * PAGE), beforeCfg); + const beforeMs = performance.now() - t0; + + console.info( + `photo timeline ${PAGES}×${PAGE}: before ${beforeCounter.n} timestamp reads / ${beforeMs.toFixed(1)} ms — after ${afterCounter.n} reads / ${afterMs.toFixed(1)} ms (${(beforeCounter.n / afterCounter.n).toFixed(1)}x fewer reads, ${(beforeMs / afterMs).toFixed(1)}x wall)` + ); + + // Incremental buckets each photo once: exactly N reads. + expect(afterCounter.n).toBe(N); + // Full rebuild is quadratic: Σ_{p=1..P} p·PAGE. + expect(beforeCounter.n).toBe((PAGES * (PAGES + 1) * PAGE) / 2); + expect(afterCounter.n).toBeLessThan(beforeCounter.n / 5); + expect(afterMs).toBeLessThan(beforeMs / 3); + }); +}); diff --git a/frontend/src/lib/utils/photoTimeline.ts b/frontend/src/lib/utils/photoTimeline.ts new file mode 100644 index 00000000..ed53abf7 --- /dev/null +++ b/frontend/src/lib/utils/photoTimeline.ts @@ -0,0 +1,279 @@ +/** + * Photo-timeline grouping + row layout, extracted from the photos view so the + * O(N²) accumulation of its `groups`/`photoRows` derives can be replaced with + * an incremental builder (and unit/benchmark-tested off the Svelte reactive + * graph). + * + * Photos arrive newest-first (`media_sort_date DESC`), so each fetched page + * only ever extends the last date bucket or appends new buckets after it — + * never mutates an earlier group. {@link PhotoTimeline} exploits that: an + * append re-buckets only the new page and recomputes rows only for the groups + * that actually changed, keeping a full scroll O(N) instead of O(N²). + * + * The pure {@link buildPhotoRows} is the verbatim reference (what the old + * `groups`→`photoRows` derive chain produced); the benchmark gate asserts the + * incremental builder stays byte-for-byte equal to it. + */ +import type { PhotoItem } from '$lib/api/endpoints/photos'; + +export type GroupMode = 'day' | 'month' | 'year'; +export type LayoutMode = 'square' | 'justified'; + +export interface JustifiedTile { + file: PhotoItem; + w: number; + h: number; +} + +export type PhotoRow = + | { kind: 'header'; key: string; height: number; label: string; count: number } + | { kind: 'tiles'; key: string; height: number; gap: number; tiles: JustifiedTile[] }; + +/** Layout constants — mirror the photos view's original values exactly. */ +export const SQUARE_GAP = 4; // .25rem, matches the old grid gap +export const SQUARE_MIN = 144; // 9rem minmax floor +export const JUSTIFIED_GAP = 8; // .photos-jrow margin-bottom +export const HEADER_H = 44; + +export interface TimelineConfig { + groupMode: GroupMode; + layoutMode: LayoutMode; + /** Usable content width of the grid, in px. */ + width: number; + /** `(max-width: 768px)` — selects the 150px vs 200px justified target. */ + mobile: boolean; + /** EXIF-aware capture timestamp (ms). Injected so the module stays pure. */ + timestampOf: (p: PhotoItem) => number; + /** Locale-aware bucket label for a group's representative date. */ + labelOf: (d: Date, mode: GroupMode) => string; +} + +interface Group { + key: string; + label: string; + photos: PhotoItem[]; +} + +/** Year/month/day bucket key for a date under `groupMode` (verbatim). */ +export function bucketKey(d: Date, groupMode: GroupMode): string { + const y = d.getFullYear(); + if (groupMode === 'year') return `${y}`; + const m = `${d.getMonth() + 1}`.padStart(2, '0'); + if (groupMode === 'month') return `${y}-${m}`; + return `${y}-${m}-${`${d.getDate()}`.padStart(2, '0')}`; +} + +/** + * Pack files into justified rows (Flickr-style): each full row is scaled to + * fill `width` while preserving every tile's aspect ratio. Missing dimensions + * fall back to 1:1. Verbatim port of the photos view's `justifiedRows`, with + * the `matchMedia` read hoisted to the `mobile` flag so it's testable. + */ +export function justifiedRows( + files: PhotoItem[], + width: number, + mobile: boolean +): Array<{ height: number; tiles: JustifiedTile[] }> { + const gap = 8; + const target = mobile ? 150 : 200; + const rows: Array<{ height: number; tiles: JustifiedTile[] }> = []; + let cur: Array<{ file: PhotoItem; aspect: number }> = []; + let aspectSum = 0; + for (const file of files) { + let aspect = file.width && file.height ? file.width / file.height : 1; + if (!Number.isFinite(aspect) || aspect <= 0) aspect = 1; + aspect = Math.min(Math.max(aspect, 0.4), 3); + cur.push({ file, aspect }); + aspectSum += aspect; + const rowWidth = aspectSum * target + (cur.length - 1) * gap; + if (rowWidth >= width) { + const h = (width - (cur.length - 1) * gap) / aspectSum; + rows.push({ + height: Math.round(h), + tiles: cur.map((tt) => ({ + file: tt.file, + w: Math.max(1, Math.round(tt.aspect * h)), + h: Math.round(h) + })) + }); + cur = []; + aspectSum = 0; + } + } + if (cur.length) { + rows.push({ + height: target, + tiles: cur.map((tt) => ({ + file: tt.file, + w: Math.max(1, Math.round(tt.aspect * target)), + h: target + })) + }); + } + return rows; +} + +/** Columns + cell size for the square layout at width `W` (verbatim). */ +function squareGeometry(W: number): { cols: number; cell: number } { + const cols = Math.max(1, Math.floor((W + SQUARE_GAP) / (SQUARE_MIN + SQUARE_GAP))); + const cell = (W - (cols - 1) * SQUARE_GAP) / cols; + return { cols, cell }; +} + +/** Flatten one group into its header + tile rows (verbatim per-group body). */ +function groupToRows(g: Group, cfg: TimelineConfig, cols: number, cell: number): PhotoRow[] { + const rows: PhotoRow[] = [ + { kind: 'header', key: `h:${g.key}`, height: HEADER_H, label: g.label, count: g.photos.length } + ]; + if (cfg.layoutMode === 'justified') { + const jrows = justifiedRows(g.photos, cfg.width, cfg.mobile); + for (let ri = 0; ri < jrows.length; ri++) { + rows.push({ + kind: 'tiles', + key: `${g.key}:j${ri}`, + height: jrows[ri].height + JUSTIFIED_GAP, + gap: JUSTIFIED_GAP, + tiles: jrows[ri].tiles + }); + } + } else { + for (let i = 0; i < g.photos.length; i += cols) { + const tiles = g.photos.slice(i, i + cols).map((file) => ({ file, w: cell, h: cell })); + rows.push({ + kind: 'tiles', + key: `${g.key}:s${i}`, + height: cell + SQUARE_GAP, + gap: SQUARE_GAP, + tiles + }); + } + } + return rows; +} + +/** Bucket `items` into date groups, first-appearance order (verbatim). */ +function buildGroups(items: PhotoItem[], cfg: TimelineConfig): Group[] { + const out: Group[] = []; + const index = new Map(); + for (const p of items) { + const d = new Date(cfg.timestampOf(p)); + const key = bucketKey(d, cfg.groupMode); + let i = index.get(key); + if (i === undefined) { + i = out.length; + index.set(key, i); + out.push({ key, label: cfg.labelOf(d, cfg.groupMode), photos: [] }); + } + out[i].photos.push(p); + } + return out; +} + +/** + * Verbatim reference: the flat `PhotoRow[]` the old `groups`→`photoRows` + * derive chain produced for `items` under `cfg`. Returns `[]` for a + * non-positive width, matching the old guard. The benchmark gate holds the + * incremental builder equal to this. + */ +export function buildPhotoRows(items: PhotoItem[], cfg: TimelineConfig): PhotoRow[] { + if (cfg.width <= 0) return []; + const { cols, cell } = squareGeometry(cfg.width); + const rows: PhotoRow[] = []; + for (const g of buildGroups(items, cfg)) { + rows.push(...groupToRows(g, cfg, cols, cell)); + } + return rows; +} + +function configEq(a: TimelineConfig, b: TimelineConfig): boolean { + return ( + a.groupMode === b.groupMode && + a.layoutMode === b.layoutMode && + a.width === b.width && + a.mobile === b.mobile && + a.timestampOf === b.timestampOf && + a.labelOf === b.labelOf + ); +} + +/** + * Incremental photo-timeline builder. Call {@link sync} with the current item + * list and config on every change; it detects the common case — the list grew + * by appending a page while config is unchanged — and re-buckets only the new + * items + re-lays-out only the groups that changed, reusing every untouched + * group's cached rows. Any other change (config, deletion, filter toggle, + * non-append) falls back to a full rebuild, so the result is always identical + * to {@link buildPhotoRows}. + */ +export class PhotoTimeline { + #cfg: TimelineConfig | null = null; + #groups: Group[] = []; + /** Items already bucketed — the append cursor into the last synced list. */ + #groupedItems: PhotoItem[] = []; + /** group.key → its cached rows for the current config. */ + #rowCache = new Map(); + #geom = { cols: 1, cell: 0 }; + + /** Whether `next` extends `prev` (same prefix objects + strictly longer). */ + #isAppend(prev: PhotoItem[], next: PhotoItem[]): boolean { + if (next.length <= prev.length) return false; + // Prefix identity via the boundary object — O(1), the list is only ever + // mutated by appending or by replacing with a filtered copy. + return prev.length === 0 || next[prev.length - 1] === prev[prev.length - 1]; + } + + #rebuild(items: PhotoItem[], cfg: TimelineConfig): void { + this.#cfg = cfg; + this.#groups = cfg.width > 0 ? buildGroups(items, cfg) : []; + this.#groupedItems = items; + this.#rowCache.clear(); + this.#geom = squareGeometry(cfg.width); + } + + #extend(items: PhotoItem[], cfg: TimelineConfig): void { + const fresh = items.slice(this.#groupedItems.length); + // The last existing group may grow, so its cached rows are stale. + if (this.#groups.length > 0) { + this.#rowCache.delete(this.#groups[this.#groups.length - 1].key); + } + for (const p of fresh) { + const d = new Date(cfg.timestampOf(p)); + const key = bucketKey(d, cfg.groupMode); + const last = this.#groups[this.#groups.length - 1]; + if (last && last.key === key) { + last.photos.push(p); + } else { + this.#groups.push({ key, label: cfg.labelOf(d, cfg.groupMode), photos: [p] }); + } + } + this.#groupedItems = items; + } + + sync(items: PhotoItem[], cfg: TimelineConfig): PhotoRow[] { + if (cfg.width <= 0) { + // Keep the item cursor so a later positive width rebuilds from scratch. + this.#cfg = cfg; + this.#groups = []; + this.#groupedItems = items; + this.#rowCache.clear(); + return []; + } + if (this.#cfg && configEq(this.#cfg, cfg) && this.#isAppend(this.#groupedItems, items)) { + this.#extend(items, cfg); + } else { + this.#rebuild(items, cfg); + } + + const { cols, cell } = this.#geom; + const out: PhotoRow[] = []; + for (const g of this.#groups) { + let rows = this.#rowCache.get(g.key); + if (rows === undefined) { + rows = groupToRows(g, cfg, cols, cell); + this.#rowCache.set(g.key, rows); + } + for (const r of rows) out.push(r); + } + return out; + } +} diff --git a/frontend/src/routes/photos/+page.svelte b/frontend/src/routes/photos/+page.svelte index 7b4cdf07..b823951d 100644 --- a/frontend/src/routes/photos/+page.svelte +++ b/frontend/src/routes/photos/+page.svelte @@ -17,6 +17,12 @@ import { filterDotfiles } from '$lib/utils/dotfileFilter'; import { dateTimeFormatFor } from '$lib/utils/display'; import { isVideo, photoTimestamp } from '$lib/utils/media'; + import { + PhotoTimeline, + type GroupMode, + type LayoutMode, + type PhotoRow + } from '$lib/utils/photoTimeline'; type Tab = 'moments' | 'places' | 'people'; let tab = $state('moments'); @@ -49,8 +55,6 @@ /** Usable content width of the grid, for the justified layout. */ let gridWidth = $state(0); - type GroupMode = 'day' | 'month' | 'year'; - type LayoutMode = 'square' | 'justified'; const GROUP_KEY = 'oxi-photos-group'; const LAYOUT_KEY = 'oxi-photos-layout'; let groupMode = $state('month'); @@ -64,18 +68,10 @@ else if (tab === 'people') void peopleView.load(); }); - /** EXIF-aware timestamp (seconds → ms), matching the OLD grouping logic. */ - function bucketKey(d: Date): string { - const y = d.getFullYear(); - if (groupMode === 'year') return `${y}`; - const m = `${d.getMonth() + 1}`.padStart(2, '0'); - if (groupMode === 'month') return `${y}-${m}`; - return `${y}-${m}-${`${d.getDate()}`.padStart(2, '0')}`; - } - - function bucketLabel(d: Date): string { - if (groupMode === 'year') return `${d.getFullYear()}`; - if (groupMode === 'month') + /** Locale-aware label for a bucket's representative date. */ + function bucketLabel(d: Date, mode: GroupMode): string { + if (mode === 'year') return `${d.getFullYear()}`; + if (mode === 'month') return dateTimeFormatFor(undefined, { year: 'numeric', month: 'long' }).format(d); return dateTimeFormatFor(undefined, { weekday: 'long', @@ -85,132 +81,32 @@ }).format(d); } - const groups = $derived.by(() => { - const out: Array<{ key: string; label: string; photos: PhotoItem[] }> = []; - // Transient scratch map built inside $derived.by and discarded — not reactive state. - // eslint-disable-next-line svelte/prefer-svelte-reactivity - const index = new Map(); - for (const p of visibleItems) { - const d = new Date(photoTimestamp(p)); - const key = bucketKey(d); - let i = index.get(key); - if (i === undefined) { - i = out.length; - index.set(key, i); - out.push({ key, label: bucketLabel(d), photos: [] }); - } - out[i].photos.push(p); - } - return out; - }); - - interface JustifiedTile { - file: PhotoItem; - w: number; - h: number; - } - - /** - * Pack files into justified rows (Flickr-style): each full row is scaled to - * fill `width` while preserving every tile's aspect ratio. Missing dimensions - * fall back to 1:1. - */ - function justifiedRows( - files: PhotoItem[], - width: number - ): Array<{ height: number; tiles: JustifiedTile[] }> { - const gap = 8; - const target = window.matchMedia('(max-width: 768px)').matches ? 150 : 200; - const rows: Array<{ height: number; tiles: JustifiedTile[] }> = []; - let cur: Array<{ file: PhotoItem; aspect: number }> = []; - let aspectSum = 0; - for (const file of files) { - let aspect = file.width && file.height ? file.width / file.height : 1; - if (!Number.isFinite(aspect) || aspect <= 0) aspect = 1; - aspect = Math.min(Math.max(aspect, 0.4), 3); - cur.push({ file, aspect }); - aspectSum += aspect; - const rowWidth = aspectSum * target + (cur.length - 1) * gap; - if (rowWidth >= width) { - const h = (width - (cur.length - 1) * gap) / aspectSum; - rows.push({ - height: Math.round(h), - tiles: cur.map((tt) => ({ - file: tt.file, - w: Math.max(1, Math.round(tt.aspect * h)), - h: Math.round(h) - })) - }); - cur = []; - aspectSum = 0; - } - } - if (cur.length) { - rows.push({ - height: target, - tiles: cur.map((tt) => ({ - file: tt.file, - w: Math.max(1, Math.round(tt.aspect * target)), - h: target - })) - }); - } - return rows; - } - // ── Virtualized row model ──────────────────────────────────────────────── - // Flatten the groups into a single list of fixed-height rows (a date header - // or a strip of sized tiles), so VirtualRows can window the whole timeline — - // only the rows near the viewport are mounted, regardless of library size. - const SQUARE_GAP = 4; // .25rem, matches the old grid gap - const SQUARE_MIN = 144; // 9rem minmax floor - const JUSTIFIED_GAP = 8; // .photos-jrow margin-bottom - const HEADER_H = 44; - - type PhotoRow = - | { kind: 'header'; key: string; height: number; label: string; count: number } - | { kind: 'tiles'; key: string; height: number; gap: number; tiles: JustifiedTile[] }; - - const photoRows = $derived.by(() => { - const W = gridWidth; - if (W <= 0) return []; - const rows: PhotoRow[] = []; - const cols = Math.max(1, Math.floor((W + SQUARE_GAP) / (SQUARE_MIN + SQUARE_GAP))); - const cell = (W - (cols - 1) * SQUARE_GAP) / cols; - for (const g of groups) { - rows.push({ - kind: 'header', - key: `h:${g.key}`, - height: HEADER_H, - label: g.label, - count: g.photos.length - }); - if (layoutMode === 'justified') { - const jrows = justifiedRows(g.photos, W); - for (let ri = 0; ri < jrows.length; ri++) { - rows.push({ - kind: 'tiles', - key: `${g.key}:j${ri}`, - height: jrows[ri].height + JUSTIFIED_GAP, - gap: JUSTIFIED_GAP, - tiles: jrows[ri].tiles - }); - } - } else { - for (let i = 0; i < g.photos.length; i += cols) { - const tiles = g.photos.slice(i, i + cols).map((file) => ({ file, w: cell, h: cell })); - rows.push({ - kind: 'tiles', - key: `${g.key}:s${i}`, - height: cell + SQUARE_GAP, - gap: SQUARE_GAP, - tiles - }); - } - } - } - return rows; - }); + // Flatten the date groups into a single list of fixed-height rows (a header + // or a strip of sized tiles) that VirtualRows windows. Because pages arrive + // newest-first, each append only extends the last group or adds new ones, so + // PhotoTimeline re-buckets only the fresh page and re-lays-out only the + // groups that changed — a full scroll stays O(N), not O(N²) (the old + // `groups`→`photoRows` derive chain re-grouped + re-packed the whole library + // on every 60-item page). See photoGrouping.bench.test.ts. + // `sync` mutates the timeline's (non-reactive) internal group/row caches and + // returns the flat rows. Driven from `$derived.by` for idempotence: if the + // deps re-fire without an actual append, `sync` sees a non-growing list and + // safely full-rebuilds — same output as the pure `buildPhotoRows`. + const timeline = new PhotoTimeline(); + const photoRows = $derived.by(() => + timeline.sync(visibleItems, { + groupMode, + layoutMode, + width: gridWidth, + mobile: + typeof window !== 'undefined' && + typeof window.matchMedia === 'function' && + window.matchMedia('(max-width: 768px)').matches, + timestampOf: photoTimestamp, + labelOf: bucketLabel + }) + ); async function loadMore() { if (loading || exhausted) return; diff --git a/src/application/services/file_retrieval_service.rs b/src/application/services/file_retrieval_service.rs index de03ac08..882ecb46 100644 --- a/src/application/services/file_retrieval_service.rs +++ b/src/application/services/file_retrieval_service.rs @@ -287,22 +287,6 @@ impl FileRetrievalService { Ok(files.into_iter().map(FileDto::from).collect()) } - /// Range read that first consults the RAM content cache (see - /// [`Self::get_file_range_preloaded`]). - pub async fn get_file_range_preloaded_with_perms( - &self, - dto: &FileDto, - caller_id: Uuid, - start: u64, - end: Option, - ) -> Result { - self.require_file(&dto.id, Permission::Read, caller_id) - .await?; - // Same throttled Recent recording as the streaming variant. - self.notify_file_accessed(caller_id, &dto.id); - self.get_file_range_preloaded(dto, start, end).await - } - /// Range read for HTTP Range Requests, cache-aware. /// /// Media players and PDF viewers fetch these files *exclusively* through diff --git a/src/interfaces/api/handlers/file_handler.rs b/src/interfaces/api/handlers/file_handler.rs index f1095780..f729865c 100644 --- a/src/interfaces/api/handlers/file_handler.rs +++ b/src/interfaces/api/handlers/file_handler.rs @@ -712,13 +712,15 @@ impl FileHandler { let disposition = Self::content_disposition(&file_dto.name, &file_dto.mime_type, ¶ms); + // `file_dto` was already Read-authorized (and the access + // recorded) by `get_file_with_perms` above — every seek in + // a media/PDF scrub is a separate Range request, so + // re-authorizing + re-notifying per seek doubled that work + // for nothing. Use the non-perms range read, matching the + // share-landing and WebDAV range paths which authorize once + // then stream (benches/ROUND7.md). match retrieval - .get_file_range_preloaded_with_perms( - &file_dto, - auth_user.id, - start, - Some(end + 1), - ) + .get_file_range_preloaded(&file_dto, start, Some(end + 1)) .await { Ok(content) => { diff --git a/src/interfaces/api/handlers/folder_handler.rs b/src/interfaces/api/handlers/folder_handler.rs index f5c525f4..cc707977 100644 --- a/src/interfaces/api/handlers/folder_handler.rs +++ b/src/interfaces/api/handlers/folder_handler.rs @@ -476,7 +476,9 @@ pub async fn list_folder_resources( let dto = FolderDto { etag: resource_id.clone(), id: resource_id, - name: row.name.clone(), + // Folders use fixed icon classes (below), so `name` + // is never borrowed again — move it instead of cloning. + name: row.name, path: String::new(), // cleared — share recipients must not see hierarchy parent_id: row.parent_id.map(|u| u.to_string()), drive_id: row.drive_id, @@ -514,20 +516,26 @@ pub async fn list_folder_resources( } else { File::compute_etag(&content_hash, modified_at_u) }; + // Compute the name-derived icon/category classes first + // (they borrow `&row.name`), so `name` can be moved into + // the DTO below instead of cloned — one fewer String + // alloc per file row (benches/ROUND7.md). + let icon_class = intern_display(icon_class_for(&row.name, mime)); + let icon_special_class = + intern_display(icon_special_class_for(&row.name, mime)); + let category = intern_display(category_for(&row.name, mime)); let dto = FileDto { id: row.id.to_string(), - name: row.name.clone(), + name: row.name, path: String::new(), size: size_bytes, mime_type: intern_mime(mime), folder_id: row.parent_id.map(|u| u.to_string()), created_at: row.created_at.timestamp() as u64, modified_at: row.modified_at.timestamp() as u64, - icon_class: intern_display(icon_class_for(&row.name, mime)), - icon_special_class: intern_display(icon_special_class_for( - &row.name, mime, - )), - category: intern_display(category_for(&row.name, mime)), + icon_class, + icon_special_class, + category, size_formatted: format_file_size(size_bytes), sort_date: None, content_hash,