cjw/Oxicloud
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

perf: round 7 — photos timeline O(N²)→incremental, range-seek authz duplication, resources row-map clone

Browse Source 
Benchmark-gated (equivalence + BEFORE/AFTER; results + reproduce commands in
benches/ROUND7.md):

- Photos timeline re-grouped + re-laid-out the whole accumulated library on
  every 60-item page (both `groups` and `photoRows` were $derived over the
  full list), Σ ≈ O(N²/60) main-thread work during a scroll. Pages arrive
  newest-first so grouping is append-only: the new PhotoTimeline
  (lib/utils/photoTimeline.ts) re-buckets only the fresh page and re-lays-out
  only changed groups, reusing untouched groups' cached rows, falling back to
  a full rebuild on any config/deletion/non-append change. The pure
  buildPhotoRows is the verbatim reference the gate holds it equal to at every
  page. 50×60 drain: 76 500 → 3 000 grouping ops (25.5x), 23.0 → 2.2 ms
  (10.6x).

- Range downloads paid authz + access-notify twice: download_file_impl
  resolves the file via get_file_with_perms, then the Range branch re-ran
  require_file + notify_file_accessed per request. Media/PDF viewers fetch
  exclusively via Range (one request per seek), so every seek in a scrub
  re-authorized an already-cleared file. Now routed through the non-perms
  get_file_range_preloaded (matching the share-landing + WebDAV range paths);
  the unused _with_perms range method is removed. The request-level gate still
  denies before the branch runs (bench asserts member granted, outsider
  denied). Per seek removed: WARM 0.67 µs, COLD 1362.66 µs — a grant-cascade
  drive-resolve query per seek for a shared-drive recipient on a cold cache.

- /api/folders/{id}/resources row→DTO mapping cloned row.name into the DTO
  though the row is owned; folders move it (fixed icons), files compute the
  name-derived icon/category classes first then move it. 500-row page:
  10.004 → 9.004 allocs/row (500 clones removed), output identical.

Deferred with rationale in ROUND7.md: thumbnail ACL-before-304 (security
posture — needs a security review, not a perf tweak), batch_operations
Arc<str>→String widening, list-view O(N²) on smaller lists, and the serial→
join! pairs (decide-by-bench with injected latency, per the round-6 rejection).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
This commit is contained in:
Claude
2026-07-18 13:11:41 +00:00
parent f53fba42a1
commit 7626dc95c1
10 changed files with 1228 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Cargo.toml
+17
View File
@@ -350,6 +350,23 @@ name = "bench_micro_allocs"
path = "examples/bench_micro_allocs.rs"
required-features = ["bench"]
# Round-7 battery ─────────────────────────────────────────────────────────────
# Range-seek per-request authz duplication — the per-seek require the range
# branch used to run (warm CPU + cold drive-resolve query) vs 0 after routing
# through the non-perms range read (needs the dev Postgres up).
[[example]]
name = "bench_range_seek_authz"
path = "examples/bench_range_seek_authz.rs"
required-features = ["bench"]
# `/api/folders/{id}/resources` row→DTO mapping — per-row name clone vs move
# (pure CPU; counting allocator).
[[example]]
name = "bench_resource_row_map"
path = "examples/bench_resource_row_map.rs"
required-features = ["bench"]
# Round-6 battery ─────────────────────────────────────────────────────────────
# CardDAV whole-book REPORT/PROPFIND — buffered double-residency vs cursor
benches/ROUND7.md
+146
View File
@@ -0,0 +1,146 @@
# Round 7 — photo timeline O(N²) → incremental, range-seek authz duplication, row-map clone
Benchmark-gated changes, same rule as ROUND2-6: every change ships with a
BEFORE/AFTER benchmark; an AFTER that doesn't beat its BEFORE gets rolled
back. Equivalence gates (identical output / byte-identical responses) guard
every behavior-preserving rewrite. Frontend changes carry vitest benchmark
gates (verbatim BEFORE replica + equivalence + perf assertion) committed
beside the code so CI re-verifies the win on every run.
Measured on 4 cores / 15 GiB, local PostgreSQL 16 (fsync off), release
profile; frontend on Node 22 / vitest 4 (jsdom). Reproduce any row with the
command in its section.
## Summary
| # | change | key metric | before → after |
|--:|---|---|---|
| 1 | Photos timeline incremental grouping/layout | 50-page (3k-photo) scroll drain | 76 500 → 3 000 group ops (**25.5x**) / 23.0 → 2.2 ms (**10.6x**) |
| 2 | Range-seek per-request authz duplication removed | per-seek authz on a shared-drive scrub | WARM 0.67 → 0 µs/seek; **COLD 1362.66 → 0 µs/seek** (a drive-resolve query per seek) |
| 3 | `/resources` row→DTO name clone → move | allocs/row (500-row page) | 10.004 → 9.004 (**500 allocs saved**, 1.00/row) |
## [1] Photos timeline — O(N²) re-group + re-layout per page → incremental builder
The photos view appended each 60-item page with `items = [...items, ...page]`
and re-derived both `groups` (O(N), a `new Date()` per photo) and `photoRows`
(O(N) row layout) over the whole accumulated list on every page — so paging to
photo N re-grouped + re-laid-out everything loaded so far, Σ ≈ O(N²/60) of
main-thread work during the scroll (the exact class ROUND6 fixed for the files
listing). The DOM was already windowed (`VirtualRows`); this was the derivation
feeding it.
Because photos arrive newest-first (`media_sort_date DESC`), grouping is
append-only: a page only ever extends the last date bucket or adds buckets
after it, never mutates an earlier group. The new `PhotoTimeline`
(`lib/utils/photoTimeline.ts`) exploits that — an append re-buckets only the
fresh page and re-lays-out only the groups that changed, reusing every
untouched group's cached rows; any other change (config, deletion, filter
toggle, non-append) falls back to a full rebuild. The pure `buildPhotoRows` is
the verbatim reference the gate holds it equal to.
Gates: the incremental output is deep-equal to `buildPhotoRows` at EVERY page
of the drain (both square + justified layouts); config-change / deletion /
width=0 fall back to a correct full rebuild; grouping work collapses ≥5x and
wall ≥3x.
```
cd frontend && npx vitest run src/lib/utils/photoTimeline.bench.test.ts --disable-console-intercept
# photo timeline 50×60: before 76500 timestamp reads / 23.0 ms
# after 3000 timestamp reads / 2.2 ms
# (25.5x fewer grouping ops, 10.6x wall)
```
## [2] Range downloads — duplicate per-seek authz + access-notify removed
`download_file_impl` resolves the file once via `get_file_with_perms` (authz +
access-notify + metadata), then the Range branch called
`get_file_range_preloaded_with_perms`, which re-ran `require_file` (authz) +
`notify_file_accessed` per request. Media players and PDF viewers fetch a file
*exclusively* through Range requests — a `bytes=0-` probe then one request per
seek — so every seek in a scrub re-authorized a file the request-level gate had
already cleared. The share-landing and WebDAV range paths already authorize
once then read via the non-perms `get_file_range_preloaded`; the REST handler
now does the same (and the now-unused `_with_perms` range method is deleted).
Safety: the request-level `get_file_with_perms` still gates every request
(denies before the Range branch runs), so the removed per-seek re-check
bypasses nothing — the bench asserts the member is granted and a non-member
denied.
```
cargo run --release --features bench --example bench_range_seek_authz
# seeks/scrub=200 (member of a shared drive, viewer grant)
# arm wall ms µs/seek
# BEFORE per-seek (WARM) 0.13 0.67 <- moka hit + uuid parse, removed
# BEFORE per-seek (COLD) 272.53 1362.66 <- a grant-cascade drive-resolve
# QUERY per seek, removed
# AFTER per-seek (removed) 0.00 0.00
# A 200-seek scrub of a shared video stops paying ~272 ms of authz queries
# when the drive-role cache is cold (cross-drive recipient, or 30 s TTL expiry
# mid-scrub). notify_file_accessed (a throttled hook call) is likewise removed
# per seek.
```
## [3] `/api/folders/{id}/resources` row→DTO mapping — clone name → move name
The listing maps each owned `FolderResourceRow` into a DTO but cloned
`row.name` into it (`name: row.name.clone()`) — one avoidable `String` heap
alloc per listed folder/file. The folder branch uses fixed icon classes, so
`row.name` is simply moved; the file branch computes its name-derived icon /
category classes first (they borrow `&row.name`), then moves `row.name` in. One
fewer alloc per row, identical output.
```
cargo run --release --features bench --example bench_resource_row_map
# rows=500
# arm allocs wall ms allocs/row
# BEFORE (clone) 5002 0.841 10.004
# AFTER (move) 4502 0.810 9.004
# Saved 500 allocs (1.00/row) — the per-row name clone removed; output identical.
```
## Deferred / flagged (not shipped this round)
- **Thumbnail ACL-before-304 (security posture — needs maintainer decision).**
`get_thumbnail_impl` runs `require_permission(Read)` before the ETag-304 and
moka/disk short-circuits, so a shared-album recipient pays a grant-cascade
query per thumbnail revalidation. Moving authz *after* the cache would make
thumbnails "authorized at creation time only" — a user whose access was
revoked could still fetch cached thumbnails of files they once could see.
That is a deliberate security-posture change, not a perf tweak; left for a
security review. The safe alternative (back the non-owner authz with the
existing `drive_role_cache`, or a `Borrow<str>` cache key that removes the
per-request `to_string`) is queued for round 8 with an alloc/query bench.
- **`batch_operations` `Arc<str>` → `String` per item.** `copy_file_with_perms`
/ `move_file_with_perms` take `Option<String>`, so the batch path's
`target_folder: Arc<str>` is re-`to_string()`-ed per item, defeating the
Arc. Widening those `_with_perms` signatures to `Option<&str>` touches the
trait + impl + stub + ~7 call sites — a contained refactor better done
deliberately with its own alloc bench; queued for round 8.
- **List-view O(N²) re-derive (favorites / recent / trash / shared-with-me /
shared swimlanes).** Same class as [1] but on typically-smaller lists;
each infinite-scroll page re-derives `entries` / `byId` / `sections` /
`lanes` over the full accumulated set. Deferred — the incremental-builder
cost isn't yet justified at those sizes; revisit if any surface reaches
thousands of rows.
- **Serial independent DB pairs → `join!` (token refresh, login, cross-drive
move, CardDAV discovery, NC PROPFIND enrichment).** Overlapping independent
round-trips saves 1 RTT *under real PG latency*, but the ROUND6 authz-fan-out
rejection showed the overhead can wash the win out on local-socket PG. These
need a decide-by-bench with an injected-latency arm (like the ROUND6 `::text`
A/B) before adoption — queued for round 8, not guessed at here.
## Correctness-adjacent (surfaced by the round-7 hunt — not perf, flagged for follow-up)
- **`fetchFolderListing` returns empty `favoriteIds`/`sharedIds`**
(`frontend/src/lib/api/endpoints/folders.ts`) since the combined `/listing`
route was removed — the files-grid star/shared badges are seeded empty on
every navigation. The same removal also dropped the 304 conditional
fast-path, so a folder navigation now pages the full body (`cache: no-store`)
instead of a bodiless 304 on unchanged folders (mitigated only by the
in-memory `folderCache`). Functional regression, not perf.
- **Search page lacks a stale-response guard**
(`frontend/src/routes/search/+page.svelte`): the query `$effect` awaits
`searchFiles` with no `seq`/AbortController, so a slow stale query can
resolve after and clobber a newer one. The files view's `loadSeq` is the
pattern to mirror.
examples/bench_range_seek_authz.rs
+283
View File
@@ -0,0 +1,283 @@
//! Range-seek per-request authz duplication benchmark.
//!
//! `download_file_impl` calls `get_file_with_perms` once (authz + access
//! notify + metadata) and THEN, in the Range branch, called
//! `get_file_range_preloaded_with_perms` — which re-ran `require_file`
//! (authz) + `notify_file_accessed` per request. Media players and PDF
//! viewers fetch a file *exclusively* through Range requests: a `bytes=0-`
//! probe then one request per seek. So every seek in a scrub re-authorized a
//! file the request-level gate had already cleared.
//!
//! Round 7 drops the range branch to the non-perms `get_file_range_preloaded`
//! (the share-landing and WebDAV range paths already do exactly this). This
//! bench isolates the per-seek `require` that AFTER eliminates, driving the
//! REAL `PgAclEngine`:
//! - WARM: the cache the initial `get_file_with_perms` warmed — each removed
//! seek-check was a moka hit + uuid parse (pure CPU/alloc).
//! - COLD: a shared-drive recipient whose drive-role cache expired mid-scrub
//! (30 s TTL) — each removed seek-check was a full drive-resolve query.
//!
//! Safety gate: the surviving request-level gate still authorizes correctly —
//! the member is granted, a non-member is denied — so removing the per-seek
//! re-check bypasses nothing.
//!
//! Run (needs Postgres up; reads DATABASE_URL from .env):
//! cargo run --release --features bench --example bench_range_seek_authz
//! Tunables (env): BENCH_SEEKS (200), BENCH_POOL (8).
use std::env;
use std::sync::Arc;
use std::time::{Duration, Instant};
use oxicloud::application::ports::authorization_ports::AuthorizationEngine;
use oxicloud::domain::services::authorization::{Permission, Resource, Subject};
use oxicloud::infrastructure::repositories::pg::{
FileBlobReadRepository, FolderDbRepository, SubjectGroupPgRepository,
};
use oxicloud::infrastructure::services::dedup_service::DedupService;
use oxicloud::infrastructure::services::local_blob_backend::LocalBlobBackend;
use oxicloud::infrastructure::services::pg_acl_engine::PgAclEngine;
use sqlx::PgPool;
use sqlx::postgres::PgPoolOptions;
use uuid::Uuid;
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
env::var(key)
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(default)
}
struct Seeded {
member: Uuid,
outsider: Uuid,
drive_id: Uuid,
root_folder: Uuid,
blob_hash: String,
file_id: Uuid,
}
async fn seed(pool: &PgPool) -> Seeded {
let mut tx = pool.begin().await.expect("begin");
let member: Uuid = sqlx::query_scalar(
"INSERT INTO auth.users (username, email, role)
VALUES ('bench_rangeseek', 'bench_rangeseek@bench.invalid', 'user') RETURNING id",
)
.fetch_one(&mut *tx)
.await
.expect("seed member");
let outsider: Uuid = sqlx::query_scalar(
"INSERT INTO auth.users (username, email, role)
VALUES ('bench_rangeseek_out', 'bench_rangeseek_out@bench.invalid', 'user') RETURNING id",
)
.fetch_one(&mut *tx)
.await
.expect("seed outsider");
let drive_id: Uuid =
sqlx::query_scalar("INSERT INTO storage.drives (kind) VALUES ('shared') RETURNING id")
.fetch_one(&mut *tx)
.await
.expect("seed drive");
let root_folder: Uuid = sqlx::query_scalar(
"INSERT INTO storage.folders (name, path, lpath, drive_id)
VALUES ('Bench Seek', '/Bench Seek', 'x', $1) RETURNING id",
)
.bind(drive_id)
.fetch_one(&mut *tx)
.await
.expect("seed folder");
sqlx::query("UPDATE storage.drives SET root_folder_id = $1 WHERE id = $2")
.bind(root_folder)
.bind(drive_id)
.execute(&mut *tx)
.await
.expect("stamp root");
sqlx::query(
"INSERT INTO storage.role_grants
(subject_type, subject_id, resource_type, resource_id, role, granted_by)
VALUES ('user', $1, 'drive', $2, 'viewer'::storage.grant_role, $1)",
)
.bind(member)
.bind(drive_id)
.execute(&mut *tx)
.await
.expect("seed grant");
let blob_hash = "benchrangeseek00000000000000000000000000000000000000000000000b3".to_string();
sqlx::query("INSERT INTO storage.blobs (hash, size, ref_count) VALUES ($1, 1048576, 1)")
.bind(&blob_hash)
.execute(&mut *tx)
.await
.expect("seed blob");
let file_id: Uuid = sqlx::query_scalar(
"INSERT INTO storage.files (name, folder_id, blob_hash, size, mime_type, drive_id)
VALUES ('clip.mp4', $1, $2, 1048576, 'video/mp4', $3) RETURNING id",
)
.bind(root_folder)
.bind(&blob_hash)
.bind(drive_id)
.fetch_one(&mut *tx)
.await
.expect("seed file");
tx.commit().await.expect("commit");
Seeded {
member,
outsider,
drive_id,
root_folder,
blob_hash,
file_id,
}
}
async fn cleanup(pool: &PgPool, s: &Seeded) {
let _ = sqlx::query("DELETE FROM storage.role_grants WHERE resource_id = $1")
.bind(s.drive_id)
.execute(pool)
.await;
let _ = sqlx::query("DELETE FROM storage.files WHERE drive_id = $1")
.bind(s.drive_id)
.execute(pool)
.await;
let _ = sqlx::query("DELETE FROM storage.drives WHERE id = $1")
.bind(s.drive_id)
.execute(pool)
.await;
let _ = sqlx::query("DELETE FROM storage.folders WHERE id = $1")
.bind(s.root_folder)
.execute(pool)
.await;
let _ = sqlx::query("DELETE FROM storage.blobs WHERE hash = $1")
.bind(&s.blob_hash)
.execute(pool)
.await;
let _ = sqlx::query("DELETE FROM auth.users WHERE id IN ($1, $2)")
.bind(s.member)
.bind(s.outsider)
.execute(pool)
.await;
}
fn fresh_engine(pool: &Arc<PgPool>) -> Arc<PgAclEngine> {
let folder_repo = Arc::new(FolderDbRepository::new(pool.clone()));
let backend = Arc::new(LocalBlobBackend::new(std::path::Path::new(
"/tmp/bench-rangeseek-blobs",
)));
let dedup = Arc::new(DedupService::new(backend, pool.clone(), pool.clone()));
let file_repo = Arc::new(FileBlobReadRepository::new(
pool.clone(),
dedup,
folder_repo.clone(),
));
let group_repo = Arc::new(SubjectGroupPgRepository::new(pool.clone()));
Arc::new(PgAclEngine::new(
pool.clone(),
folder_repo,
file_repo,
group_repo,
))
}
/// The per-seek check the range branch used to run (verbatim: uuid parse +
/// `authz.require`, exactly `require_file`'s body).
async fn seek_require(engine: &Arc<PgAclEngine>, caller: Uuid, file_id: Uuid) -> bool {
engine
.require(
Subject::User(caller),
Permission::Read,
Resource::File(file_id),
)
.await
.is_ok()
}
#[tokio::main(flavor = "multi_thread")]
async fn main() {
dotenvy::dotenv().ok();
let url = env::var("DATABASE_URL")
.or_else(|_| env::var("OXICLOUD_DB_CONNECTION_STRING"))
.expect("set DATABASE_URL — the dev Postgres URL");
let seeks: usize = env_or("BENCH_SEEKS", 200);
let pool_size: u32 = env_or("BENCH_POOL", 8);
let pool = Arc::new(
PgPoolOptions::new()
.max_connections(pool_size)
.min_connections(pool_size)
.acquire_timeout(Duration::from_secs(10))
.connect(&url)
.await
.expect("connect Postgres"),
);
let s = seed(&pool).await;
// ── Safety gate: the surviving request-level gate authorizes correctly ──
let gate = fresh_engine(&pool);
let member_ok = seek_require(&gate, s.member, s.file_id).await;
let outsider_denied = !seek_require(&gate, s.outsider, s.file_id).await;
if !member_ok || !outsider_denied {
eprintln!(
"SAFETY GATE FAILED: member_ok={member_ok} outsider_denied={outsider_denied} \
(the single request-level authz must still grant the member and deny the outsider)"
);
cleanup(&pool, &s).await;
std::process::exit(1);
}
println!("\n#################################################################");
println!("# range-seek authz duplication: per-seek require (BEFORE) vs 0 (AFTER)");
println!("# seeks/scrub={seeks} (member of a shared drive, viewer grant)");
println!("#################################################################\n");
println!("| {:<26} | {:>10} | {:>12} |", "arm", "wall ms", "µs/seek");
// WARM: one require warms owner_cache + drive_role_cache (as the handler's
// get_file_with_perms does), then the scrub's per-seek re-checks are moka
// hits — pure CPU/alloc the AFTER path removes.
{
let engine = fresh_engine(&pool);
seek_require(&engine, s.member, s.file_id).await; // warm
let t = Instant::now();
for _ in 0..seeks {
std::hint::black_box(seek_require(&engine, s.member, s.file_id).await);
}
let el = t.elapsed();
println!(
"| {:<26} | {:>10.2} | {:>12.2} |",
"BEFORE per-seek (WARM)",
el.as_secs_f64() * 1e3,
el.as_secs_f64() * 1e6 / seeks as f64
);
}
// COLD: a fresh engine per seek models a cross-drive recipient or a
// drive-role-cache entry that expired mid-scrub (30 s TTL) — each removed
// re-check was a full grant-cascade drive-resolve query.
{
let t = Instant::now();
for _ in 0..seeks {
let engine = fresh_engine(&pool);
std::hint::black_box(seek_require(&engine, s.member, s.file_id).await);
}
let el = t.elapsed();
println!(
"| {:<26} | {:>10.2} | {:>12.2} |",
"BEFORE per-seek (COLD)",
el.as_secs_f64() * 1e3,
el.as_secs_f64() * 1e6 / seeks as f64
);
}
println!(
"| {:<26} | {:>10.2} | {:>12.2} |",
"AFTER per-seek (removed)", 0.0, 0.0
);
cleanup(&pool, &s).await;
println!("\n(AFTER runs zero per-seek authz: the request-level get_file_with_perms");
println!(" already authorized + recorded the access. WARM = the moka/CPU cost removed");
println!(" per seek; COLD = the drive-resolve query removed per seek when the cache");
println!(" isn't warm. notify_file_accessed (a throttled hook call) is likewise");
println!(" removed per seek. Safety gate: member granted, outsider denied.)");
}
examples/bench_resource_row_map.rs
+282
View File
@@ -0,0 +1,282 @@
//! `/api/folders/{id}/resources` row→DTO mapping micro-alloc benchmark.
//!
//! The listing maps each `FolderResourceRow` into a `FolderResourceItemDto`.
//! BEFORE cloned `row.name` into the DTO (`name: row.name.clone()`) even
//! though the row is owned by the mapping closure — one avoidable `String`
//! heap alloc per listed folder/file. AFTER computes the name-derived icon /
//! category classes first (they borrow `&row.name`), then MOVES `row.name`
//! into the DTO — the same output, one fewer alloc per row.
//!
//! Run:
//! cargo run --release --features bench --example bench_resource_row_map
//! Tunables (env): BENCH_ROWS (500).
use std::alloc::{GlobalAlloc, Layout, System};
use std::env;
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::Instant;
use chrono::{DateTime, TimeZone, Utc};
use oxicloud::application::dtos::display_helpers::{
category_for, format_file_size, icon_class_for, icon_special_class_for, intern_display,
intern_mime,
};
use oxicloud::application::dtos::file_dto::FileDto;
use oxicloud::application::dtos::folder_dto::{FolderDto, FolderResourceRow};
use oxicloud::domain::entities::file::File;
use uuid::Uuid;
static ALLOC_CALLS: AtomicU64 = AtomicU64::new(0);
struct CountingAlloc;
unsafe impl GlobalAlloc for CountingAlloc {
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
unsafe { System.alloc(layout) }
}
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
unsafe { System.dealloc(ptr, layout) }
}
unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
unsafe { System.realloc(ptr, layout, new_size) }
}
unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
ALLOC_CALLS.fetch_add(1, Ordering::Relaxed);
unsafe { System.alloc_zeroed(layout) }
}
}
#[global_allocator]
static GLOBAL: CountingAlloc = CountingAlloc;
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
env::var(key)
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(default)
}
fn rows(n: usize) -> Vec<FolderResourceRow> {
let ts: DateTime<Utc> = Utc.timestamp_opt(1_700_000_000, 0).unwrap();
(0..n)
.map(|i| {
let is_folder = i % 4 == 0;
FolderResourceRow {
resource_type: if is_folder { "folder" } else { "file" }.to_string(),
id: Uuid::new_v4(),
name: if is_folder {
format!("Folder {i:05}")
} else {
format!("document-{i:05}.pdf")
},
parent_id: Some(Uuid::new_v4()),
mime_type: if is_folder {
None
} else {
Some("application/pdf".to_string())
},
size: if is_folder { -1 } else { 4096 },
created_at: ts,
modified_at: ts,
drive_id: Uuid::new_v4(),
blob_hash: if is_folder {
None
} else {
Some("a".repeat(64))
},
sort_str: format!("row {i}"),
type_order: 0,
folder_first: if is_folder { 0 } else { 1 },
}
})
.collect()
}
/// (name, icon_class, category) triple extracted from each produced DTO — the
/// fields the move-vs-clone touches. Used for the equivalence gate.
type Probe = (String, std::sync::Arc<str>, std::sync::Arc<str>);
/// BEFORE — verbatim: `name: row.name.clone()` in both branches.
fn map_before(rows: Vec<FolderResourceRow>) -> Vec<Probe> {
rows.into_iter()
.map(|row| {
if row.resource_type == "folder" {
let resource_id = row.id.to_string();
let dto = FolderDto {
etag: resource_id.clone(),
id: resource_id,
name: row.name.clone(),
path: String::new(),
parent_id: row.parent_id.map(|u| u.to_string()),
drive_id: row.drive_id,
created_at: row.created_at.timestamp() as u64,
modified_at: row.modified_at.timestamp() as u64,
is_root: false,
icon_class: intern_display("fas fa-folder"),
icon_special_class: intern_display("folder-icon"),
category: intern_display("Folder"),
created_by: None,
updated_by: None,
};
(dto.name, dto.icon_class, dto.category)
} else {
let mime = row
.mime_type
.as_deref()
.unwrap_or("application/octet-stream");
let size_bytes = row.size.max(0) as u64;
let modified_at_u = row.modified_at.timestamp() as u64;
let content_hash = row.blob_hash.clone().unwrap_or_default();
let etag = if content_hash.is_empty() {
String::new()
} else {
File::compute_etag(&content_hash, modified_at_u)
};
let dto = FileDto {
id: row.id.to_string(),
name: row.name.clone(),
path: String::new(),
size: size_bytes,
mime_type: intern_mime(mime),
folder_id: row.parent_id.map(|u| u.to_string()),
created_at: row.created_at.timestamp() as u64,
modified_at: row.modified_at.timestamp() as u64,
icon_class: intern_display(icon_class_for(&row.name, mime)),
icon_special_class: intern_display(icon_special_class_for(&row.name, mime)),
category: intern_display(category_for(&row.name, mime)),
size_formatted: format_file_size(size_bytes),
sort_date: None,
content_hash,
etag,
created_by: None,
updated_by: None,
};
(dto.name, dto.icon_class, dto.category)
}
})
.collect()
}
/// AFTER — icons/category first (borrow `&row.name`), then move `row.name`.
fn map_after(rows: Vec<FolderResourceRow>) -> Vec<Probe> {
rows.into_iter()
.map(|row| {
if row.resource_type == "folder" {
let resource_id = row.id.to_string();
let dto = FolderDto {
etag: resource_id.clone(),
id: resource_id,
name: row.name,
path: String::new(),
parent_id: row.parent_id.map(|u| u.to_string()),
drive_id: row.drive_id,
created_at: row.created_at.timestamp() as u64,
modified_at: row.modified_at.timestamp() as u64,
is_root: false,
icon_class: intern_display("fas fa-folder"),
icon_special_class: intern_display("folder-icon"),
category: intern_display("Folder"),
created_by: None,
updated_by: None,
};
(dto.name, dto.icon_class, dto.category)
} else {
let mime = row
.mime_type
.as_deref()
.unwrap_or("application/octet-stream");
let size_bytes = row.size.max(0) as u64;
let modified_at_u = row.modified_at.timestamp() as u64;
let content_hash = row.blob_hash.clone().unwrap_or_default();
let etag = if content_hash.is_empty() {
String::new()
} else {
File::compute_etag(&content_hash, modified_at_u)
};
let icon_class = intern_display(icon_class_for(&row.name, mime));
let icon_special_class = intern_display(icon_special_class_for(&row.name, mime));
let category = intern_display(category_for(&row.name, mime));
let dto = FileDto {
id: row.id.to_string(),
name: row.name,
path: String::new(),
size: size_bytes,
mime_type: intern_mime(mime),
folder_id: row.parent_id.map(|u| u.to_string()),
created_at: row.created_at.timestamp() as u64,
modified_at: row.modified_at.timestamp() as u64,
icon_class,
icon_special_class,
category,
size_formatted: format_file_size(size_bytes),
sort_date: None,
content_hash,
etag,
created_by: None,
updated_by: None,
};
(dto.name, dto.icon_class, dto.category)
}
})
.collect()
}
fn main() {
let n: usize = env_or("BENCH_ROWS", 500);
// Equivalence gate: identical (name, icon_class, category) for every row.
if map_before(rows(n)) != map_after(rows(n)) {
eprintln!("EQUIVALENCE GATE FAILED: mapping output differs");
std::process::exit(1);
}
// Warm the string interner so its first-sight allocs sit outside the
// measured windows (they're identical for both arms anyway).
std::hint::black_box(map_before(rows(n)));
std::hint::black_box(map_after(rows(n)));
let a0 = ALLOC_CALLS.load(Ordering::Relaxed);
let t = Instant::now();
std::hint::black_box(map_before(rows(n)));
let before_ms = t.elapsed().as_secs_f64() * 1e3;
let before_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a0;
let a1 = ALLOC_CALLS.load(Ordering::Relaxed);
let t = Instant::now();
std::hint::black_box(map_after(rows(n)));
let after_ms = t.elapsed().as_secs_f64() * 1e3;
let after_allocs = ALLOC_CALLS.load(Ordering::Relaxed) - a1;
// Both arms build the same `rows(n)` input inside the timed window, so the
// input allocs are equal and cancel in the delta; the difference is the
// per-row name clone the AFTER path avoids.
println!("\n#################################################################");
println!("# resources row→DTO mapping: clone name vs move name");
println!("# rows={n}");
println!("#################################################################\n");
println!(
"| {:<20} | {:>12} | {:>10} | {:>14} |",
"arm", "allocs", "wall ms", "allocs/row"
);
println!(
"| {:<20} | {:>12} | {:>10.3} | {:>14.3} |",
"BEFORE (clone)",
before_allocs,
before_ms,
before_allocs as f64 / n as f64
);
println!(
"| {:<20} | {:>12} | {:>10.3} | {:>14.3} |",
"AFTER (move)",
after_allocs,
after_ms,
after_allocs as f64 / n as f64
);
println!(
"\nSaved {} allocs ({:.2}/row) — the per-row name clone removed.",
before_allocs.saturating_sub(after_allocs),
(before_allocs.saturating_sub(after_allocs)) as f64 / n as f64
);
}
frontend/src/lib/utils/photoTimeline.bench.test.ts
+163
View File
@@ -0,0 +1,163 @@
import { describe, expect, it } from 'vitest';
import type { PhotoItem } from '$lib/api/endpoints/photos';
import {
PhotoTimeline,
buildPhotoRows,
type GroupMode,
type LayoutMode,
type TimelineConfig
} from './photoTimeline';
/**
* Benchmark gate for the incremental photo timeline (PhotoTimeline) that
* replaced the photos view's `groups`→`photoRows` derive chain.
*
* Audit finding: `loadMore` does `items = [...items, ...page]` (60/page), and
* both `groups` (O(N), a `new Date()` per photo) and `photoRows` (O(N) row
* layout) are `$derived` over the whole accumulated list — so paging to photo
* N re-groups + re-lays-out everything loaded so far, Σ ≈ O(N²/60) main-thread
* work during the scroll (the same class ROUND6 fixed for the files listing).
* Since pages arrive newest-first, grouping is append-only; PhotoTimeline
* re-buckets only the fresh page and re-lays-out only the groups that changed.
*
* Gates:
* 1. Equivalence — at EVERY page of the drain, the incremental output is
* deep-equal to the verbatim full-rebuild reference (buildPhotoRows), for
* both layouts; plus config-change, deletion and width=0 fall back to a
* correct full rebuild.
* 2. Perf — grouping work (timestamp reads) collapses from Σ O(N²/60) to O(N)
* across the drain (deterministic count), and wall drops ≥3x.
*/
const DAY = 86_400; // seconds
/** A photo with a descending sort_date and a deterministic aspect ratio. */
function photo(i: number): PhotoItem {
// Newest-first: photo 0 is most recent; ~half a day apart spans ~4 years
// over 3k photos, so month/day buckets are bounded (realistic library).
const sortDate = 1_700_000_000 - i * (DAY / 2);
const w = 200 + ((i * 37) % 400);
const h = 200 + ((i * 53) % 300);
return {
category: 'image',
created_at: sortDate,
icon_class: '',
icon_special_class: '',
id: `p-${i.toString().padStart(6, '0')}`,
mime_type: 'image/jpeg',
modified_at: sortDate,
name: `photo ${i}.jpg`,
created_by: null,
updated_by: null,
folder_id: 'f',
path: `/photo ${i}.jpg`,
size: 1000,
size_formatted: '1 KB',
sort_date: sortDate,
etag: `e${i}`,
content_hash: `h${i}`,
width: w,
height: h
} as PhotoItem;
}
/** Instrumented config: counts every timestamp read (the grouping hot op). */
function makeConfig(
groupMode: GroupMode,
layoutMode: LayoutMode,
width: number,
counter?: { n: number }
): TimelineConfig {
const timestampOf = (p: PhotoItem) => {
if (counter) counter.n++;
const v = p.sort_date || p.created_at || 0;
return v < 1e12 ? v * 1000 : v;
};
// Stable label fn (reference identity matters for the config-unchanged path).
const labelOf = (d: Date, mode: GroupMode) =>
mode === 'year'
? `${d.getFullYear()}`
: mode === 'month'
? `${d.getFullYear()}-${d.getMonth() + 1}`
: `${d.getFullYear()}-${d.getMonth() + 1}-${d.getDate()}`;
return { groupMode, layoutMode, width, mobile: false, timestampOf, labelOf };
}
const PAGE = 60;
const PAGES = 50; // 3 000-photo drain
const WIDTH = 1200;
describe('incremental photo timeline (benchmark gate)', () => {
for (const layout of ['square', 'justified'] as LayoutMode[]) {
it(`stays deep-equal to the full rebuild at every page — ${layout}`, () => {
const all = Array.from({ length: PAGE * PAGES }, (_, i) => photo(i));
const cfg = makeConfig('month', layout, WIDTH);
const timeline = new PhotoTimeline();
for (let p = 1; p <= PAGES; p++) {
const cumulative = all.slice(0, p * PAGE);
const incremental = timeline.sync(cumulative, cfg);
const reference = buildPhotoRows(cumulative, cfg);
expect(incremental, `page ${p}`).toEqual(reference);
}
});
}
it('falls back to a correct full rebuild on config change, deletion and width=0', () => {
const all = Array.from({ length: 600 }, (_, i) => photo(i));
const timeline = new PhotoTimeline();
const monthSquare = makeConfig('month', 'square', WIDTH);
// Drain a few pages, then flip layout — must equal a fresh full rebuild.
timeline.sync(all.slice(0, 300), monthSquare);
const justified = makeConfig('month', 'justified', WIDTH);
expect(timeline.sync(all.slice(0, 300), justified)).toEqual(
buildPhotoRows(all.slice(0, 300), justified)
);
// Change group mode.
const yearJust = makeConfig('year', 'justified', WIDTH);
expect(timeline.sync(all.slice(0, 300), yearJust)).toEqual(
buildPhotoRows(all.slice(0, 300), yearJust)
);
// Deletion (list shrinks / prefix changes) → rebuild.
const shrunk = all.slice(0, 300).filter((_, i) => i % 7 !== 0);
expect(timeline.sync(shrunk, yearJust)).toEqual(buildPhotoRows(shrunk, yearJust));
// width=0 yields [] and doesn't wedge the next positive-width sync.
const zero = makeConfig('year', 'justified', 0);
expect(timeline.sync(shrunk, zero)).toEqual([]);
expect(timeline.sync(shrunk, yearJust)).toEqual(buildPhotoRows(shrunk, yearJust));
});
it('collapses grouping work from Σ O(N²/page) to O(N) and runs ≥3x faster', () => {
const N = PAGE * PAGES;
const all = Array.from({ length: N }, (_, i) => photo(i));
// AFTER: incremental — each photo is bucketed exactly once across the drain.
const afterCounter = { n: 0 };
const afterCfg = makeConfig('month', 'square', WIDTH, afterCounter);
const timeline = new PhotoTimeline();
const t1 = performance.now();
for (let p = 1; p <= PAGES; p++) timeline.sync(all.slice(0, p * PAGE), afterCfg);
const afterMs = performance.now() - t1;
// BEFORE: full rebuild per page — re-buckets the whole cumulative list.
const beforeCounter = { n: 0 };
const beforeCfg = makeConfig('month', 'square', WIDTH, beforeCounter);
const t0 = performance.now();
for (let p = 1; p <= PAGES; p++) buildPhotoRows(all.slice(0, p * PAGE), beforeCfg);
const beforeMs = performance.now() - t0;
console.info(
`photo timeline ${PAGES}×${PAGE}: before ${beforeCounter.n} timestamp reads / ${beforeMs.toFixed(1)} ms — after ${afterCounter.n} reads / ${afterMs.toFixed(1)} ms (${(beforeCounter.n / afterCounter.n).toFixed(1)}x fewer reads, ${(beforeMs / afterMs).toFixed(1)}x wall)`
);
// Incremental buckets each photo once: exactly N reads.
expect(afterCounter.n).toBe(N);
// Full rebuild is quadratic: Σ_{p=1..P} p·PAGE.
expect(beforeCounter.n).toBe((PAGES * (PAGES + 1) * PAGE) / 2);
expect(afterCounter.n).toBeLessThan(beforeCounter.n / 5);
expect(afterMs).toBeLessThan(beforeMs / 3);
});
});
frontend/src/lib/utils/photoTimeline.ts
+279
View File
@@ -0,0 +1,279 @@
/**
* Photo-timeline grouping + row layout, extracted from the photos view so the
* O(N²) accumulation of its `groups`/`photoRows` derives can be replaced with
* an incremental builder (and unit/benchmark-tested off the Svelte reactive
* graph).
*
* Photos arrive newest-first (`media_sort_date DESC`), so each fetched page
* only ever extends the last date bucket or appends new buckets after it —
* never mutates an earlier group. {@link PhotoTimeline} exploits that: an
* append re-buckets only the new page and recomputes rows only for the groups
* that actually changed, keeping a full scroll O(N) instead of O(N²).
*
* The pure {@link buildPhotoRows} is the verbatim reference (what the old
* `groups`→`photoRows` derive chain produced); the benchmark gate asserts the
* incremental builder stays byte-for-byte equal to it.
*/
import type { PhotoItem } from '$lib/api/endpoints/photos';
export type GroupMode = 'day' | 'month' | 'year';
export type LayoutMode = 'square' | 'justified';
export interface JustifiedTile {
file: PhotoItem;
w: number;
h: number;
}
export type PhotoRow =
| { kind: 'header'; key: string; height: number; label: string; count: number }
| { kind: 'tiles'; key: string; height: number; gap: number; tiles: JustifiedTile[] };
/** Layout constants — mirror the photos view's original values exactly. */
export const SQUARE_GAP = 4; // .25rem, matches the old grid gap
export const SQUARE_MIN = 144; // 9rem minmax floor
export const JUSTIFIED_GAP = 8; // .photos-jrow margin-bottom
export const HEADER_H = 44;
export interface TimelineConfig {
groupMode: GroupMode;
layoutMode: LayoutMode;
/** Usable content width of the grid, in px. */
width: number;
/** `(max-width: 768px)` — selects the 150px vs 200px justified target. */
mobile: boolean;
/** EXIF-aware capture timestamp (ms). Injected so the module stays pure. */
timestampOf: (p: PhotoItem) => number;
/** Locale-aware bucket label for a group's representative date. */
labelOf: (d: Date, mode: GroupMode) => string;
}
interface Group {
key: string;
label: string;
photos: PhotoItem[];
}
/** Year/month/day bucket key for a date under `groupMode` (verbatim). */
export function bucketKey(d: Date, groupMode: GroupMode): string {
const y = d.getFullYear();
if (groupMode === 'year') return `${y}`;
const m = `${d.getMonth() + 1}`.padStart(2, '0');
if (groupMode === 'month') return `${y}-${m}`;
return `${y}-${m}-${`${d.getDate()}`.padStart(2, '0')}`;
}
/**
* Pack files into justified rows (Flickr-style): each full row is scaled to
* fill `width` while preserving every tile's aspect ratio. Missing dimensions
* fall back to 1:1. Verbatim port of the photos view's `justifiedRows`, with
* the `matchMedia` read hoisted to the `mobile` flag so it's testable.
*/
export function justifiedRows(
files: PhotoItem[],
width: number,
mobile: boolean
): Array<{ height: number; tiles: JustifiedTile[] }> {
const gap = 8;
const target = mobile ? 150 : 200;
const rows: Array<{ height: number; tiles: JustifiedTile[] }> = [];
let cur: Array<{ file: PhotoItem; aspect: number }> = [];
let aspectSum = 0;
for (const file of files) {
let aspect = file.width && file.height ? file.width / file.height : 1;
if (!Number.isFinite(aspect) || aspect <= 0) aspect = 1;
aspect = Math.min(Math.max(aspect, 0.4), 3);
cur.push({ file, aspect });
aspectSum += aspect;
const rowWidth = aspectSum * target + (cur.length - 1) * gap;
if (rowWidth >= width) {
const h = (width - (cur.length - 1) * gap) / aspectSum;
rows.push({
height: Math.round(h),
tiles: cur.map((tt) => ({
file: tt.file,
w: Math.max(1, Math.round(tt.aspect * h)),
h: Math.round(h)
}))
});
cur = [];
aspectSum = 0;
}
}
if (cur.length) {
rows.push({
height: target,
tiles: cur.map((tt) => ({
file: tt.file,
w: Math.max(1, Math.round(tt.aspect * target)),
h: target
}))
});
}
return rows;
}
/** Columns + cell size for the square layout at width `W` (verbatim). */
function squareGeometry(W: number): { cols: number; cell: number } {
const cols = Math.max(1, Math.floor((W + SQUARE_GAP) / (SQUARE_MIN + SQUARE_GAP)));
const cell = (W - (cols - 1) * SQUARE_GAP) / cols;
return { cols, cell };
}
/** Flatten one group into its header + tile rows (verbatim per-group body). */
function groupToRows(g: Group, cfg: TimelineConfig, cols: number, cell: number): PhotoRow[] {
const rows: PhotoRow[] = [
{ kind: 'header', key: `h:${g.key}`, height: HEADER_H, label: g.label, count: g.photos.length }
];
if (cfg.layoutMode === 'justified') {
const jrows = justifiedRows(g.photos, cfg.width, cfg.mobile);
for (let ri = 0; ri < jrows.length; ri++) {
rows.push({
kind: 'tiles',
key: `${g.key}:j${ri}`,
height: jrows[ri].height + JUSTIFIED_GAP,
gap: JUSTIFIED_GAP,
tiles: jrows[ri].tiles
});
}
} else {
for (let i = 0; i < g.photos.length; i += cols) {
const tiles = g.photos.slice(i, i + cols).map((file) => ({ file, w: cell, h: cell }));
rows.push({
kind: 'tiles',
key: `${g.key}:s${i}`,
height: cell + SQUARE_GAP,
gap: SQUARE_GAP,
tiles
});
}
}
return rows;
}
/** Bucket `items` into date groups, first-appearance order (verbatim). */
function buildGroups(items: PhotoItem[], cfg: TimelineConfig): Group[] {
const out: Group[] = [];
const index = new Map<string, number>();
for (const p of items) {
const d = new Date(cfg.timestampOf(p));
const key = bucketKey(d, cfg.groupMode);
let i = index.get(key);
if (i === undefined) {
i = out.length;
index.set(key, i);
out.push({ key, label: cfg.labelOf(d, cfg.groupMode), photos: [] });
}
out[i].photos.push(p);
}
return out;
}
/**
* Verbatim reference: the flat `PhotoRow[]` the old `groups`→`photoRows`
* derive chain produced for `items` under `cfg`. Returns `[]` for a
* non-positive width, matching the old guard. The benchmark gate holds the
* incremental builder equal to this.
*/
export function buildPhotoRows(items: PhotoItem[], cfg: TimelineConfig): PhotoRow[] {
if (cfg.width <= 0) return [];
const { cols, cell } = squareGeometry(cfg.width);
const rows: PhotoRow[] = [];
for (const g of buildGroups(items, cfg)) {
rows.push(...groupToRows(g, cfg, cols, cell));
}
return rows;
}
function configEq(a: TimelineConfig, b: TimelineConfig): boolean {
return (
a.groupMode === b.groupMode &&
a.layoutMode === b.layoutMode &&
a.width === b.width &&
a.mobile === b.mobile &&
a.timestampOf === b.timestampOf &&
a.labelOf === b.labelOf
);
}
/**
* Incremental photo-timeline builder. Call {@link sync} with the current item
* list and config on every change; it detects the common case — the list grew
* by appending a page while config is unchanged — and re-buckets only the new
* items + re-lays-out only the groups that changed, reusing every untouched
* group's cached rows. Any other change (config, deletion, filter toggle,
* non-append) falls back to a full rebuild, so the result is always identical
* to {@link buildPhotoRows}.
*/
export class PhotoTimeline {
#cfg: TimelineConfig | null = null;
#groups: Group[] = [];
/** Items already bucketed — the append cursor into the last synced list. */
#groupedItems: PhotoItem[] = [];
/** group.key → its cached rows for the current config. */
#rowCache = new Map<string, PhotoRow[]>();
#geom = { cols: 1, cell: 0 };
/** Whether `next` extends `prev` (same prefix objects + strictly longer). */
#isAppend(prev: PhotoItem[], next: PhotoItem[]): boolean {
if (next.length <= prev.length) return false;
// Prefix identity via the boundary object — O(1), the list is only ever
// mutated by appending or by replacing with a filtered copy.
return prev.length === 0 || next[prev.length - 1] === prev[prev.length - 1];
}
#rebuild(items: PhotoItem[], cfg: TimelineConfig): void {
this.#cfg = cfg;
this.#groups = cfg.width > 0 ? buildGroups(items, cfg) : [];
this.#groupedItems = items;
this.#rowCache.clear();
this.#geom = squareGeometry(cfg.width);
}
#extend(items: PhotoItem[], cfg: TimelineConfig): void {
const fresh = items.slice(this.#groupedItems.length);
// The last existing group may grow, so its cached rows are stale.
if (this.#groups.length > 0) {
this.#rowCache.delete(this.#groups[this.#groups.length - 1].key);
}
for (const p of fresh) {
const d = new Date(cfg.timestampOf(p));
const key = bucketKey(d, cfg.groupMode);
const last = this.#groups[this.#groups.length - 1];
if (last && last.key === key) {
last.photos.push(p);
} else {
this.#groups.push({ key, label: cfg.labelOf(d, cfg.groupMode), photos: [p] });
}
}
this.#groupedItems = items;
}
sync(items: PhotoItem[], cfg: TimelineConfig): PhotoRow[] {
if (cfg.width <= 0) {
// Keep the item cursor so a later positive width rebuilds from scratch.
this.#cfg = cfg;
this.#groups = [];
this.#groupedItems = items;
this.#rowCache.clear();
return [];
}
if (this.#cfg && configEq(this.#cfg, cfg) && this.#isAppend(this.#groupedItems, items)) {
this.#extend(items, cfg);
} else {
this.#rebuild(items, cfg);
}
const { cols, cell } = this.#geom;
const out: PhotoRow[] = [];
for (const g of this.#groups) {
let rows = this.#rowCache.get(g.key);
if (rows === undefined) {
rows = groupToRows(g, cfg, cols, cell);
this.#rowCache.set(g.key, rows);
}
for (const r of rows) out.push(r);
}
return out;
}
}
frontend/src/routes/photos/+page.svelte
+35 -139
View File
@@ -17,6 +17,12 @@
import { filterDotfiles } from '$lib/utils/dotfileFilter';
import { dateTimeFormatFor } from '$lib/utils/display';
import { isVideo, photoTimestamp } from '$lib/utils/media';
import {
PhotoTimeline,
type GroupMode,
type LayoutMode,
type PhotoRow
} from '$lib/utils/photoTimeline';
type Tab = 'moments' | 'places' | 'people';
let tab = $state<Tab>('moments');
@@ -49,8 +55,6 @@
/** Usable content width of the grid, for the justified layout. */
let gridWidth = $state(0);
type GroupMode = 'day' | 'month' | 'year';
type LayoutMode = 'square' | 'justified';
const GROUP_KEY = 'oxi-photos-group';
const LAYOUT_KEY = 'oxi-photos-layout';
let groupMode = $state<GroupMode>('month');
@@ -64,18 +68,10 @@
else if (tab === 'people') void peopleView.load();
});
/** EXIF-aware timestamp (seconds → ms), matching the OLD grouping logic. */
function bucketKey(d: Date): string {
const y = d.getFullYear();
if (groupMode === 'year') return `${y}`;
const m = `${d.getMonth() + 1}`.padStart(2, '0');
if (groupMode === 'month') return `${y}-${m}`;
return `${y}-${m}-${`${d.getDate()}`.padStart(2, '0')}`;
}
function bucketLabel(d: Date): string {
if (groupMode === 'year') return `${d.getFullYear()}`;
if (groupMode === 'month')
/** Locale-aware label for a bucket's representative date. */
function bucketLabel(d: Date, mode: GroupMode): string {
if (mode === 'year') return `${d.getFullYear()}`;
if (mode === 'month')
return dateTimeFormatFor(undefined, { year: 'numeric', month: 'long' }).format(d);
return dateTimeFormatFor(undefined, {
weekday: 'long',
@@ -85,132 +81,32 @@
}).format(d);
}
const groups = $derived.by(() => {
const out: Array<{ key: string; label: string; photos: PhotoItem[] }> = [];
// Transient scratch map built inside $derived.by and discarded — not reactive state.
// eslint-disable-next-line svelte/prefer-svelte-reactivity
const index = new Map<string, number>();
for (const p of visibleItems) {
const d = new Date(photoTimestamp(p));
const key = bucketKey(d);
let i = index.get(key);
if (i === undefined) {
i = out.length;
index.set(key, i);
out.push({ key, label: bucketLabel(d), photos: [] });
}
out[i].photos.push(p);
}
return out;
});
interface JustifiedTile {
file: PhotoItem;
w: number;
h: number;
}
/**
* Pack files into justified rows (Flickr-style): each full row is scaled to
* fill `width` while preserving every tile's aspect ratio. Missing dimensions
* fall back to 1:1.
*/
function justifiedRows(
files: PhotoItem[],
width: number
): Array<{ height: number; tiles: JustifiedTile[] }> {
const gap = 8;
const target = window.matchMedia('(max-width: 768px)').matches ? 150 : 200;
const rows: Array<{ height: number; tiles: JustifiedTile[] }> = [];
let cur: Array<{ file: PhotoItem; aspect: number }> = [];
let aspectSum = 0;
for (const file of files) {
let aspect = file.width && file.height ? file.width / file.height : 1;
if (!Number.isFinite(aspect) || aspect <= 0) aspect = 1;
aspect = Math.min(Math.max(aspect, 0.4), 3);
cur.push({ file, aspect });
aspectSum += aspect;
const rowWidth = aspectSum * target + (cur.length - 1) * gap;
if (rowWidth >= width) {
const h = (width - (cur.length - 1) * gap) / aspectSum;
rows.push({
height: Math.round(h),
tiles: cur.map((tt) => ({
file: tt.file,
w: Math.max(1, Math.round(tt.aspect * h)),
h: Math.round(h)
}))
});
cur = [];
aspectSum = 0;
}
}
if (cur.length) {
rows.push({
height: target,
tiles: cur.map((tt) => ({
file: tt.file,
w: Math.max(1, Math.round(tt.aspect * target)),
h: target
}))
});
}
return rows;
}
// ── Virtualized row model ────────────────────────────────────────────────
// Flatten the groups into a single list of fixed-height rows (a date header
// or a strip of sized tiles), so VirtualRows can window the whole timeline —
// only the rows near the viewport are mounted, regardless of library size.
const SQUARE_GAP = 4; // .25rem, matches the old grid gap
const SQUARE_MIN = 144; // 9rem minmax floor
const JUSTIFIED_GAP = 8; // .photos-jrow margin-bottom
const HEADER_H = 44;
type PhotoRow =
| { kind: 'header'; key: string; height: number; label: string; count: number }
| { kind: 'tiles'; key: string; height: number; gap: number; tiles: JustifiedTile[] };
const photoRows = $derived.by<PhotoRow[]>(() => {
const W = gridWidth;
if (W <= 0) return [];
const rows: PhotoRow[] = [];
const cols = Math.max(1, Math.floor((W + SQUARE_GAP) / (SQUARE_MIN + SQUARE_GAP)));
const cell = (W - (cols - 1) * SQUARE_GAP) / cols;
for (const g of groups) {
rows.push({
kind: 'header',
key: `h:${g.key}`,
height: HEADER_H,
label: g.label,
count: g.photos.length
});
if (layoutMode === 'justified') {
const jrows = justifiedRows(g.photos, W);
for (let ri = 0; ri < jrows.length; ri++) {
rows.push({
kind: 'tiles',
key: `${g.key}:j${ri}`,
height: jrows[ri].height + JUSTIFIED_GAP,
gap: JUSTIFIED_GAP,
tiles: jrows[ri].tiles
});
}
} else {
for (let i = 0; i < g.photos.length; i += cols) {
const tiles = g.photos.slice(i, i + cols).map((file) => ({ file, w: cell, h: cell }));
rows.push({
kind: 'tiles',
key: `${g.key}:s${i}`,
height: cell + SQUARE_GAP,
gap: SQUARE_GAP,
tiles
});
}
}
}
return rows;
});
// Flatten the date groups into a single list of fixed-height rows (a header
// or a strip of sized tiles) that VirtualRows windows. Because pages arrive
// newest-first, each append only extends the last group or adds new ones, so
// PhotoTimeline re-buckets only the fresh page and re-lays-out only the
// groups that changed — a full scroll stays O(N), not O(N²) (the old
// `groups`→`photoRows` derive chain re-grouped + re-packed the whole library
// on every 60-item page). See photoGrouping.bench.test.ts.
// `sync` mutates the timeline's (non-reactive) internal group/row caches and
// returns the flat rows. Driven from `$derived.by` for idempotence: if the
// deps re-fire without an actual append, `sync` sees a non-growing list and
// safely full-rebuilds — same output as the pure `buildPhotoRows`.
const timeline = new PhotoTimeline();
const photoRows = $derived.by<PhotoRow[]>(() =>
timeline.sync(visibleItems, {
groupMode,
layoutMode,
width: gridWidth,
mobile:
typeof window !== 'undefined' &&
typeof window.matchMedia === 'function' &&
window.matchMedia('(max-width: 768px)').matches,
timestampOf: photoTimestamp,
labelOf: bucketLabel
})
);
async function loadMore() {
if (loading || exhausted) return;
src/application/services/file_retrieval_service.rs
-16
View File
@@ -287,22 +287,6 @@ impl FileRetrievalService {
Ok(files.into_iter().map(FileDto::from).collect())
}
/// Range read that first consults the RAM content cache (see
/// [`Self::get_file_range_preloaded`]).
pub async fn get_file_range_preloaded_with_perms(
&self,
dto: &FileDto,
caller_id: Uuid,
start: u64,
end: Option<u64>,
) -> Result<RangeContent, DomainError> {
self.require_file(&dto.id, Permission::Read, caller_id)
.await?;
// Same throttled Recent recording as the streaming variant.
self.notify_file_accessed(caller_id, &dto.id);
self.get_file_range_preloaded(dto, start, end).await
}
/// Range read for HTTP Range Requests, cache-aware.
///
/// Media players and PDF viewers fetch these files *exclusively* through
src/interfaces/api/handlers/file_handler.rs
+8 -6
View File
@@ -712,13 +712,15 @@ impl FileHandler {
let disposition =
Self::content_disposition(&file_dto.name, &file_dto.mime_type, &params);
// `file_dto` was already Read-authorized (and the access
// recorded) by `get_file_with_perms` above — every seek in
// a media/PDF scrub is a separate Range request, so
// re-authorizing + re-notifying per seek doubled that work
// for nothing. Use the non-perms range read, matching the
// share-landing and WebDAV range paths which authorize once
// then stream (benches/ROUND7.md).
match retrieval
.get_file_range_preloaded_with_perms(
&file_dto,
auth_user.id,
start,
Some(end + 1),
)
.get_file_range_preloaded(&file_dto, start, Some(end + 1))
.await
{
Ok(content) => {
src/interfaces/api/handlers/folder_handler.rs
+15 -7
View File
@@ -476,7 +476,9 @@ pub async fn list_folder_resources(
let dto = FolderDto {
etag: resource_id.clone(),
id: resource_id,
name: row.name.clone(),
// Folders use fixed icon classes (below), so `name`
// is never borrowed again — move it instead of cloning.
name: row.name,
path: String::new(), // cleared — share recipients must not see hierarchy
parent_id: row.parent_id.map(|u| u.to_string()),
drive_id: row.drive_id,
@@ -514,20 +516,26 @@ pub async fn list_folder_resources(
} else {
File::compute_etag(&content_hash, modified_at_u)
};
// Compute the name-derived icon/category classes first
// (they borrow `&row.name`), so `name` can be moved into
// the DTO below instead of cloned — one fewer String
// alloc per file row (benches/ROUND7.md).
let icon_class = intern_display(icon_class_for(&row.name, mime));
let icon_special_class =
intern_display(icon_special_class_for(&row.name, mime));
let category = intern_display(category_for(&row.name, mime));
let dto = FileDto {
id: row.id.to_string(),
name: row.name.clone(),
name: row.name,
path: String::new(),
size: size_bytes,
mime_type: intern_mime(mime),
folder_id: row.parent_id.map(|u| u.to_string()),
created_at: row.created_at.timestamp() as u64,
modified_at: row.modified_at.timestamp() as u64,
icon_class: intern_display(icon_class_for(&row.name, mime)),
icon_special_class: intern_display(icon_special_class_for(
&row.name, mime,
)),
category: intern_display(category_for(&row.name, mime)),
icon_class,
icon_special_class,
category,
size_formatted: format_file_size(size_bytes),
sort_date: None,
content_hash,