perf: round 9 — decorator PUT reactivation, session/search/dedup alloc purges, PROPFIND join!, folder-level cascade
Benchmark-gated round (benches/ROUND9.md): every change carries a BEFORE/AFTER bench with equivalence/safety gates; verdicts below are from the committed harnesses on 4 cores / local PG 16. Backend: - Blob decorators (Retry/Cached) now forward put_blob_from_bytes_unsynced + sync_blobs — the trait default had silently reinstated HEAD-before-PUT per chunk on decorated remote stacks, undoing ROUND3 §8. Full production stack: 500 probes -> 0, 1.9x wall at 10 ms RTT (bench_s3_put §3). - NC PROPFIND per-page enrichment triple (favorites / oc:fileid / dead props) overlapped with tokio::join!: 2.07x local, 2.86x at 5 ms RTT (bench_nc_enrich_join, injected-latency decide-by-bench). - Search enrichment consumes its DTOs and carries the interned Arc<str> display fields end-to-end (SearchFileResultDto type change, OpenAPI shape preserved): enrich_file 2.0x, 11.6 -> 2.2 allocs/row; the NC REPORT conversion stops re-running all three classifiers per row (bench_search_enrich). - NC session Arc end-to-end: SharedNcSession extractor (8 -> 0 allocs), Arc<FolderDto> chroot cache (4 -> 0/hit), single shared Arc<CurrentUser> + lazy span render (11 -> 6/build) (bench_nc_session). - Storage micro-pack: atomic create_new chunk writes (2.1x fresh), stream_chunks over the manifest Arc (4097 -> 0 allocs/read incl. the Range path), manifest single-flight (herd 64 -> 1 loads), hex_lower for chunk Content-MD5 (18 -> 1 allocs) (bench_storage_micro). - OCS capabilities memoized into OnceLock<[Bytes;2]>: 237x, 102 -> 0 allocs/poll, byte-identical (bench_capabilities_static). - Drive::is_empty COUNT(*) sum -> EXISTS: 34.4x on a 100k-file drive (bench_drive_is_empty). - favorites/recents row-map ROUND7 port: path/name/blob_hash moved, -2.75 allocs/row (bench_resource_row_map §2). - Folder rows decode binary UUIDs (ROUND6 §10 port): 1.03-1.07x page fetch, honest verdict incl. one noise-band wash documented (bench_folder_uuid_decode). - Authz: file cascade decision decomposed into memoized folder-level decision + direct-grant lookup (ROUND8 deferred item). Cold shared-album first view 592 -> 418 µs/thumb; warm path unchanged; safety gates incl. new direct-grant sibling isolation, revoke-flush re-verified, full integration authz suite green (bench_thumbnail_cascade_cache). Frontend (vitest gates committed beside the code): - resolveLabel/resolveRecipient O(directory) scan -> id-keyed Map: 13.9x (recipients.bench.test.ts). - ResourceList selection-prune effect skips when nothing is selected (100 -> 0 Set builds per drain) and the photos timeline reads a listener-fed mobile flag instead of matchMedia per recompute (listDerives.bench.test.ts). Verification: cargo fmt + clippy --all-features --all-targets -D warnings clean; 524 unit + 554 integration (--cfg integration_tests) tests pass; frontend npm run check clean with 293 vitest tests green. Deferred with rationale in ROUND9.md: CalDAV authz-before-fetch reorder (maintainer sign-off), per-page batched parent resolution, JWT-claims Arc<str>, batch_operations signature widening. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDc9VtXvskJ6dnMRraSndn
This commit is contained in:
Claude
@@ -0,0 +1,247 @@
|
||||
//! Folder-listing UUID decode benchmark — `id::text`/`parent_id::text`
|
||||
//! server casts vs binary `Uuid` decode + one app-side render.
|
||||
//!
|
||||
//! Round 6 adopted binary decode for the FILE listing rows
|
||||
//! (`row_to_file`, benches/ROUND6.md §10: 1.17x on 500-row pages) and
|
||||
//! queued "other repos with the same shape" — `FolderDbRepository` never
|
||||
//! got the port. Its rows (`list_folders`, `list_folders_batch` — every
|
||||
//! Depth:1 PROPFIND subfolder page — descendants, suggest) still shipped
|
||||
//! two `::text` casts per row: 36+36 B on the wire instead of 16+16 and
|
||||
//! a server-side cast per column.
|
||||
//!
|
||||
//! Same methodology as `bench_uuid_text_cast` (the round-6 A/B this
|
||||
//! ports): seeded page, equivalence gate on identical `(id, parent_id,
|
||||
//! name, path)` string tuples, warm-up, interleaved passes.
|
||||
//!
|
||||
//! Run (needs Postgres up; reads DATABASE_URL from .env):
|
||||
//! cargo run --release --features bench --example bench_folder_uuid_decode
|
||||
//! Tunables (env): BENCH_ROWS (500), BENCH_PASSES (200)
|
||||
|
||||
use std::env;
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use sqlx::PgPool;
|
||||
use sqlx::postgres::PgPoolOptions;
|
||||
use uuid::Uuid;
|
||||
|
||||
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
|
||||
env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
struct Seeded {
|
||||
drive_id: Uuid,
|
||||
parent_id: Uuid,
|
||||
}
|
||||
|
||||
async fn seed(pool: &PgPool, rows: usize) -> Seeded {
|
||||
let mut tx = pool.begin().await.expect("begin");
|
||||
let drive_id: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO storage.drives (kind, quota_bytes) VALUES ('shared', NULL) RETURNING id",
|
||||
)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("drive");
|
||||
let root: Uuid = sqlx::query_scalar(
|
||||
"INSERT INTO storage.folders (name, path, lpath, drive_id)
|
||||
VALUES ('bench_uuid_folders', '/bench_uuid_folders', 'bench_uuid_folders', $1)
|
||||
RETURNING id",
|
||||
)
|
||||
.bind(drive_id)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.expect("root");
|
||||
sqlx::query("UPDATE storage.drives SET root_folder_id = $1 WHERE id = $2")
|
||||
.bind(root)
|
||||
.bind(drive_id)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.expect("stamp root");
|
||||
tx.commit().await.expect("commit");
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO storage.folders (name, parent_id, path, lpath, drive_id)
|
||||
SELECT 'sub' || i, $1, '/bench_uuid_folders/sub' || i,
|
||||
('bench_uuid_folders.sub' || i)::ltree, $2
|
||||
FROM generate_series(1, $3) AS i",
|
||||
)
|
||||
.bind(root)
|
||||
.bind(drive_id)
|
||||
.bind(rows as i32)
|
||||
.execute(pool)
|
||||
.await
|
||||
.expect("seed subfolders");
|
||||
|
||||
Seeded {
|
||||
drive_id,
|
||||
parent_id: root,
|
||||
}
|
||||
}
|
||||
|
||||
async fn cleanup(pool: &PgPool, s: &Seeded) {
|
||||
sqlx::query("DELETE FROM storage.folders WHERE drive_id = $1 AND parent_id IS NOT NULL")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("UPDATE storage.drives SET root_folder_id = NULL WHERE id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM storage.folders WHERE drive_id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
sqlx::query("DELETE FROM storage.drives WHERE id = $1")
|
||||
.bind(s.drive_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.ok();
|
||||
}
|
||||
|
||||
/// Materialized tuple both arms must produce identically.
|
||||
type FolderTuple = (String, String, String, Option<String>);
|
||||
|
||||
/// BEFORE — verbatim old query shape: two server-side `::text` casts,
|
||||
/// decode as String.
|
||||
async fn fetch_text_cast(pool: &PgPool, parent_id: Uuid) -> Vec<FolderTuple> {
|
||||
sqlx::query_as::<_, (String, String, String, Option<String>)>(
|
||||
r#"
|
||||
SELECT id::text, name, path, parent_id::text
|
||||
FROM storage.folders
|
||||
WHERE parent_id = $1 AND NOT is_trashed
|
||||
ORDER BY name
|
||||
"#,
|
||||
)
|
||||
.bind(parent_id)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.expect("text-cast fetch")
|
||||
}
|
||||
|
||||
/// AFTER — the production shape: binary decode, one `to_string` app-side
|
||||
/// (exactly what `row_to_folder` does now).
|
||||
async fn fetch_binary_uuid(pool: &PgPool, parent_id: Uuid) -> Vec<FolderTuple> {
|
||||
let rows = sqlx::query_as::<_, (Uuid, String, String, Option<Uuid>)>(
|
||||
r#"
|
||||
SELECT id, name, path, parent_id
|
||||
FROM storage.folders
|
||||
WHERE parent_id = $1 AND NOT is_trashed
|
||||
ORDER BY name
|
||||
"#,
|
||||
)
|
||||
.bind(parent_id)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.expect("binary fetch");
|
||||
rows.into_iter()
|
||||
.map(|(id, name, path, pid)| (id.to_string(), name, path, pid.map(|u| u.to_string())))
|
||||
.collect()
|
||||
}
|
||||
|
||||
struct Stats {
|
||||
mean_ms: f64,
|
||||
p50_ms: f64,
|
||||
p95_ms: f64,
|
||||
}
|
||||
|
||||
fn summarize(mut xs: Vec<f64>) -> Stats {
|
||||
xs.sort_by(|a, b| a.partial_cmp(b).unwrap());
|
||||
let n = xs.len();
|
||||
Stats {
|
||||
mean_ms: xs.iter().sum::<f64>() / n as f64,
|
||||
p50_ms: xs[n / 2],
|
||||
p95_ms: xs[((n as f64 * 0.95) as usize).min(n - 1)],
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::main(flavor = "multi_thread")]
|
||||
async fn main() {
|
||||
dotenvy::dotenv().ok();
|
||||
let url = env::var("DATABASE_URL")
|
||||
.or_else(|_| env::var("OXICLOUD_DB_CONNECTION_STRING"))
|
||||
.expect("set DATABASE_URL — the dev Postgres URL");
|
||||
let rows: usize = env_or("BENCH_ROWS", 500);
|
||||
let passes: usize = env_or("BENCH_PASSES", 200);
|
||||
|
||||
let pool = Arc::new(
|
||||
PgPoolOptions::new()
|
||||
.max_connections(4)
|
||||
.min_connections(4)
|
||||
.acquire_timeout(Duration::from_secs(10))
|
||||
.connect(&url)
|
||||
.await
|
||||
.expect("connect Postgres"),
|
||||
);
|
||||
|
||||
let seeded = seed(&pool, rows).await;
|
||||
|
||||
// Equivalence gate: identical string tuples in identical order.
|
||||
let a = fetch_text_cast(&pool, seeded.parent_id).await;
|
||||
let b = fetch_binary_uuid(&pool, seeded.parent_id).await;
|
||||
if a != b || a.len() != rows {
|
||||
eprintln!(
|
||||
"EQUIVALENCE GATE FAILED: rows differ (a={}, b={})",
|
||||
a.len(),
|
||||
b.len()
|
||||
);
|
||||
cleanup(&pool, &seeded).await;
|
||||
std::process::exit(1);
|
||||
}
|
||||
println!("# equivalence gate: {rows} identical (id, name, path, parent_id) tuples — OK");
|
||||
|
||||
for _ in 0..10 {
|
||||
std::hint::black_box(fetch_text_cast(&pool, seeded.parent_id).await);
|
||||
std::hint::black_box(fetch_binary_uuid(&pool, seeded.parent_id).await);
|
||||
}
|
||||
|
||||
// Interleaved A/B passes so drift (autovacuum, CPU governor) hits both.
|
||||
let mut lat_a = Vec::with_capacity(passes);
|
||||
let mut lat_b = Vec::with_capacity(passes);
|
||||
for _ in 0..passes {
|
||||
let t = Instant::now();
|
||||
std::hint::black_box(fetch_text_cast(&pool, seeded.parent_id).await);
|
||||
lat_a.push(t.elapsed().as_secs_f64() * 1e3);
|
||||
let t = Instant::now();
|
||||
std::hint::black_box(fetch_binary_uuid(&pool, seeded.parent_id).await);
|
||||
lat_b.push(t.elapsed().as_secs_f64() * 1e3);
|
||||
}
|
||||
|
||||
let sa = summarize(lat_a);
|
||||
let sb = summarize(lat_b);
|
||||
|
||||
println!("\n#################################################################");
|
||||
println!("# folder page: `::text` casts vs binary UUID decode + app fmt");
|
||||
println!("# rows/page={rows} passes={passes} (interleaved)");
|
||||
println!("#################################################################\n");
|
||||
println!(
|
||||
"| {:<22} | {:>9} | {:>9} | {:>9} |",
|
||||
"arm", "mean ms", "p50 ms", "p95 ms"
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>9.3} | {:>9.3} | {:>9.3} |",
|
||||
"A ::text (before)", sa.mean_ms, sa.p50_ms, sa.p95_ms
|
||||
);
|
||||
println!(
|
||||
"| {:<22} | {:>9.3} | {:>9.3} | {:>9.3} |",
|
||||
"B binary (after)", sb.mean_ms, sb.p50_ms, sb.p95_ms
|
||||
);
|
||||
println!(
|
||||
"\nB/A mean ratio: {:.3} ({:.2}x)",
|
||||
sb.mean_ms / sa.mean_ms,
|
||||
sa.mean_ms / sb.mean_ms
|
||||
);
|
||||
|
||||
cleanup(&pool, &seeded).await;
|
||||
|
||||
if sb.mean_ms >= sa.mean_ms {
|
||||
eprintln!("GATE FAIL: binary decode not faster than ::text — rollback");
|
||||
std::process::exit(1);
|
||||
}
|
||||
println!("GATE PASS");
|
||||
}
|
||||
Reference in New Issue
Block a user