Files
Oxicloud/examples/bench_quota_path.rs
Claude aba89c4f5d perf: eliminate N+1 hot-path queries, cache immutable lookups, stop re-compressing compressed bytes
Every change is benchmark-verified (harness + before/after numbers in
benches/, measured on this branch; reproduction commands in each doc):

DAV / sync-client hot paths
- PROPFIND dead-properties: one = ANY($1) query per 500-child page instead
  of one sequential query per child, and indexable `=` predicates instead
  of IS NOT DISTINCT FROM (seq scans). 2,000-child folder: 1.07-4.54 s of
  DB chatter -> 4-6 ms (258-773x). Applied to native + NC PROPFIND and
  both NC REPORT handlers. [benches/DEAD-PROPS.md]
- Folder paging: keyset cursor (name > $last) + new partial index
  (folder_id, name) replaces LIMIT/OFFSET full-folder rescan per page.
  Full 20k-file walk: 1266 ms -> 77 ms (16.5x). New migration
  20260917000000. [benches/PROPFIND-PAGING.md]
- NC chroot / default-drive resolution: moka caches (30 s TTL, explicit
  invalidation on drive mutations) for find_default_for_user and the
  markerless chroot FolderDto. 2 uncached queries + 2 pool checkouts per
  NC/WebDAV/WOPI request -> sub-us moka hit (p50 0.7-3.6 ms -> ~1 us).
  [benches/CHROOT-CACHE.md]
- Quota: PROPFINDs whose prop list never names a quota prop skip the
  2-query resolution entirely (wants_quota()); the remaining lookups read
  2 columns instead of the full auth.users row with its <=512 KiB avatar
  (11-16x, p50 3.4 ms -> 0.29 ms). Same narrow read now gates every
  upload quota check. [benches/QUOTA-PATH.md]

CPU on the request path
- ZIP exports (folder download, share ZIP, batch download): entries whose
  MIME says already-compressed (JPEG/MP4/zip/pdf/...) are Stored instead
  of Deflate - deflate ran inline on the tokio writer task at ~41 MB/s
  for ~0% size gain. Mixed media corpus: 4.31x wall and CPU, archive size
  unchanged. Shared predicate in common::mime_detect. [benches/ZIP-MEDIA.md]
- Compression layers: tower-http's default maps to Brotli QUALITY 11
  (verified in brotli-8.0.2 source and empirically: 90 ms per 64 KiB JSON
  response, 1.3 s per 700 KiB bundle). Both layers pinned to Precise(4):
  99x less CPU for ~15% more bytes. SPA assets are now precompressed at
  build time (scripts/precompress.mjs, 77% smaller) and served via
  ServeDir::precompressed_br/gzip: 2016x less per-request work, and
  clients get the better q11 bytes. [benches/STATIC-PRECOMPRESSED.md]

Batched / cached backend paths [benches/NPLUS1-AND-CACHES.md]
- Content-search ReBAC re-verification: new
  AuthorizationEngine::check_files_read_batch (default = old loop;
  PgAclEngine override batches drive resolution + reuses role cache).
  200 sequential point SELECTs per search -> 1-2 queries.
- Batch-ZIP subtree downloads: drop per-file re-authz + per-file Recent
  recording (2 writes/file) for subtree entries already authorized at the
  root - mirrors the native folder-download path. ~6,000 statements
  removed from a 2,000-file archive.
- CDC chunk manifests: immutable by content address, now moka-cached
  (weight-bounded 32 MiB, 60 s TTL, positive-only, invalidated on delete)
  - removes one manifest query (p50 0.44-4.4 ms) from every stream,
  range and full blob read.
- People tab: grouped COUNT + batched cover lookup instead of dragging
  every face row with its 2 KiB embedding (10k faces: 30.4 ms & 21 MB ->
  3.8 ms & 1.3 KB, 8.1x); merge() is one set-based UPDATE.
  [benches/PEOPLE-LIST.md]
- Photos timeline cursor: raw timestamptz comparison instead of
  EXTRACT(EPOCH ...) wrapper + IS NULL OR disjunction - cursor is an
  index boundary again, deep scroll stops re-scanning skipped rows.
- Public share landing: one atomic UPDATE ... access_count + 1 (was
  SELECT + full-row write-back: racy, lost updates, clobbered concurrent
  owner edits) - 3 round-trips -> 2 per visit.
- move_to_trash: dead full-entity SELECT feeding a documented no-op
  removed from both branches; dead fields dropped from TrashService.
- NFC normalization: is_nfc_quick fast path skips the decompose/recompose
  state machine for the ~100% already-NFC case (every row loaded from PG).

Frontend
- Large folders paint after page one (~200 items) via fetchFolderListing's
  new onPage hook instead of waiting for every sequential page.
- Tested-and-reverted (kept for the record): cached Intl.Collator for name
  sorts - vitest showed it 2x SLOWER than V8's argument-less localeCompare
  fast path (5.6 ms vs 12.1 ms / 5k names). Sort order untouched.

New bench harnesses under examples/ (bench feature): zip_media,
dead_props, chroot_cache, quota_path, people_list, propfind_paging,
static_precompress.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w
2026-07-16 14:20:20 +00:00

170 lines
5.5 KiB
Rust

//! Quota-path benchmark — full `auth.users` row vs narrow 2-column read.
//!
//! `check_storage_quota` (every upload) and `get_user_storage_info` (every
//! quota-reporting PROPFIND) used to call `get_user_by_id`, whose SELECT
//! drags the whole user row — including `image`, an avatar data URI of up
//! to 512 KiB — across the wire to read two i64s. The change reads only
//! `(storage_used_bytes, storage_quota_bytes)`
//! (`UserPgRepository::get_storage_usage`). Companion change measured here
//! as "SKIP": PROPFINDs whose prop list never names a quota prop now skip
//! the resolution entirely (`PropFindRequest::wants_quota`).
//!
//! Run (needs Postgres up; reads DATABASE_URL from .env):
//! cargo run --release --features bench --example bench_quota_path
//! Tunables: BENCH_SECONDS (4), BENCH_CONCURRENCIES ("8,64"), BENCH_IMAGE_KB (512)
use std::env;
use std::sync::Arc;
use std::time::{Duration, Instant};
use sqlx::PgPool;
use sqlx::postgres::PgPoolOptions;
use uuid::Uuid;
fn env_or<T: std::str::FromStr>(key: &str, default: T) -> T {
env::var(key)
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(default)
}
async fn seed(pool: &PgPool, image_kb: usize) -> Uuid {
// Realistic worst-ish case: an avatar data URI at the documented cap.
let image = format!("data:image/png;base64,{}", "A".repeat(image_kb * 1024 - 22));
sqlx::query_scalar(
"INSERT INTO auth.users (username, email, role, image)
VALUES ('bench_quota', 'bench_quota@bench.invalid', 'user', $1)
RETURNING id",
)
.bind(&image)
.fetch_one(pool)
.await
.expect("seed user")
}
async fn cleanup(pool: &PgPool, user_id: Uuid) {
let _ = sqlx::query("DELETE FROM auth.users WHERE id = $1")
.bind(user_id)
.execute(pool)
.await;
}
/// BEFORE: the full-row SELECT `get_user_by_id` runs (same column list).
async fn one_op_full(pool: &PgPool, id: Uuid) {
let _row = sqlx::query(
r#"
SELECT
id, username, email, password_hash, role::text as role_text,
storage_quota_bytes, storage_used_bytes,
created_at, updated_at, last_login_at, active,
oidc_provider, oidc_subject, image, is_external,
given_name, family_name, email_verified_at, preferred_locale, notify_on_share,
ui_preferences
FROM auth.users
WHERE id = $1
"#,
)
.bind(id)
.fetch_one(pool)
.await
.expect("full row");
}
/// AFTER: the narrow `get_storage_usage` SELECT.
async fn one_op_narrow(pool: &PgPool, id: Uuid) {
let _row: (i64, i64) = sqlx::query_as(
"SELECT storage_used_bytes, storage_quota_bytes FROM auth.users WHERE id = $1",
)
.bind(id)
.fetch_one(pool)
.await
.expect("narrow row");
}
struct Stats {
rps: f64,
p50: f64,
p99: f64,
}
fn summarize(mut lats: Vec<f64>, secs: u64) -> Stats {
lats.sort_by(|a, b| a.partial_cmp(b).unwrap());
let n = lats.len();
let pct = |p: f64| {
if n == 0 {
0.0
} else {
lats[((n as f64 * p) as usize).min(n - 1)]
}
};
Stats {
rps: n as f64 / secs as f64,
p50: pct(0.50),
p99: pct(0.99),
}
}
#[tokio::main(flavor = "multi_thread")]
async fn main() {
dotenvy::dotenv().ok();
let url = env::var("DATABASE_URL").expect("set DATABASE_URL");
let secs: u64 = env_or("BENCH_SECONDS", 4);
let image_kb: usize = env_or("BENCH_IMAGE_KB", 512);
let concurrencies: Vec<usize> = env::var("BENCH_CONCURRENCIES")
.ok()
.map(|s| s.split(',').filter_map(|x| x.trim().parse().ok()).collect())
.unwrap_or_else(|| vec![8, 64]);
let pool = Arc::new(
PgPoolOptions::new()
.max_connections(20)
.min_connections(20)
.acquire_timeout(Duration::from_secs(10))
.connect(&url)
.await
.expect("connect"),
);
let user_id = seed(&pool, image_kb).await;
println!("\n# quota lookup: full user row (incl. {image_kb} KiB avatar) vs 2-column read");
println!(
"| {:>5} | {:<7} | {:>10} | {:>9} | {:>9} |",
"conc", "mode", "ops/s", "p50 µs", "p99 µs"
);
for &conc in &concurrencies {
for mode in ["FULL", "NARROW"] {
let deadline = Instant::now() + Duration::from_secs(secs);
let mut handles = Vec::new();
for _ in 0..conc {
let pool = pool.clone();
let mode = mode.to_string();
handles.push(tokio::spawn(async move {
let mut lats = Vec::new();
while Instant::now() < deadline {
let t = Instant::now();
if mode == "FULL" {
one_op_full(&pool, user_id).await;
} else {
one_op_narrow(&pool, user_id).await;
}
lats.push(t.elapsed().as_secs_f64() * 1e6);
}
lats
}));
}
let mut all = Vec::new();
for h in handles {
all.extend(h.await.unwrap());
}
let s = summarize(all, secs);
println!(
"| {:>5} | {:<7} | {:>10.0} | {:>9.1} | {:>9.1} |",
conc, mode, s.rps, s.p50, s.p99
);
}
}
println!("\n(SKIP: PROPFINDs not naming quota props now issue NEITHER query — 0 round-trips.)");
cleanup(&pool, user_id).await;
}