cjw/Oxicloud
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

perf: round 12 — auth write-path narrowing, fused quota gate, moka blob-cache index, media single-read, sized listing JSON

Browse Source 
Benchmark-gated round (benches/ROUND12.md; every change ships with a
BEFORE/AFTER harness + equivalence gates, one candidate rejected by its
own bench):

DB / query shapes (bench_round12_queries):
- NC sharee search: username-only projection instead of the 21-column row
  (incl. the <=512 KiB avatar) per match, + gin_trgm_ops indexes on
  auth.users for the leading-wildcard ILIKE (4.98x; 54.7x with index).
- Password login: delete the redundant full-row update_user — create_session
  already stamps last_login_at in its own txn (4.45x per login).
- Email-verified stamp: narrow conditional UPDATE (8.9x); OIDC repeat login
  now compares profile state in memory and issues ZERO queries when nothing
  changed (was: full 17-column rewrite per login).
- Refresh rotation: revoke+insert+stamp fused into one transaction via new
  rotate_session port method (1.18x).
- WOPI CheckFileInfo / authorize_wopi_access: require(Read) + get_file +
  check(Update) overlapped with tokio::join!, original result precedence
  (cold 1.34x).
- Upload quota gate: user-envelope + drive-cap checks fused into ONE
  round-trip (check_upload_quotas) — the NC chunked PUT pays this per
  chunk (1.81x, 2 -> 1 queries/chunk); shared verdict evaluators keep
  error shapes byte-identical.

CPU / allocs (bench_round12_micro):
- sized_json: pre-sized listing serialization replacing axum Json's 128 B
  seed + doubling-realloc chain on files/folder-resources/photos/search
  responses (1.40x, 13 -> 2 allocs per 500-row page; byte-identical).
- Security headers: 4 SetResponseHeaderLayer folded into the CSP middleware
  pass (5 layers -> 1; 1.43x per request, -26 allocs; header set gated
  byte-identical incl. 304s).
- Media capture-metadata: single-read extraction — nom-exif now parses the
  buffer kamadak already read (zero-copy Bytes) and videos open once with a
  kind() dispatch; per-image opens 2-3 -> 1 (1.44x warm geomean, 1.6-3.2x
  cold cache; extraction outputs gated identical incl. the MIME-mislabel
  track fallback).
- Chunked-upload session ops: owner gate folded into the operation's own
  DashMap lookup + stack-encoded uuid compare (5 -> 3 lookups, -2 allocs,
  1.28x per chunk).

Blob cache (bench_blob_cache_index + round-3 regression guard):
- CachedBlobBackend index: tokio::sync::Mutex<LruCache> -> moka::sync::Cache
  with byte weigher. The mutex serialized every cached chunk read and scaled
  NEGATIVELY (2.08 -> 1.07 Mops/s from 1 -> 2 readers); moka probes are
  lock-free (2.17x at K=2). Byte budget now enforced by moka (manual
  current_size + collect_evictions machinery deleted); eviction listener
  unlinks size-evicted files only (Replaced entries keep their file —
  gated). Single-flight miss gate unchanged (16 concurrent misses -> 1
  fetch re-verified via the round-3 harness).
- put_blob now populates the cache BEFORE the inner backend consumes the
  source file (the old order failed 100% of the time — local renames,
  S3/Azure delete the source — so the first read after a whole-file put
  re-downloaded from the remote); inner-put failure invalidates the entry.

Frontend (vitest gates):
- List-view thumbnails request the 150px icon rendition instead of 400px
  preview into a 40px slot (~7.1x fewer pixels, ~4-5x fewer bytes per
  thumbnail across list views); grid keeps preview.

Rejected by its own bench (kept as evidence in bench_round12_micro §2):
- Single-pass compression predicate: the monomorphized And-chain already
  costs ~4.6 ns / 0 allocs total; the fused node measured within noise.

New migration: 20260719000000_users_search_trgm.sql (trgm indexes).
Deferred with prepared design: grouped file/grid view virtualization
(single-VirtualRows flatten, the photos pattern) — next round's headline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
This commit is contained in:
Claude
2026-07-19 01:32:00 +00:00
parent a793cd62eb
commit 50eca0627f
33 changed files with 3989 additions and 410 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/application/services/storage_usage_service.rs
+134 -30
View File
@@ -16,6 +16,10 @@ use uuid::Uuid;
* Storage usage is calculated directly from the `storage.files` table
* by summing file sizes for each user (using the `user_id` column).
*/
/// Fused quota-gate row: `(user_used, user_quota, drive_used, drive_quota,
/// drive_found)` — see [`StorageUsageService::check_upload_quotas`].
type QuotaPairRow = (i64, i64, Option<i64>, Option<i64>, bool);
pub struct StorageUsageService {
pool: Arc<PgPool>,
user_repository: Arc<UserPgRepository>,
@@ -372,6 +376,18 @@ impl StorageUsageService {
// first, so this branch fires only on a deleted-drive race.
return Err(DomainError::not_found("Drive", drive_id.to_string()));
};
Self::eval_drive_cap(used, quota, additional_bytes)
}
/// Drive-cap verdict over already-fetched counters. Shared by
/// [`Self::check_drive_quota`] and the fused
/// [`Self::check_upload_quotas`] pair so both produce byte-identical
/// errors.
fn eval_drive_cap(
used: i64,
quota: Option<i64>,
additional_bytes: u64,
) -> Result<(), DomainError> {
let Some(quota) = quota else {
return Ok(()); // unlimited
};
@@ -391,6 +407,123 @@ impl StorageUsageService {
Ok(())
}
/// User-envelope verdict over already-fetched counters. Shared by
/// `check_storage_quota` and the fused [`Self::check_upload_quotas`]
/// pair so both produce byte-identical errors.
fn eval_user_envelope(used: i64, quota: i64, additional_bytes: u64) -> Result<(), DomainError> {
// Quota of 0 means unlimited
if quota <= 0 {
return Ok(());
}
let additional = additional_bytes as i64;
// Case 1: the single file alone exceeds the entire quota
if additional > quota {
let quota_fmt = format_bytes(quota);
let file_fmt = format_bytes(additional);
return Err(DomainError::quota_exceeded(format!(
"File size ({}) exceeds your total storage quota ({})",
file_fmt, quota_fmt
)));
}
// Case 2: the upload would push usage over the quota
if used + additional > quota {
let available = (quota - used).max(0);
let avail_fmt = format_bytes(available);
let file_fmt = format_bytes(additional);
return Err(DomainError::quota_exceeded(format!(
"Not enough storage space. File size: {}, available: {}",
file_fmt, avail_fmt
)));
}
Ok(())
}
/// Fused pre-upload gate: user envelope + drive cap in ONE round-trip.
///
/// Upload entry points used to run `check_storage_quota` then
/// `check_drive_quota` as two serial point reads — and the NC chunked
/// PUT pays that pair on EVERY chunk. One `LEFT JOIN` row carries both
/// counter pairs; verdict precedence (user envelope first, then drive
/// existence, then drive cap) and every error shape are identical to
/// the two-call sequence (benches/ROUND12.md §6, 1.81x).
///
/// Row shape shared with [`Self::check_upload_quotas_by_folder`]:
/// `(user_used, user_quota, drive_used, drive_quota, drive_found)`.
pub async fn check_upload_quotas(
&self,
user_id: Uuid,
drive_id: Uuid,
additional_bytes: u64,
) -> Result<(), DomainError> {
let row: Option<QuotaPairRow> = sqlx::query_as(
r#"
SELECT u.storage_used_bytes, u.storage_quota_bytes,
d.used_bytes, d.quota_bytes, (d.id IS NOT NULL)
FROM auth.users u
LEFT JOIN storage.drives d ON d.id = $2
WHERE u.id = $1
"#,
)
.bind(user_id)
.bind(drive_id)
.fetch_optional(self.pool.as_ref())
.await
.map_err(|e| {
DomainError::internal_error("StorageUsage", format!("upload quota lookup: {e}"))
})?;
let Some((uused, uquota, dused, dquota, drive_found)) = row else {
return Err(DomainError::not_found("User", user_id.to_string()));
};
Self::eval_user_envelope(uused, uquota, additional_bytes)?;
if !drive_found {
return Err(DomainError::not_found("Drive", drive_id.to_string()));
}
Self::eval_drive_cap(dused.unwrap_or(0), dquota, additional_bytes)
}
/// [`Self::check_upload_quotas`] with the drive resolved from a parent
/// folder id — for the REST upload paths, which hold `folder_id`.
/// A missing folder (or a folder whose drive vanished mid-race) maps to
/// `not_found("Folder")`, exactly like `check_drive_quota_by_folder`.
pub async fn check_upload_quotas_by_folder(
&self,
user_id: Uuid,
folder_id: Uuid,
additional_bytes: u64,
) -> Result<(), DomainError> {
let row: Option<QuotaPairRow> = sqlx::query_as(
r#"
SELECT u.storage_used_bytes, u.storage_quota_bytes,
d.used_bytes, d.quota_bytes, (d.id IS NOT NULL)
FROM auth.users u
LEFT JOIN storage.folders f ON f.id = $2
LEFT JOIN storage.drives d ON d.id = f.drive_id
WHERE u.id = $1
"#,
)
.bind(user_id)
.bind(folder_id)
.fetch_optional(self.pool.as_ref())
.await
.map_err(|e| {
DomainError::internal_error("StorageUsage", format!("upload quota lookup: {e}"))
})?;
let Some((uused, uquota, dused, dquota, drive_found)) = row else {
return Err(DomainError::not_found("User", user_id.to_string()));
};
Self::eval_user_envelope(uused, uquota, additional_bytes)?;
if !drive_found {
return Err(DomainError::not_found("Folder", folder_id.to_string()));
}
Self::eval_drive_cap(dused.unwrap_or(0), dquota, additional_bytes)
}
/// Same as [`Self::check_drive_quota`] but resolves the drive id
/// from a parent folder id. Mirrors
/// [`Self::add_drive_storage_usage_delta_by_folder`] so the upload
@@ -563,36 +696,7 @@ impl StorageUsagePort for StorageUsageService {
// Narrow 2-column read — the full user row carries the up-to-512 KiB
// avatar `image` column, paid on every upload quota check otherwise.
let (used, quota) = self.user_repository.get_storage_usage(user_id).await?;
// Quota of 0 means unlimited
if quota <= 0 {
return Ok(());
}
let additional = additional_bytes as i64;
// Case 1: the single file alone exceeds the entire quota
if additional > quota {
let quota_fmt = format_bytes(quota);
let file_fmt = format_bytes(additional);
return Err(DomainError::quota_exceeded(format!(
"File size ({}) exceeds your total storage quota ({})",
file_fmt, quota_fmt
)));
}
// Case 2: the upload would push usage over the quota
if used + additional > quota {
let available = (quota - used).max(0);
let avail_fmt = format_bytes(available);
let file_fmt = format_bytes(additional);
return Err(DomainError::quota_exceeded(format!(
"Not enough storage space. File size: {}, available: {}",
file_fmt, avail_fmt
)));
}
Ok(())
Self::eval_user_envelope(used, quota, additional_bytes)
}
async fn get_user_storage_info(&self, user_id: Uuid) -> Result<(i64, i64), DomainError> {