perf(quota): stop recomputing storage usage on every GET /api/auth/me

GET /api/auth/me ran a synchronous O(N) SUM(size) over all the user's
files plus an unconditional UPDATE of auth.users on every call — one of
the most frequently hit endpoints — adding per-request latency, DB write
load, dead tuples and WAL even when nothing changed.

- /api/auth/me now serves the cached storage_used_bytes column instead of
  recomputing it inline.
- New StorageUsageService::start_reconciliation_job runs a periodic sweep
  on the maintenance pool that keeps the cached value current for every
  mutation (uploads, deletes, trash), so freshness no longer depends on
  hitting /me. Interval via OXICLOUD_STORAGE_USAGE_RECONCILE_SECS (default
  600s, floored at 30s; first sweep deferred one interval to avoid boot load).
- update_storage_usage only writes when the value actually changes
  (IS DISTINCT FROM), so the sweep produces no dead tuple / WAL on no-ops.
- New covering partial index idx_files_user_size_active makes the usage
  SUM an index-only scan instead of a heap scan over all the user's files.

Also collapse the same pre-existing clippy collapsible_else_if in
carddav_handler that blocks the -D warnings gate on this base.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
DioCrafts
2026-06-07 01:13:58 +02:00
parent c5e0800336
commit d7c6894c80
8 changed files with 98 additions and 33 deletions
@@ -52,7 +52,12 @@ impl StorageUsageService {
}
/// Calculates a user's storage usage by summing all their file sizes.
/// Uses a direct SQL query for O(1) performance.
///
/// This is `SUM(size)` over the user's non-trashed files — O(number of
/// files), backed by the `idx_files_user_size_active` covering partial
/// index so it runs as an index-only scan. It is NOT called on the request
/// path; only by the per-upload update and the background reconciliation
/// sweep.
async fn calculate_user_storage_usage(&self, user_id: Uuid) -> Result<i64, DomainError> {
debug!("Calculating storage for user: {}", user_id);
@@ -105,6 +110,37 @@ impl StorageUsageService {
Ok(total_usage)
}
/// Spawn a background task that periodically reconciles every user's cached
/// `storage_used_bytes` against the actual sum of their files.
///
/// `GET /api/auth/me` no longer recomputes usage on the request path; this
/// sweep (plus the per-upload update) keeps the cached value current for
/// all mutations — including deletes and trash — without any O(N) work on a
/// hot endpoint. Runs on the maintenance pool. The first sweep is deferred
/// by one interval so it never adds load at boot.
pub fn start_reconciliation_job(&self, interval_secs: u64) {
// Floor the interval so a misconfiguration can't busy-loop the sweep.
let interval_secs = interval_secs.max(30);
let service = self.clone();
info!(
"Starting storage-usage reconciliation job (every {}s)",
interval_secs
);
task::spawn(async move {
let mut ticker = tokio::time::interval(std::time::Duration::from_secs(interval_secs));
// tokio's first `tick()` fires immediately — consume it so the
// first real sweep happens one interval after startup.
ticker.tick().await;
loop {
ticker.tick().await;
debug!("Running scheduled storage-usage reconciliation");
if let Err(e) = service.update_all_users_storage_usage().await {
error!("Scheduled storage-usage reconciliation failed: {}", e);
}
}
});
}
}
/**