perf(quota): stop recomputing storage usage on every GET /api/auth/me
GET /api/auth/me ran a synchronous O(N) SUM(size) over all the user's files plus an unconditional UPDATE of auth.users on every call — one of the most frequently hit endpoints — adding per-request latency, DB write load, dead tuples and WAL even when nothing changed. - /api/auth/me now serves the cached storage_used_bytes column instead of recomputing it inline. - New StorageUsageService::start_reconciliation_job runs a periodic sweep on the maintenance pool that keeps the cached value current for every mutation (uploads, deletes, trash), so freshness no longer depends on hitting /me. Interval via OXICLOUD_STORAGE_USAGE_RECONCILE_SECS (default 600s, floored at 30s; first sweep deferred one interval to avoid boot load). - update_storage_usage only writes when the value actually changes (IS DISTINCT FROM), so the sweep produces no dead tuple / WAL on no-ops. - New covering partial index idx_files_user_size_active makes the usage SUM an index-only scan instead of a heap scan over all the user's files. Also collapse the same pre-existing clippy collapsible_else_if in carddav_handler that blocks the -D warnings gate on this base. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -52,7 +52,12 @@ impl StorageUsageService {
|
||||
}
|
||||
|
||||
/// Calculates a user's storage usage by summing all their file sizes.
|
||||
/// Uses a direct SQL query for O(1) performance.
|
||||
///
|
||||
/// This is `SUM(size)` over the user's non-trashed files — O(number of
|
||||
/// files), backed by the `idx_files_user_size_active` covering partial
|
||||
/// index so it runs as an index-only scan. It is NOT called on the request
|
||||
/// path; only by the per-upload update and the background reconciliation
|
||||
/// sweep.
|
||||
async fn calculate_user_storage_usage(&self, user_id: Uuid) -> Result<i64, DomainError> {
|
||||
debug!("Calculating storage for user: {}", user_id);
|
||||
|
||||
@@ -105,6 +110,37 @@ impl StorageUsageService {
|
||||
|
||||
Ok(total_usage)
|
||||
}
|
||||
|
||||
/// Spawn a background task that periodically reconciles every user's cached
|
||||
/// `storage_used_bytes` against the actual sum of their files.
|
||||
///
|
||||
/// `GET /api/auth/me` no longer recomputes usage on the request path; this
|
||||
/// sweep (plus the per-upload update) keeps the cached value current for
|
||||
/// all mutations — including deletes and trash — without any O(N) work on a
|
||||
/// hot endpoint. Runs on the maintenance pool. The first sweep is deferred
|
||||
/// by one interval so it never adds load at boot.
|
||||
pub fn start_reconciliation_job(&self, interval_secs: u64) {
|
||||
// Floor the interval so a misconfiguration can't busy-loop the sweep.
|
||||
let interval_secs = interval_secs.max(30);
|
||||
let service = self.clone();
|
||||
info!(
|
||||
"Starting storage-usage reconciliation job (every {}s)",
|
||||
interval_secs
|
||||
);
|
||||
task::spawn(async move {
|
||||
let mut ticker = tokio::time::interval(std::time::Duration::from_secs(interval_secs));
|
||||
// tokio's first `tick()` fires immediately — consume it so the
|
||||
// first real sweep happens one interval after startup.
|
||||
ticker.tick().await;
|
||||
loop {
|
||||
ticker.tick().await;
|
||||
debug!("Running scheduled storage-usage reconciliation");
|
||||
if let Err(e) = service.update_all_users_storage_usage().await {
|
||||
error!("Scheduled storage-usage reconciliation failed: {}", e);
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
Reference in New Issue
Block a user