perf: eliminate N+1 hot-path queries, cache immutable lookups, stop re-compressing compressed bytes

Every change is benchmark-verified (harness + before/after numbers in benches/, measured on this branch; reproduction commands in each doc): DAV / sync-client hot paths - PROPFIND dead-properties: one = ANY($1) query per 500-child page instead of one sequential query per child, and indexable `=` predicates instead of IS NOT DISTINCT FROM (seq scans). 2,000-child folder: 1.07-4.54 s of DB chatter -> 4-6 ms (258-773x). Applied to native + NC PROPFIND and both NC REPORT handlers. [benches/DEAD-PROPS.md] - Folder paging: keyset cursor (name > $last) + new partial index (folder_id, name) replaces LIMIT/OFFSET full-folder rescan per page. Full 20k-file walk: 1266 ms -> 77 ms (16.5x). New migration 20260917000000. [benches/PROPFIND-PAGING.md] - NC chroot / default-drive resolution: moka caches (30 s TTL, explicit invalidation on drive mutations) for find_default_for_user and the markerless chroot FolderDto. 2 uncached queries + 2 pool checkouts per NC/WebDAV/WOPI request -> sub-us moka hit (p50 0.7-3.6 ms -> ~1 us). [benches/CHROOT-CACHE.md] - Quota: PROPFINDs whose prop list never names a quota prop skip the 2-query resolution entirely (wants_quota()); the remaining lookups read 2 columns instead of the full auth.users row with its <=512 KiB avatar (11-16x, p50 3.4 ms -> 0.29 ms). Same narrow read now gates every upload quota check. [benches/QUOTA-PATH.md] CPU on the request path - ZIP exports (folder download, share ZIP, batch download): entries whose MIME says already-compressed (JPEG/MP4/zip/pdf/...) are Stored instead of Deflate - deflate ran inline on the tokio writer task at ~41 MB/s for ~0% size gain. Mixed media corpus: 4.31x wall and CPU, archive size unchanged. Shared predicate in common::mime_detect. [benches/ZIP-MEDIA.md] - Compression layers: tower-http's default maps to Brotli QUALITY 11 (verified in brotli-8.0.2 source and empirically: 90 ms per 64 KiB JSON response, 1.3 s per 700 KiB bundle). Both layers pinned to Precise(4): 99x less CPU for ~15% more bytes. SPA assets are now precompressed at build time (scripts/precompress.mjs, 77% smaller) and served via ServeDir::precompressed_br/gzip: 2016x less per-request work, and clients get the better q11 bytes. [benches/STATIC-PRECOMPRESSED.md] Batched / cached backend paths [benches/NPLUS1-AND-CACHES.md] - Content-search ReBAC re-verification: new AuthorizationEngine::check_files_read_batch (default = old loop; PgAclEngine override batches drive resolution + reuses role cache). 200 sequential point SELECTs per search -> 1-2 queries. - Batch-ZIP subtree downloads: drop per-file re-authz + per-file Recent recording (2 writes/file) for subtree entries already authorized at the root - mirrors the native folder-download path. ~6,000 statements removed from a 2,000-file archive. - CDC chunk manifests: immutable by content address, now moka-cached (weight-bounded 32 MiB, 60 s TTL, positive-only, invalidated on delete) - removes one manifest query (p50 0.44-4.4 ms) from every stream, range and full blob read. - People tab: grouped COUNT + batched cover lookup instead of dragging every face row with its 2 KiB embedding (10k faces: 30.4 ms & 21 MB -> 3.8 ms & 1.3 KB, 8.1x); merge() is one set-based UPDATE. [benches/PEOPLE-LIST.md] - Photos timeline cursor: raw timestamptz comparison instead of EXTRACT(EPOCH ...) wrapper + IS NULL OR disjunction - cursor is an index boundary again, deep scroll stops re-scanning skipped rows. - Public share landing: one atomic UPDATE ... access_count + 1 (was SELECT + full-row write-back: racy, lost updates, clobbered concurrent owner edits) - 3 round-trips -> 2 per visit. - move_to_trash: dead full-entity SELECT feeding a documented no-op removed from both branches; dead fields dropped from TrashService. - NFC normalization: is_nfc_quick fast path skips the decompose/recompose state machine for the ~100% already-NFC case (every row loaded from PG). Frontend - Large folders paint after page one (~200 items) via fetchFolderListing's new onPage hook instead of waiting for every sequential page. - Tested-and-reverted (kept for the record): cached Intl.Collator for name sorts - vitest showed it 2x SLOWER than V8's argument-less localeCompare fast path (5.6 ms vs 12.1 ms / 5k names). Sort order untouched. New bench harnesses under examples/ (bench feature): zip_media, dead_props, chroot_cache, quota_path, people_list, propfind_paging, static_precompress. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w
2026-07-16 14:20:20 +00:00
parent b69c18b934
commit aba89c4f5d
52 changed files with 3262 additions and 444 deletions
@@ -1112,6 +1112,78 @@ impl AuthorizationEngine for PgAclEngine {
        result
    }

+    /// Batched Read check over a page of file ids (see the trait docs).
+    ///
+    /// Decision-equivalent to looping `check`: (1) resolve every file's
+    /// drive in one `= ANY($1)` query (same rows as N ×
+    /// `get_file_drive_id`; absent ids decide `false` exactly like the
+    /// per-file `NotFound` path), (2) evaluate the drive-role floor once
+    /// per distinct drive through the same `drive_role_cache`, (3) send
+    /// only the drive-floor misses through the full per-file cascade —
+    /// preserving per-file grant resolution. `Read` is never gated by the
+    /// read-only drive freeze, so skipping that branch changes nothing.
+    async fn check_files_read_batch(
+        &self,
+        subject: Subject,
+        file_ids: &[Uuid],
+    ) -> Result<std::collections::HashSet<Uuid>, DomainError> {
+        use std::collections::{HashMap, HashSet};
+        let start = std::time::Instant::now();
+        let counters = QueryCounters::default();
+
+        counters.sql_queries.fetch_add(1, Ordering::Relaxed);
+        let pairs = self.file_repo.get_file_drive_ids(file_ids).await?;
+
+        // Prime the resource→drive cache — later single checks on these
+        // files (download, share) skip their point lookup too.
+        for (file_id, drive_id) in &pairs {
+            self.owner_cache
+                .insert(Resource::File(*file_id), *drive_id)
+                .await;
+        }
+
+        let mut drive_readable: HashMap<Uuid, bool> = HashMap::new();
+        for (_, drive_id) in &pairs {
+            if !drive_readable.contains_key(drive_id) {
+                let ok = self
+                    .caller_role_on_drive_cached(subject, *drive_id, &counters)
+                    .await?
+                    .is_some_and(|role| role.expand().contains(&Permission::Read));
+                drive_readable.insert(*drive_id, ok);
+            }
+        }
+
+        let mut allowed: HashSet<Uuid> = HashSet::with_capacity(pairs.len());
+        for (file_id, drive_id) in &pairs {
+            if drive_readable.get(drive_id).copied().unwrap_or(false) {
+                allowed.insert(*file_id);
+            } else if self
+                .check_inner(
+                    subject,
+                    Permission::Read,
+                    Resource::File(*file_id),
+                    &counters,
+                )
+                .await?
+            {
+                // Per-file / folder-cascade grant inside a drive the caller
+                // has no role on — rare, but must keep resolving.
+                allowed.insert(*file_id);
+            }
+        }
+
+        tracing::debug!(
+            target: "oxicloud::authz",
+            event = "authz.check_files_read_batch",
+            subject = %subject,
+            files = file_ids.len(),
+            allowed = allowed.len(),
+            duration_us = start.elapsed().as_micros() as u64,
+            sql_queries = counters.sql_queries.load(Ordering::Relaxed),
+        );
+        Ok(allowed)
+    }
+
    async fn list_incoming_grants(&self, subject: Subject) -> Result<Vec<Grant>, DomainError> {
        let counters = QueryCounters::default();
        let (subject_types, subject_ids) = self.subject_match_set(subject, &counters).await?;