Oxicloud/benches/ROUND23.md

Round 23 — Postgres query-shape pass: typed JSONB decode, drive-policy borrow-deserialize, user-profile join!, subject-group CTE reuse, dedup unzip

Benchmark-gated, same rule as ROUND2–22: every change ships with a BEFORE/AFTER benchmark and a value equivalence gate; an AFTER that doesn't beat its BEFORE is rolled back (never applied). This round is the PostgreSQL pass — the candidates the earlier rounds deferred as "needs a database to bench" — so it ships two harnesses:

• bench_round23_micro (no Postgres) — the deterministic allocation gate for the decode/clone candidates (counting global allocator; a non-winning AFTER std::process::exit(1)s with GATE FAIL … rollback).
• bench_round23_queries (live Postgres) — end-to-end p50 latency + a strict equivalence gate (identical decoded rows / ids / user-sets from BEFORE and AFTER; a mismatch exits 1) against seeded fixtures.

Reproduce (the queries harness reads DATABASE_URL from .env):

cargo run --release --features bench --example bench_round23_micro
cargo run --release --features bench --example bench_round23_queries

Summary

#	change	metric	before → after
J1	contact_pg_repository::row_to_contact (+ the inlined contact_group_pg_repository sibling) decoded each of the 3 JSONB columns (email/phone/address) with row.get::<serde_json::Value> + serde_json::from_value::<Vec<Dto>> — a throwaway Value DOM built per column and then walked a second time to produce the typed Vec. Now row.try_get::<sqlx::types::Json<Vec<Dto>>> decodes the JSONB bytes straight into the typed Vec in one from_slice pass, no DOM. Runs per contact row of every contact list / multiget / CardDAV sync.	micro allocs · PG p50	84 → 33 allocs/op (2.15× wall) · 3794 → 2360 ns/contact (1.61×)
J2	DrivePolicies::from_value did serde_json::from_value(value.clone()) — cloning the entire policies Value DOM before walking it, on every drive-policy read (move/copy, shared-link creation, grant). Now DrivePolicies::deserialize(value) deserializes straight from the borrow (serde_json's Deserializer for &Value), no clone — a one-line body change, byte-identical, all 7 call sites unchanged.	micro allocs	5 → 0 allocs/op (11.51× wall)
P1	AuthApplicationService::get_user_profile issued two independent, serial get_user_by_id point reads (caller then target; the self-case short-circuit compares input UUIDs, not fetched data). Now the self-case does a single fetch and the non-self path overlaps caller+target with tokio::join! (caller_res? first preserves the caller-error precedence).	PG p50	577 → 312 µs/call (1.85×)
G1	SubjectGroupService::remove_member ran the child group's transitive-user recursive CTE twice for a nested Group removal — once in the would-empty pre-check, once in invalidation_targets after the remove. The edge delete is above the child, so its descendants can't change; now the CTE runs once and the result is reused for both.	PG p50	829 → 412 µs/removal (2.01×)
U1	dedup_service (store_loose_chunks final registration + the ingest run_rollback) built Vec<String>/Vec<i64> by cloning every 64-byte hash out of an owned, dead-after Vec<(String,i64)> purely to reshape for sync_blobs(&[String]) + the UNNEST bind. Now into_iter().unzip() moves the hashes out — no per-hash content copy.	micro allocs	256 → 0 hash clones (1283 → 1027 allocs/op on a 256-chunk batch)

The micro allocs/op is the deterministic gate (identical run to run); the PG p50 is single-machine, warm-pool, and noise-bounded. Every section carries a value-equivalence gate; the shipped source matches each AFTER arm.

[J1] Contact JSONB — typed Json<T> decode, no intermediate Value DOM

row_to_contact (reached by 11 call sites — every contact GET / list / paginated list / multiget / CardDAV cursor stream / search / by-email / by-group / create+update RETURNING) and the identical inlined block in contact_group_pg_repository::get_contacts_in_group both did:

let email_json: JsonValue = row.get("email");        // sqlx JSONB → Value DOM (alloc tree)
let emails = serde_json::from_value::<Vec<EmailPersistenceDto>>(email_json)  // walk the DOM again
    .map(emails_from_persistence).unwrap_or_default();
// … same for phone, address

sqlx::types::Json<T> decodes the raw JSONB bytes with a single serde_json::from_slice::<T> (sqlx-core 0.8.6 types/json.rs), skipping the Value tree entirely:

let emails = row
    .try_get::<sqlx::types::Json<Vec<EmailPersistenceDto>>, _>("email")
    .map(|j| emails_from_persistence(j.0))
    .unwrap_or_default();

try_get (not get) preserves the exact malformed-shape fallback — get would panic on a decode error, whereas the old from_value(...).unwrap_or_default() tolerated it. The columns are JSONB NOT NULL DEFAULT '[]', so SQL NULL never occurs. Byte-identical: both paths run the same derived Deserialize<Vec<Dto>> over the same bytes — the bench_round23_queries §Q1 gate asserts the two decode the 500 seeded contacts field-for-field identically. The micro shows the 3 discarded DOMs/row (84 → 33 allocs); on the real rows the decode is 1.61×.

[J2] Drive policies — deserialize from the borrow, don't clone the DOM

DrivePolicies::from_value(value: &serde_json::Value) is called on every drive-policy read (get_policies_for_file/_folder, get_drive_id_and_policies_for_*, update_policies RETURNING, the ACL engine's enforcement read, and Drive::typed_policies). It built the typed struct with serde_json::from_value(value.clone()) — a full clone of the policies DOM purely because from_value consumes its argument. serde_json implements Deserializer for &Value, so the struct can be built straight from the borrow:

use serde::Deserialize as _;
Self::deserialize(value).unwrap_or_default()   // was: serde_json::from_value(value.clone())

Byte-identical (same derived Deserialize, same lenient unwrap_or_default fallback that keeps unknown keys on disk), a one-line body change, and every caller keeps its &Value argument unchanged — so typed_policies(&self) (which only has a borrow of self.policies) also stops cloning. The micro (a realistic bag with a preserved unknown key) drops 5 → 0 allocs/op.

[P1] get_user_profile — overlap the two independent reads with join!

The profile lookup fetched the caller and the target user in two serial round-trips. The self-case (caller_id == target_id) is decided by comparing the input UUIDs, so on the common non-self path the two reads are independent — query 2 never depends on query 1. AFTER:

if caller_id == target_id {                       // self: one fetch, unchanged
    let caller = self.user_storage.get_user_by_id(caller_id).await?;
    return Ok(UserDto::from(caller));
}
let (caller_res, target_res) = tokio::join!(       // non-self: overlap
    self.user_storage.get_user_by_id(caller_id),
    self.user_storage.get_user_by_id(target_id));
let caller = caller_res?;                          // caller-error precedence preserved
let target = match target_res { … };               // identical NotFound→anonymized-404 + audit

Every observable outcome is preserved (self still 1 fetch, the anti-enumeration audit unchanged). The §Q4 gate asserts identical ids from both shapes; two warm-pool serial reads vs the join! measured 1.85×.

[G1] remove_member — compute the child's transitive users once, reuse it

For a nested GroupMember::Group(child_id) removal the child's transitive-user set (a recursive WITH RECURSIVE CTE over subject_group_members) was computed twice: once in the would-empty self-defense pre-check, and again inside invalidation_targets after remove_member deleted the parent→child edge. That edge is above the child, so the child's own descendants are unchanged — verified empirically on the live DB (child set {u2,u3} identical before and after the edge delete). AFTER computes the CTE once, up front, and reuses it for both the pre-check and the cache-invalidation set (invalidation_targets stays for add_member). The §Q6 gate asserts the child set is both stable and the expected {u2,u3}; 2 CTEs vs 1 measured 2.01× on the seeded 3-level tree.

[U1] dedup hash reshape — move via unzip, don't clone

store_loose_chunks's final registration and the ingest run_rollback both reshaped an owned Vec<(String,i64)> (dead after the block) into the Vec<String> + Vec<i64> that sync_blobs(&[String]) and the UNNEST bind need, by cloning every 64-char hash:

let hashes: Vec<String> = new_rows.iter().map(|(h, _)| h.clone()).collect();  // N clones
let sizes:  Vec<i64>    = new_rows.iter().map(|(_, s)| *s).collect();

Since the source is owned and never read again, into_iter().unzip() moves the hashes out — 0 per-hash content copies:

let (hashes, sizes): (Vec<String>, Vec<i64>) = new_rows.into_iter().unzip();

Byte-identical rows inserted; the micro (256 distinct new chunks) drops exactly the 256 hash clones. (This is the move-not-borrow refinement of the ROUND21 §R2 &[&str] pattern — sync_blobs takes &[String], so a borrow would force a port-signature change across 6 backends, whereas the move needs none.)

Not shipped — deferred to a dedicated pass

• batch_operations::download_zip per-item N+1 (the audit's #2, highest raw-latency candidate): the file loop calls get_file_with_perms (itself authz + get_file = 2 round-trips) per selected file, then add_file_entry_streamed — which re-authorizes internally via get_file_stream_with_perms. Collapsing the per-item metadata+authz into a bulk get_files_by_ids + check_files_read_batch prefetch is a real win (2N+2M serial round-trips → ~3 batch queries), but it moves the sole authorization from before the stream to inside it, so it needs a careful AuthZ-ordering + anti-enumeration proof (the project's rule: authz lives in the service layer, denials audit-log and return the anti-enum shape). That is its own validated pass, not a perf banner — queued with a download_zip fixture that seeds a large multi-select and asserts identical ZIP entry set+order across the change.
• Contact/Drive JSONB — the SQL-NULL edge: the typed try_get/deserialize paths return the empty/default on SQL NULL where the old row.get::<Value> would have panicked. Both columns are NOT NULL DEFAULT today so this never fires; noted only so a future nullable-column change re-checks it.

Environment / methodology

• A local PostgreSQL 16 dev instance was provisioned for this round (schema applied via the 67 migrations/*.sql in order; pg_trgm + ltree extensions). bench_round23_queries seeds its own fixtures (unique bench23-* markers) and tears them down (idempotent cleanup) around the run.
• Build note: this session's host intermittently SIGILLed rustc/LLVM codegen under the repo's default -C target-cpu=native (a cascadelake with AVX-512 whose passthrough faulted after a host migration). All Round-23 builds/benches were run with RUSTFLAGS="-C target-cpu=x86-64-v3" (AVX2, no AVX-512) to sidestep it. This is a local build-flag override only — the checked-in .cargo/config.toml is unchanged, and the primary gate (allocs/op) is target-cpu-independent; the PG p50 comparisons use the same flag for both arms, so the relative speedups hold.
• Each micro section: BEFORE (verbatim shipped-before shape) vs AFTER (verbatim shipped-after shape) + a value-equivalence assert + a GATE FAIL … rollback exit if the AFTER fails to reduce allocations. Each PG section: BEFORE vs AFTER shape against real seeded rows + an equivalence gate (mismatch → exit 1)
  • p50 over BENCH_PASSES.
• Verified beyond the benches: cargo fmt --all --check clean, cargo clippy --features bench --all-targets -D warnings clean, and the touched modules' unit tests pass (contact, drive, subject_group, dedup, auth profile).