aba89c4f5d
Every change is benchmark-verified (harness + before/after numbers in benches/, measured on this branch; reproduction commands in each doc): DAV / sync-client hot paths - PROPFIND dead-properties: one = ANY($1) query per 500-child page instead of one sequential query per child, and indexable `=` predicates instead of IS NOT DISTINCT FROM (seq scans). 2,000-child folder: 1.07-4.54 s of DB chatter -> 4-6 ms (258-773x). Applied to native + NC PROPFIND and both NC REPORT handlers. [benches/DEAD-PROPS.md] - Folder paging: keyset cursor (name > $last) + new partial index (folder_id, name) replaces LIMIT/OFFSET full-folder rescan per page. Full 20k-file walk: 1266 ms -> 77 ms (16.5x). New migration 20260917000000. [benches/PROPFIND-PAGING.md] - NC chroot / default-drive resolution: moka caches (30 s TTL, explicit invalidation on drive mutations) for find_default_for_user and the markerless chroot FolderDto. 2 uncached queries + 2 pool checkouts per NC/WebDAV/WOPI request -> sub-us moka hit (p50 0.7-3.6 ms -> ~1 us). [benches/CHROOT-CACHE.md] - Quota: PROPFINDs whose prop list never names a quota prop skip the 2-query resolution entirely (wants_quota()); the remaining lookups read 2 columns instead of the full auth.users row with its <=512 KiB avatar (11-16x, p50 3.4 ms -> 0.29 ms). Same narrow read now gates every upload quota check. [benches/QUOTA-PATH.md] CPU on the request path - ZIP exports (folder download, share ZIP, batch download): entries whose MIME says already-compressed (JPEG/MP4/zip/pdf/...) are Stored instead of Deflate - deflate ran inline on the tokio writer task at ~41 MB/s for ~0% size gain. Mixed media corpus: 4.31x wall and CPU, archive size unchanged. Shared predicate in common::mime_detect. [benches/ZIP-MEDIA.md] - Compression layers: tower-http's default maps to Brotli QUALITY 11 (verified in brotli-8.0.2 source and empirically: 90 ms per 64 KiB JSON response, 1.3 s per 700 KiB bundle). Both layers pinned to Precise(4): 99x less CPU for ~15% more bytes. SPA assets are now precompressed at build time (scripts/precompress.mjs, 77% smaller) and served via ServeDir::precompressed_br/gzip: 2016x less per-request work, and clients get the better q11 bytes. [benches/STATIC-PRECOMPRESSED.md] Batched / cached backend paths [benches/NPLUS1-AND-CACHES.md] - Content-search ReBAC re-verification: new AuthorizationEngine::check_files_read_batch (default = old loop; PgAclEngine override batches drive resolution + reuses role cache). 200 sequential point SELECTs per search -> 1-2 queries. - Batch-ZIP subtree downloads: drop per-file re-authz + per-file Recent recording (2 writes/file) for subtree entries already authorized at the root - mirrors the native folder-download path. ~6,000 statements removed from a 2,000-file archive. - CDC chunk manifests: immutable by content address, now moka-cached (weight-bounded 32 MiB, 60 s TTL, positive-only, invalidated on delete) - removes one manifest query (p50 0.44-4.4 ms) from every stream, range and full blob read. - People tab: grouped COUNT + batched cover lookup instead of dragging every face row with its 2 KiB embedding (10k faces: 30.4 ms & 21 MB -> 3.8 ms & 1.3 KB, 8.1x); merge() is one set-based UPDATE. [benches/PEOPLE-LIST.md] - Photos timeline cursor: raw timestamptz comparison instead of EXTRACT(EPOCH ...) wrapper + IS NULL OR disjunction - cursor is an index boundary again, deep scroll stops re-scanning skipped rows. - Public share landing: one atomic UPDATE ... access_count + 1 (was SELECT + full-row write-back: racy, lost updates, clobbered concurrent owner edits) - 3 round-trips -> 2 per visit. - move_to_trash: dead full-entity SELECT feeding a documented no-op removed from both branches; dead fields dropped from TrashService. - NFC normalization: is_nfc_quick fast path skips the decompose/recompose state machine for the ~100% already-NFC case (every row loaded from PG). Frontend - Large folders paint after page one (~200 items) via fetchFolderListing's new onPage hook instead of waiting for every sequential page. - Tested-and-reverted (kept for the record): cached Intl.Collator for name sorts - vitest showed it 2x SLOWER than V8's argument-less localeCompare fast path (5.6 ms vs 12.1 ms / 5k names). Sort order untouched. New bench harnesses under examples/ (bench feature): zip_media, dead_props, chroot_cache, quota_path, people_list, propfind_paging, static_precompress. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w
268 lines
9.0 KiB
Rust
268 lines
9.0 KiB
Rust
//! People (faces) use cases: identity clustering + the read/mutation methods
|
|
//! the HTTP layer calls.
|
|
//!
|
|
//! Clustering is a full re-cluster over the user's faces: a union-find groups
|
|
//! faces whose embeddings are within a cosine threshold (connected
|
|
//! components), and groups of at least `min_faces` become a "person". This is
|
|
//! O(n²) in the user's face count — fine for moderate libraries; an ANN index
|
|
//! (pgvector/VectorChord) is the documented scale-up.
|
|
//!
|
|
//! Strictly user-scoped (the repository filters by user), so — like
|
|
//! `RecentService` / `PlacesService` — no `AuthorizationEngine` check is
|
|
//! needed: the `caller_id` parameter is the access scope.
|
|
|
|
use std::collections::HashMap;
|
|
use std::sync::Arc;
|
|
|
|
use chrono::Utc;
|
|
use uuid::Uuid;
|
|
|
|
use crate::application::dtos::people_dto::{FaceBoxDto, PersonDto};
|
|
use crate::application::ports::face_ports::FaceRepository;
|
|
use crate::common::errors::DomainError;
|
|
use crate::domain::entities::face::Person;
|
|
use crate::infrastructure::repositories::pg::FacePgRepository;
|
|
|
|
/// Cosine similarity of two equal-length vectors. Embeddings are produced
|
|
/// L2-normalized, so this is ~a dot product; we normalize anyway for safety.
|
|
fn cosine(a: &[f32], b: &[f32]) -> f32 {
|
|
if a.len() != b.len() || a.is_empty() {
|
|
return 0.0;
|
|
}
|
|
let (mut dot, mut na, mut nb) = (0.0f32, 0.0f32, 0.0f32);
|
|
for (&x, &y) in a.iter().zip(b.iter()) {
|
|
dot += x * y;
|
|
na += x * x;
|
|
nb += y * y;
|
|
}
|
|
if na == 0.0 || nb == 0.0 {
|
|
return 0.0;
|
|
}
|
|
dot / (na.sqrt() * nb.sqrt())
|
|
}
|
|
|
|
/// Disjoint-set with path-halving + union by rank.
|
|
struct UnionFind {
|
|
parent: Vec<usize>,
|
|
rank: Vec<usize>,
|
|
}
|
|
|
|
impl UnionFind {
|
|
fn new(n: usize) -> Self {
|
|
Self {
|
|
parent: (0..n).collect(),
|
|
rank: vec![0; n],
|
|
}
|
|
}
|
|
fn find(&mut self, mut x: usize) -> usize {
|
|
while self.parent[x] != x {
|
|
self.parent[x] = self.parent[self.parent[x]];
|
|
x = self.parent[x];
|
|
}
|
|
x
|
|
}
|
|
fn union(&mut self, a: usize, b: usize) {
|
|
let (ra, rb) = (self.find(a), self.find(b));
|
|
if ra == rb {
|
|
return;
|
|
}
|
|
match self.rank[ra].cmp(&self.rank[rb]) {
|
|
std::cmp::Ordering::Less => self.parent[ra] = rb,
|
|
std::cmp::Ordering::Greater => self.parent[rb] = ra,
|
|
std::cmp::Ordering::Equal => {
|
|
self.parent[rb] = ra;
|
|
self.rank[ra] += 1;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
pub struct PeopleService {
|
|
repo: Arc<FacePgRepository>,
|
|
/// Min cosine similarity to link two faces into the same identity.
|
|
cluster_threshold: f32,
|
|
/// Min faces in a cluster before it becomes a named-able "person".
|
|
min_faces: usize,
|
|
}
|
|
|
|
impl PeopleService {
|
|
pub fn new(repo: Arc<FacePgRepository>) -> Self {
|
|
Self {
|
|
repo,
|
|
cluster_threshold: 0.5,
|
|
min_faces: 3,
|
|
}
|
|
}
|
|
|
|
/// Re-cluster a user's faces. Returns the number of new persons created.
|
|
pub async fn recluster(&self, user_id: Uuid) -> Result<usize, DomainError> {
|
|
let faces = self.repo.faces_for_user(user_id).await?;
|
|
let n = faces.len();
|
|
if n == 0 {
|
|
return Ok(0);
|
|
}
|
|
|
|
let mut uf = UnionFind::new(n);
|
|
for i in 0..n {
|
|
for j in (i + 1)..n {
|
|
if cosine(&faces[i].embedding, &faces[j].embedding) >= self.cluster_threshold {
|
|
uf.union(i, j);
|
|
}
|
|
}
|
|
}
|
|
|
|
let mut groups: HashMap<usize, Vec<usize>> = HashMap::new();
|
|
for i in 0..n {
|
|
let root = uf.find(i);
|
|
groups.entry(root).or_default().push(i);
|
|
}
|
|
|
|
let mut created = 0usize;
|
|
for idxs in groups.into_values() {
|
|
if idxs.len() < self.min_faces {
|
|
// Too small to be a person — leave/reset these faces unassigned.
|
|
for &i in &idxs {
|
|
if faces[i].person_id.is_some() {
|
|
self.repo.assign_person(faces[i].id, None).await?;
|
|
}
|
|
}
|
|
continue;
|
|
}
|
|
|
|
// Reuse an existing person on this cluster (preserves a user's name)
|
|
// or mint a new one.
|
|
let existing = idxs.iter().find_map(|&i| faces[i].person_id);
|
|
let person_id = match existing {
|
|
Some(pid) => pid,
|
|
None => {
|
|
let pid = Uuid::new_v4();
|
|
let person = Person {
|
|
id: pid,
|
|
user_id,
|
|
display_name: None,
|
|
cover_face_id: Some(faces[idxs[0]].id),
|
|
is_hidden: false,
|
|
created_at: Utc::now(),
|
|
};
|
|
self.repo.create_person(&person).await?;
|
|
created += 1;
|
|
pid
|
|
}
|
|
};
|
|
for &i in &idxs {
|
|
if faces[i].person_id != Some(person_id) {
|
|
self.repo
|
|
.assign_person(faces[i].id, Some(person_id))
|
|
.await?;
|
|
}
|
|
}
|
|
let _ = self
|
|
.repo
|
|
.set_person_cover(person_id, faces[idxs[0]].id)
|
|
.await;
|
|
}
|
|
|
|
Ok(created)
|
|
}
|
|
|
|
/// People (non-empty clusters), most-photographed first.
|
|
///
|
|
/// Counts come from a grouped-COUNT query and cover photos from one
|
|
/// batched lookup of just the cover face ids — the previous
|
|
/// `faces_for_user` shipped every face row (2 KiB embedding included)
|
|
/// only to count them: ~20 MB of BYTEA per request on a 10k-face
|
|
/// library (benches/PEOPLE-LIST.md).
|
|
pub async fn list_people(&self, caller_id: Uuid) -> Result<Vec<PersonDto>, DomainError> {
|
|
let persons = self.repo.persons_for_user(caller_id).await?;
|
|
let count: HashMap<Uuid, i64> = self
|
|
.repo
|
|
.person_face_stats(caller_id)
|
|
.await?
|
|
.into_iter()
|
|
.collect();
|
|
let cover_ids: Vec<Uuid> = persons.iter().filter_map(|p| p.cover_face_id).collect();
|
|
let face_file: HashMap<Uuid, Uuid> =
|
|
self.repo.file_ids_for_faces(caller_id, &cover_ids).await?;
|
|
|
|
let mut out: Vec<PersonDto> = persons
|
|
.into_iter()
|
|
.filter_map(|p| {
|
|
let c = count.get(&p.id).copied().unwrap_or(0);
|
|
if c == 0 {
|
|
return None; // hide empty clusters (e.g. after a merge)
|
|
}
|
|
let cover_file_id = p
|
|
.cover_face_id
|
|
.and_then(|fid| face_file.get(&fid).copied())
|
|
.map(|u| u.to_string());
|
|
Some(PersonDto {
|
|
id: p.id.to_string(),
|
|
name: p.display_name,
|
|
cover_file_id,
|
|
face_count: c,
|
|
is_hidden: p.is_hidden,
|
|
})
|
|
})
|
|
.collect();
|
|
out.sort_by_key(|p| std::cmp::Reverse(p.face_count));
|
|
Ok(out)
|
|
}
|
|
|
|
/// File ids of a person's photos (most recent first).
|
|
pub async fn person_photos(
|
|
&self,
|
|
caller_id: Uuid,
|
|
person_id: Uuid,
|
|
) -> Result<Vec<String>, DomainError> {
|
|
let files = self.repo.files_for_person(caller_id, person_id).await?;
|
|
Ok(files.into_iter().map(|u| u.to_string()).collect())
|
|
}
|
|
|
|
/// Face boxes within a photo (for lightbox tagging), caller-scoped.
|
|
pub async fn faces_for_file(
|
|
&self,
|
|
caller_id: Uuid,
|
|
file_id: Uuid,
|
|
) -> Result<Vec<FaceBoxDto>, DomainError> {
|
|
let faces = self.repo.faces_for_file(file_id).await?;
|
|
Ok(faces
|
|
.into_iter()
|
|
.filter(|f| f.user_id == caller_id)
|
|
.map(|f| FaceBoxDto {
|
|
id: f.id.to_string(),
|
|
person_id: f.person_id.map(|u| u.to_string()),
|
|
x: f.bbox.x,
|
|
y: f.bbox.y,
|
|
w: f.bbox.w,
|
|
h: f.bbox.h,
|
|
})
|
|
.collect())
|
|
}
|
|
|
|
pub async fn rename_person(
|
|
&self,
|
|
caller_id: Uuid,
|
|
person_id: Uuid,
|
|
name: Option<String>,
|
|
) -> Result<(), DomainError> {
|
|
self.repo.rename_person(caller_id, person_id, name).await
|
|
}
|
|
|
|
/// Merge `from` into `into` by reassigning all of `from`'s faces. The
|
|
/// now-empty `from` person is hidden by `list_people`.
|
|
///
|
|
/// One set-based UPDATE — the previous shape loaded every face row
|
|
/// (embeddings included) and issued one UPDATE per matching face.
|
|
pub async fn merge(&self, caller_id: Uuid, into: Uuid, from: Uuid) -> Result<(), DomainError> {
|
|
self.repo
|
|
.reassign_person_faces(caller_id, from, into)
|
|
.await?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Erase all of the caller's face data (right to erasure / opt-out).
|
|
pub async fn delete_all(&self, caller_id: Uuid) -> Result<(), DomainError> {
|
|
self.repo.delete_all_for_user(caller_id).await
|
|
}
|
|
}
|