feat(faces): real ONNX face analyzer (SCRFD + ArcFace), opt-in
Implements the last Phase 2 piece: a working face detector/embedder behind
the new `faces-onnx` cargo feature (mirrors how `plugins` gates wasmtime).
Inert by default — the default build is unchanged and ships the no-op
analyzer.
Pipeline (InsightFace/immich pattern): SCRFD detection with 5-point
landmarks → least-squares similarity alignment to the canonical 112×112
template → ArcFace embedding → L2-normalized 512-d vector.
- face_geometry.rs (always compiled, unit-tested): SCRFD anchor/distance
decode, NMS, the closed-form (complex-number) similarity transform,
bilinear affine warp, NCHW normalization, L2-norm, Laplacian sharpness.
11 unit tests cover the error-prone math with no model needed.
- onnx_face_analyzer.rs (feature `faces-onnx`): wires the geometry to ONNX
Runtime via `ort` (load-dynamic, so libonnxruntime is dlopen'd at runtime
and the crate builds without it). Inference runs on spawn_blocking; each
session is serialized behind a Mutex. Loads via `ort::init_from` (fallible)
not ORT's lazy loader, which would panic under `panic = "abort"`.
- config: FacesConfig + OXICLOUD_FACES_{ORT_DYLIB,DETECTOR_MODEL,
EMBEDDER_MODEL,DET_SIZE,DET_THRESHOLD,NMS_THRESHOLD,INTRA_THREADS}.
- di: build_face_analyzer() loads the real analyzer when the feature is
compiled in and runtime+models are configured; any missing piece or load
failure degrades to the no-op analyzer (logged) so startup never fails.
- ort/ndarray added as optional deps; example.env documents the setup.
Models and the ONNX Runtime dylib are operator-provided at runtime and are
never committed. Cannot be exercised in CI (no models/dylib); the geometry
is unit-tested and the ONNX seam is isolated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JW6ghFMDtnRYuYNzZhb47M
This commit is contained in:
+37
@@ -224,6 +224,43 @@ DATABASE_URL=postgres://postgres:postgres@localhost:5432/oxicloud
|
||||
# Set to false to prevent users from browsing the user directory.
|
||||
#OXICLOUD_EXPOSE_SYSTEM_USERS=true
|
||||
|
||||
# ── People (face recognition) ────────────────────────────────────────────
|
||||
# Biometric data (GDPR Art. 9) — OFF by default, opt-in per deployment.
|
||||
# Detects faces and clusters them into people in the photo library.
|
||||
#
|
||||
# Requires ALL of:
|
||||
# 1. a binary built with the `faces-onnx` cargo feature
|
||||
# (`cargo build --release --features faces-onnx`),
|
||||
# 2. OXICLOUD_ENABLE_FACES=true,
|
||||
# 3. the ONNX Runtime shared library + two operator-provided ONNX models
|
||||
# (a SCRFD/RetinaFace detector with 5-point landmarks, and an ArcFace
|
||||
# 512-d embedder — e.g. InsightFace `buffalo_l`). Models are NOT shipped.
|
||||
# Without all three, the People pipeline stays inert (no-op analyzer) and the
|
||||
# server still boots; the People tab stays hidden in the UI.
|
||||
#OXICLOUD_ENABLE_FACES=false
|
||||
|
||||
# Path to libonnxruntime.{so,dylib,dll}. Falls back to ORT_DYLIB_PATH.
|
||||
# Use the ONNX Runtime build matching this app's `ort` crate (>= 1.24).
|
||||
#OXICLOUD_FACES_ORT_DYLIB=/opt/onnxruntime/lib/libonnxruntime.so
|
||||
|
||||
# Face detector model (SCRFD/RetinaFace, 5-point landmarks).
|
||||
#OXICLOUD_FACES_DETECTOR_MODEL=/var/lib/oxicloud/models/scrfd_10g_bnkps.onnx
|
||||
|
||||
# Face embedder model (ArcFace, 112x112 input -> 512-d output).
|
||||
#OXICLOUD_FACES_EMBEDDER_MODEL=/var/lib/oxicloud/models/w600k_r50.onnx
|
||||
|
||||
# Detector square input size in px (default: 640)
|
||||
#OXICLOUD_FACES_DET_SIZE=640
|
||||
|
||||
# Minimum detector confidence to keep a face, 0..1 (default: 0.5)
|
||||
#OXICLOUD_FACES_DET_THRESHOLD=0.5
|
||||
|
||||
# IoU threshold for non-maximum suppression, 0..1 (default: 0.4)
|
||||
#OXICLOUD_FACES_NMS_THRESHOLD=0.4
|
||||
|
||||
# ONNX Runtime intra-op threads; 0 = let ONNX Runtime decide (default: 0)
|
||||
#OXICLOUD_FACES_INTRA_THREADS=0
|
||||
|
||||
# WASM plugin runtime (Extism). Requires a binary built with the `plugins`
|
||||
# cargo feature (`cargo run --features plugins`); without that feature these
|
||||
# vars are inert. Untrusted plugins run sandboxed: no filesystem, no network,
|
||||
|
||||
Reference in New Issue
Block a user