feat(faces): real ONNX face analyzer (SCRFD + ArcFace), opt-in
Implements the last Phase 2 piece: a working face detector/embedder behind
the new `faces-onnx` cargo feature (mirrors how `plugins` gates wasmtime).
Inert by default — the default build is unchanged and ships the no-op
analyzer.
Pipeline (InsightFace/immich pattern): SCRFD detection with 5-point
landmarks → least-squares similarity alignment to the canonical 112×112
template → ArcFace embedding → L2-normalized 512-d vector.
- face_geometry.rs (always compiled, unit-tested): SCRFD anchor/distance
decode, NMS, the closed-form (complex-number) similarity transform,
bilinear affine warp, NCHW normalization, L2-norm, Laplacian sharpness.
11 unit tests cover the error-prone math with no model needed.
- onnx_face_analyzer.rs (feature `faces-onnx`): wires the geometry to ONNX
Runtime via `ort` (load-dynamic, so libonnxruntime is dlopen'd at runtime
and the crate builds without it). Inference runs on spawn_blocking; each
session is serialized behind a Mutex. Loads via `ort::init_from` (fallible)
not ORT's lazy loader, which would panic under `panic = "abort"`.
- config: FacesConfig + OXICLOUD_FACES_{ORT_DYLIB,DETECTOR_MODEL,
EMBEDDER_MODEL,DET_SIZE,DET_THRESHOLD,NMS_THRESHOLD,INTRA_THREADS}.
- di: build_face_analyzer() loads the real analyzer when the feature is
compiled in and runtime+models are configured; any missing piece or load
failure degrades to the no-op analyzer (logged) so startup never fails.
- ort/ndarray added as optional deps; example.env documents the setup.
Models and the ONNX Runtime dylib are operator-provided at runtime and are
never committed. Cannot be exercised in CI (no models/dylib); the geometry
is unit-tested and the ONNX seam is isolated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JW6ghFMDtnRYuYNzZhb47M
This commit is contained in:
@@ -81,6 +81,8 @@ nom-exif = "3.6.1"
|
||||
extism = { version = "1.30.0", optional = true }
|
||||
toml = { version = "1.1.2", optional = true }
|
||||
file-rotate = { version = "0.7.6", optional = true }
|
||||
ort = { version = "2.0.0-rc.12", default-features = false, features = ["load-dynamic", "ndarray", "tracing", "api-24"], optional = true }
|
||||
ndarray = { version = "0.17.2", optional = true }
|
||||
|
||||
[features]
|
||||
default = []
|
||||
@@ -95,6 +97,12 @@ plugins = ["dep:extism", "dep:toml", "dep:file-rotate"]
|
||||
# this lets one `cargo build` produce both `oxicloud` and `load-seed`
|
||||
# without recompiling oxicloud with mockall in scope.
|
||||
load_seed_bin = []
|
||||
# Real ONNX-backed face analyzer (detector + embedder) for the People feature.
|
||||
# Opt-in: pulls `ort` (ONNX Runtime, load-dynamic — dlopen's libonnxruntime at
|
||||
# runtime) + `ndarray`, a heavy stack most deployments won't use. Activation also
|
||||
# requires OXICLOUD_ENABLE_FACES=true *and* operator-provided ONNX models; without
|
||||
# this feature the People pipeline falls back to the inert NoopFaceAnalyzer.
|
||||
faces-onnx = ["dep:ort", "dep:ndarray"]
|
||||
|
||||
[lints.rust]
|
||||
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(integration_tests)'] }
|
||||
|
||||
Reference in New Issue
Block a user