feat(faces): real ONNX face analyzer (SCRFD + ArcFace), opt-in
Implements the last Phase 2 piece: a working face detector/embedder behind
the new `faces-onnx` cargo feature (mirrors how `plugins` gates wasmtime).
Inert by default — the default build is unchanged and ships the no-op
analyzer.
Pipeline (InsightFace/immich pattern): SCRFD detection with 5-point
landmarks → least-squares similarity alignment to the canonical 112×112
template → ArcFace embedding → L2-normalized 512-d vector.
- face_geometry.rs (always compiled, unit-tested): SCRFD anchor/distance
decode, NMS, the closed-form (complex-number) similarity transform,
bilinear affine warp, NCHW normalization, L2-norm, Laplacian sharpness.
11 unit tests cover the error-prone math with no model needed.
- onnx_face_analyzer.rs (feature `faces-onnx`): wires the geometry to ONNX
Runtime via `ort` (load-dynamic, so libonnxruntime is dlopen'd at runtime
and the crate builds without it). Inference runs on spawn_blocking; each
session is serialized behind a Mutex. Loads via `ort::init_from` (fallible)
not ORT's lazy loader, which would panic under `panic = "abort"`.
- config: FacesConfig + OXICLOUD_FACES_{ORT_DYLIB,DETECTOR_MODEL,
EMBEDDER_MODEL,DET_SIZE,DET_THRESHOLD,NMS_THRESHOLD,INTRA_THREADS}.
- di: build_face_analyzer() loads the real analyzer when the feature is
compiled in and runtime+models are configured; any missing piece or load
failure degrades to the no-op analyzer (logged) so startup never fails.
- ort/ndarray added as optional deps; example.env documents the setup.
Models and the ONNX Runtime dylib are operator-provided at runtime and are
never committed. Cannot be exercised in CI (no models/dylib); the geometry
is unit-tested and the ONNX seam is isolated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JW6ghFMDtnRYuYNzZhb47M
This commit is contained in:
+54
-4
@@ -814,15 +814,16 @@ impl AppServiceFactory {
|
||||
service
|
||||
}
|
||||
|
||||
/// Creates the face-indexing lifecycle hook (People feature). Uses the
|
||||
/// default no-op analyzer until the operator wires a real ONNX model.
|
||||
/// Creates the face-indexing lifecycle hook (People feature). Picks the
|
||||
/// real ONNX analyzer when the `faces-onnx` feature is compiled in and the
|
||||
/// operator has configured the runtime + models; otherwise the inert no-op
|
||||
/// analyzer (see [`Self::build_face_analyzer`]).
|
||||
pub fn create_face_indexing_service(
|
||||
&self,
|
||||
db_pool: &Arc<PgPool>,
|
||||
) -> Arc<crate::infrastructure::services::face_indexing_service::FaceIndexingService> {
|
||||
let blob_root = self.storage_path.join(".blobs");
|
||||
let analyzer: Arc<dyn crate::application::ports::face_ports::FaceAnalyzerPort> =
|
||||
Arc::new(crate::infrastructure::services::noop_face_analyzer::NoopFaceAnalyzer);
|
||||
let analyzer = self.build_face_analyzer();
|
||||
Arc::new(
|
||||
crate::infrastructure::services::face_indexing_service::FaceIndexingService::new(
|
||||
db_pool.clone(),
|
||||
@@ -832,6 +833,55 @@ impl AppServiceFactory {
|
||||
)
|
||||
}
|
||||
|
||||
/// Selects the face analyzer. With the `faces-onnx` feature and a fully
|
||||
/// configured runtime + models, loads the real ONNX analyzer; any missing
|
||||
/// piece or load failure degrades gracefully to the no-op analyzer (logged)
|
||||
/// so startup never fails on biometric configuration.
|
||||
fn build_face_analyzer(
|
||||
&self,
|
||||
) -> Arc<dyn crate::application::ports::face_ports::FaceAnalyzerPort> {
|
||||
#[cfg(feature = "faces-onnx")]
|
||||
{
|
||||
let f = &self.config.faces;
|
||||
if let (Some(dylib), Some(detector), Some(embedder)) = (
|
||||
f.ort_dylib.as_ref(),
|
||||
f.detector_model.as_ref(),
|
||||
f.embedder_model.as_ref(),
|
||||
) {
|
||||
use crate::infrastructure::services::onnx_face_analyzer::{
|
||||
OnnxFaceAnalyzer, OnnxLoadConfig,
|
||||
};
|
||||
let cfg = OnnxLoadConfig {
|
||||
dylib,
|
||||
detector,
|
||||
embedder,
|
||||
det_size: f.det_size,
|
||||
det_threshold: f.det_threshold,
|
||||
nms_threshold: f.nms_threshold,
|
||||
intra_threads: f.intra_threads,
|
||||
};
|
||||
match OnnxFaceAnalyzer::load(&cfg) {
|
||||
Ok(analyzer) => {
|
||||
tracing::info!("Face analyzer: ONNX models loaded");
|
||||
return Arc::new(analyzer);
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(
|
||||
"Face analyzer: failed to load ONNX models ({e}); \
|
||||
falling back to no-op analyzer"
|
||||
);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
tracing::info!(
|
||||
"Face analyzer: faces-onnx compiled but runtime/models not fully \
|
||||
configured; using no-op analyzer"
|
||||
);
|
||||
}
|
||||
}
|
||||
Arc::new(crate::infrastructure::services::noop_face_analyzer::NoopFaceAnalyzer)
|
||||
}
|
||||
|
||||
/// Creates the People (faces) read/clustering service.
|
||||
pub fn create_people_service(&self, db_pool: &Arc<PgPool>) -> Arc<PeopleService> {
|
||||
let repo = Arc::new(
|
||||
|
||||
Reference in New Issue
Block a user