SCRFD Face Detection & ArcFace Embeddings

Deep Learning Facial Pipeline

BestShots provides automated facial clustering tailored specifically for multi-person events like weddings, galas, and graduations. Rather than transmitting unencrypted face crops to external third parties, facial intelligence runs inside our isolated Azure compute boundary using ONNX Runtime.

---

SCRFD Landmark Detection

Face detection is executed using Sample and Computation Redistribution for Efficient Face Detection (SCRFD):

1. 640×640 Letterboxed Normalization: Preserves native aspect ratio across portrait and landscape orientations.
2. Multi-Stride Decoders: Evaluates anchors at strides 8, 16, and 32 pixels, enabling detection of tiny background subjects as well as tight close-up portraits.
3. 5-Point Landmark Canonical Alignment: Detects left eye, right eye, nose tip, left mouth corner, and right mouth corner. Applies 2D affine least-squares transformations to align faces to a canonical orientation.

---

ArcFace 512-Dimensional Embeddings

Aligned face crops are processed through the ArcFace deep convolutional network to generate a 512-dimensional normalized feature vector:

• Metric Learning: ArcFace maximizes intra-class compactness and inter-class discrepancy on an angular hypersphere.
• pgvector Storage: Vectors are persisted in Neon PostgreSQL using native vector(512) columns with indexing.
• Account Isolation: Facial vectors belong exclusively to the project row in photo_faces. They are never pooled across user accounts and are never matched against external government or commercial databases.