Test & measure accuracy of face recognition (identity verification)

[MeenakshiArunsankar]

Description

Establish ground-truth accuracy metrics for the face recognition system used in proctoring identity verification.

The frontend uses @vladmandic/face-api (TinyFaceDetector + FaceRecognitionNet) to compare captured face embeddings against a reference enrollment photo. Identity mismatch is flagged when Euclidean distance exceeds MATCH_THRESHOLD = 0.55 (hardcoded). However, no accuracy evaluation exists — the similarity scores are not persisted, and no test set validates performance across varied conditions (lighting, angle, glasses, etc.).

Current Implementation

• Detection + Recognition: frontend/src/components/ai/FaceRecognitionComponent.tsx uses @vladmandic/face-api with TinyFaceDetector, FaceLandmark68, and FaceRecognitionNet
• Threshold: Line 16 — MATCH_THRESHOLD = 0.55 (Euclidean distance in 128-dim embedding space)
• Anomaly mapping: Flags AnomalyType.FACE_RECOGNITION when distance > 0.55 (different person)
• Backend logging: backend/src/modules/anomalies/controllers/AnomalyController.ts stores captured image evidence but no similarity scores

Work Required

1. Build a labeled test set (~100–200 image pairs per person, 3–5 enrollees) covering:

   • Same person, same conditions (baseline — should match)
   • Same person, varied conditions (different lighting, angle, expression, glasses/mask, time delta)
   • Different person (impostor attempts — should not match)
   • Hard negatives (similar face features, different person)

2. Persist similarity scores — modify the recognition component to log Euclidean distance alongside each recognition attempt (both matched and flagged anomalies)

3. Run offline evaluation against the test set:

   • Compute True Positive Rate (genuine matches correctly accepted) and False Negative Rate (genuine rejected)
   • Compute False Positive Rate (impostor accepted) and True Negative Rate (impostor correctly rejected)
   • ROC curve (sweep threshold 0.3–0.8) to find operating point
   • Break down performance by condition (time delta, lighting, occlusion, etc.)

4. Document results in a test report with:

   • Equal Error Rate (EER) and optimal threshold recommendation
   • False Accept Rate (FAR) and False Reject Rate (FRR) at current (0.55) and recommended thresholds
   • Accuracy by condition (same-day vs. different-day, lighting, etc.)
   • Security assessment: is EER acceptable for exam proctoring? (typical target: EER < 2–5%)
   • Comparison to industry baselines (if available)

Acceptance Criteria

• Labeled ground-truth face pair test set (CSV: pair_id, ground_truth_match, person_id, condition_tags, time_delta_days)
• Similarity scores (Euclidean distance) logged for all recognition attempts
• Offline eval script (Python/Node) computing FAR/FRR/EER/ROC at multiple thresholds
• Test report with performance broken down by condition and threshold
• Documented decision on optimal threshold + security adequacy (with EER evidence)
• Consider: should enrollment use multiple reference images (different angles, lighting) to improve robustness?

Related

• Frontend: frontend/src/components/ai/FaceRecognitionComponent.tsx
• Backend: backend/src/modules/anomalies/controllers/AnomalyController.ts
• Model: @vladmandic/face-api (face-api.js)
• Embedding space: 128-dimensional FaceRecognitionNet output