Self-hosted face match — the AEBAS-equivalent without UIDAI

This page describes Phase 2.5: an in-house face-matching pipeline that gives parliament-grade person-binding without depending on UIDAI Auth licensing or the AUA/ASA contract chain. It's complementary to (not a replacement for) the UIDAI path. Phase 2.5 is shipped — backend services, schema, admin enrollment UI, mobile ML Kit + TFLite integration, and decision-matrix config are all live. Pair it with DigiLocker eKYC at first enrollment for verified onboarding.

Why an alternative is attractive

The honest tradeoff: self-hosted shifts the privacy burden onto us. We become custodians of facial templates for ~2,300 staff plus members. That's a real DPDP Act §5 obligation — purpose limitation, retention limits, breach notification, grievance officer. The crypto and storage we already have for vanisetu's audio (Vault transit + AES-GCM) maps directly onto this — we don't need new infra, just a new schema + service.

The four-step model

Step 1 — Enrollment ceremony

This is where the binding happens. A bad enrollment defeats the whole system.

1. HR-mediated: an authorised officer (Establishment Section) verifies the employee's government ID — Aadhaar physically (not authenticated, just visually checked against the person), Parichay smart-card, or service ID.
2. Consent: explicit, signed (digital signature on the device, or paper countersigned), versioned consent text. Stored in attendance_face_consent.
3. Capture: 3 frames at slight angle variation (frontal, ±15° head turn). Reduces single-frame artefact bias.
4. Embedding: run FaceNet (or MobileFaceNet) inline at enrollment — 512-dimensional float vector, ~2 KB per template. Average across the 3 frames for robustness.
5. Encrypted storage: AES-256-GCM with a per-template DEK, KEK in Vault transit. Same envelope pattern as vanisetu's audio storage.

The captured raw frames are discarded after embedding extraction. We never persist face images — only the irreversible 512-dim embedding. That's the data-minimisation principle baked into the schema.

Step 2 — Live capture + on-device liveness

Liveness defeats photo-replay attacks. We use a layered passive + active approach:

• Passive (always): ML Kit Face Detection emits face landmark + head pose + smile/blink probability. We sample 8 frames across 1.5 s and require natural micro-motion (head pose variance > threshold, blink probability transition).
• Active (every Nth punch, randomised): prompt the user with a random challenge — "blink twice", "turn left", "say a 4-digit code". Defeats prepared video replays.
• Texture cue: ML Kit doesn't expose this directly. For higher assurance scenarios, use a paid SDK (FaceTec, Innovatrics) that adds 3D depth from monocular video.

Liveness output is a liveness_score ∈ [0, 1]. Threshold 0.7 default, configurable per cohort (members may need lower threshold given older devices).

Step 3 — Embedding + matching

Two architecture choices:

We ship 2.5b (server-side match) as the default for parliament-grade. The phone uploads the embedding (not the frame); backend looks up the encrypted template, decrypts in-memory, computes cosine similarity, returns yes/no. Embeddings are too low-resolution to reconstruct a face, so transmitting the embedding leaks less than transmitting the image.

Step 4 — Result + audit

Same pattern as Phase 1. The punch row carries:

• verdict: accepted_face_match on success.
• identity_assurance: l3 — same tier as UIDAI Aadhaar match.
• face_match_score: cosine similarity (0–1) for forensic correlation.
• liveness_score: 0–1 — useful when investigating disputed punches.
• face_template_version: lets us re-match against a newer template after employee re-enrolls.

Audit event: attendance.punch.face_match.{accepted|rejected} with the score (rounded to 2 decimals — full precision could leak template entropy across many forced-fail attempts).

Schema additions

// Phase 2.5 — face templates + consent
CREATE TABLE attendance_face_consent (
  id              bigserial PRIMARY KEY,
  employee_id     bigint NOT NULL REFERENCES attendance_employee_register(id) ON DELETE CASCADE,
  consent_version varchar(16) NOT NULL,
  consent_text_url varchar(255) NOT NULL,
  consented_at    timestamptz NOT NULL,
  withdrawn_at    timestamptz,
  enroller_user_id bigint REFERENCES users(id),    -- HR officer who supervised
  ip_address      inet,
  signature_blob  bytea,                            -- digital signature of consent text
  UNIQUE(employee_id, consent_version, withdrawn_at)
);

CREATE TABLE attendance_face_templates (
  id                bigserial PRIMARY KEY,
  employee_id       bigint NOT NULL REFERENCES attendance_employee_register(id) ON DELETE CASCADE,
  version           int NOT NULL DEFAULT 1,         -- bump on re-enrollment
  model             varchar(32) NOT NULL,           -- mobilefacenet | facenet512 | ...
  encrypted_blob    bytea NOT NULL,                 -- AES-256-GCM(embedding, DEK)
  blob_sha256       varchar(64) NOT NULL,
  dek_vault_path    varchar(255) NOT NULL,          -- Vault transit key reference
  enrolled_at       timestamptz DEFAULT now(),
  enrolled_by_user  bigint REFERENCES users(id),
  retired_at        timestamptz,                    -- on re-enroll, old version retires
  UNIQUE(employee_id, version)
);

ALTER TABLE attendance_punches
  ADD COLUMN face_match_score      decimal(4,3),   -- cosine similarity 0..1 (4 sig figs)
  ADD COLUMN liveness_score        decimal(4,3),
  ADD COLUMN face_template_version int;

-- Verdict column already widened to varchar(32) — accommodates new values:
-- 'accepted_face_match' (19), 'rejected_face_mismatch' (22), 'rejected_liveness' (17)

Service layout

app/Services/
├── FaceTemplateStore.php       # encrypt + store, load + decrypt; never logs
├── FaceMatcher.php             # cosine similarity over 512-d float vectors
├── LivenessGate.php            # validates liveness_score against per-cohort threshold
└── EnrollmentCeremony.php      # the HR-mediated flow: capture + embed + persist + consent

app/Http/Controllers/Web/
└── EnrollmentController.php    # admin UI for HR to onboard one employee at a time

app/Http/Controllers/Api/V1/
├── FaceMatchController.php     # POST /v1/punch/face-match
└── EnrollmentApiController.php # POST /v1/enrollment/face-template (HR officer mode)

The hybrid model — recommended deployment

Mix-and-match by trust requirement, not by employee:

The decision matrix is config-driven via config/attendance.php — operations can dial trust requirements per cohort and per geofence without code changes.

Privacy + compliance posture

• Data fiduciary: Rajya Sabha Secretariat is the data fiduciary under DPDP Act 2023 §2(i). Privacy policy, grievance officer, breach notification within 72 h.
• Data principal rights: employee can withdraw consent → triggers template retirement; can request access to "what's stored about me" — we expose this on the employee profile screen.
• Purpose limitation: face template is used only for attendance punch verification. Hard-coded; no API accepts the embedding for any other purpose.
• Retention: template retired on employee separation + 6 months audit window, then hard-deleted (the row, not just deleted_at — biometric data has stricter retention).
• No raw image storage: enrollment frames discarded post-embedding. Punch frames never uploaded; only the embedding + liveness score.
• Encryption: AES-256-GCM with envelope encryption — DEK in Vault transit, ciphertext in Postgres. Compromise of either alone is non-disclosing.
• Audit: every read of a face template emits attendance.face_template.read in the audit chain. Sustained read activity by an admin account = obvious anomaly signal.

What this is NOT

• Not Aadhaar authentication. The match is against our enrolled template, not UIDAI's database. We rely on HR to do the enrollment ceremony correctly.
• Not 1:N (identification). We never search "who is this face among 2,300 employees" — that's a different problem and weaker accuracy. We only do 1:1 against the claimed employee.
• Not unspoofable. Sufficiently sophisticated 3D-mask attacks beat MobileFaceNet + ML Kit liveness. For threat-modelled high-stakes events, layer with UIDAI or with a paid SDK that does monocular depth.

Implementation effort

Comparison to NIC's AEBAS

Net: with Phase 1 (today) + Phase 2.5 (12 days of work), Sanchalan reaches AEBAS-equivalent person-binding with strictly more deployment flexibility (BYOD + offline + cohort-tunable trust) and without UIDAI runtime dependency. Adding UIDAI later as Phase 3 is then defense-in-depth — both layers run for high-stakes events.