Project Code Overview

The Liveness Engine is a software library designed to verify that a user in front of a camera is a live person, thereby preventing spoofing attacks that use photos or videos. It functions as a challenge-response system, periodically instructing the user to perform simple actions such as blinking or turning their head. The engine then analyzes the video feed to confirm these actions were completed successfully.

Key Components

Core Classes

• LivenessEngine: The primary engine class that manages the liveness detection process. It generates challenges for the user and verifies their actions.
• EngineBase: A foundational class providing the common framework for video processing, resource management, and visualization shared across different engines.
• core::Detection: A standard data structure that holds information about a detected face, including its bounding box and detailed facial landmarks.

Action Detection and Analysis

The Liveness Engine heavily relies on gaze and pose estimation components to monitor the user's actions.

• gaze::GazeEstimation: This class acts as a central hub for acquiring all necessary facial state data. It combines the outputs from the head and eye estimators.
• gaze::HeadGazeEstimation: This component is crucial for verifying challenges that involve head movement. It calculates the 3D orientation of the user's head in real-time.
• gaze::EyeGazeEstimation: This component is used to detect blinking. It calculates an "eye openness" metric, allowing the engine to identify the distinct pattern of an eye blink.

Face and Pose Tracking

core::face::FaceDetectorYunet: The first step in the pipeline is to reliably detect a face in the video stream using this robust DNN-based detector.

core::face::FaceMesh: Once a face is found, this class generates a detailed mesh of facial landmarks. These landmarks are the essential input for the head pose and eye state estimation modules.

core::filter::PoseKalmanFilter: To ensure that the head pose tracking is smooth and less susceptible to noise, a Kalman filter is used. This provides a more stable stream of orientation data for verifying head-turning challenges.

Function Highlights

• Engine Initialization: LivenessEngine::init(resourcePath) initializes the engine and all its dependencies, including the face detector and gaze estimation modules.
• Image Processing: LivenessEngine::process(input, output) is the main loop where the engine analyzes the current frame, updates the user's facial state (head pose, eye openness), checks if the state matches the current challenge, and provides visual feedback.
• Gaze and Pose Update: gaze::GazeEstimation::process(...) is called within the main loop to get the latest analysis of the user's head and eye movements.
• Face Detection: FaceDetectorYunet::process(frame) detects the user's face in each frame.
• Landmark Generation: FaceMesh::process(frame, detections) provides the detailed landmark points required by the gaze and eye estimators.

File and Dependency Structure

The project files are organized in the C:/Projects/Engine/AntalEngine/Engine directory. Key header files include:

• LivenessEngine.h: Contains the declaration for the main engine class that manages the liveness test.
• GazeEstimation.h: A critical dependency that provides the facial state information.
• HeadGazeEstimation.h: Handles head pose calculation.
• EyeGazeEstimation.h: Handles eye state and blink detection.
• PoseKalmanFilter.h: Defines the Kalman Filter used for smoothing pose data.
• FaceDetectorYunet.h: Provides the core face detection functionality.