Project Code Overview

The Heart Rate Engine is a software library designed to estimate a person's heart rate in real-time from a standard video stream of their face. This is achieved using a non-invasive technique that analyzes subtle color changes in the skin caused by blood flow (remote photoplethysmography - rPPG). The engine detects a face, tracks a region of interest, and processes the color signal from this region to calculate the beats per minute (BPM).

Key Components

Core Classes

• HeartRateEngine: The main engine class that orchestrates the entire process, from face detection to displaying the final heart rate estimation.
• PulseSensor: A high-level class that manages the pulse estimation process, capable of handling multiple users by creating and updating individual Sensor instances.
• Sensor: This is the core algorithmic class that estimates the heart rate from a video feed. It processes the average color values from a facial region over time to build a signal, which is then analyzed to find the dominant frequency corresponding to the pulse.
• EngineBase: A foundational class providing the common framework for video processing, resource management, and visualization.

Face and Landmark Detection

core::face::FaceDetectorYunet: The engine first uses this class to robustly detect and locate faces in the input video stream.

core::face::FaceMesh: After detection, this class generates a detailed mesh of facial landmarks. These landmarks are crucial for identifying and tracking a stable Region of Interest (ROI) on the face (e.g., the forehead), which is essential for accurate signal extraction.

Signal Processing and Smoothing

core::filter::LandmarkFlowFilter: To ensure the ROI remains stable even with minor head movements, this filter is used to track the facial landmarks smoothly across consecutive frames, preventing signal noise caused by jitter.

Function Highlights

• Engine Initialization: HeartRateEngine::init(resourcePath) initializes the engine and all its sub-components, including the face detector and pulse sensor.
• Image Processing: HeartRateEngine::process(input, output) runs the full pipeline on each frame, performing face detection, updating the pulse sensor, and visualizing the results.
• Pulse Update: PulseSensor::update(frame, userId, landmarks, bpm) is the key function for getting a new heart rate reading. It takes the latest frame and landmarks for a user and updates the internal Sensor, returning the calculated BPM.
• Core Signal Analysis: Sensor::process(frame, mask, bpm) performs the low-level color analysis on a masked facial region to extract the raw pulse signal and compute the heart rate.
• Face Detection: FaceDetectorYunet::process(frame) finds all faces present in the video frame.
• Landmark Generation: FaceMesh::process(frame, detections) provides the detailed facial landmark points for each detected face.

File and Dependency Structure

The project files are organized in the C:/Projects/Engine/AntalEngine/Engine directory. Key header files include:

• HeartRateEngine.h: Contains the declaration for the main engine class.
• PulseSensor.h: Declares the class for managing pulse estimation for multiple users.
• Sensor.h: Defines the core class for heart rate estimation via color analysis.
• FaceDetectorYunet.h: Provides the face detection functionality.
• FaceMesh.h: Provides the facial landmark detection functionality.
• EngineBase.h: Contains the definition for the base engine class that other engines inherit from.