Project Code Overview

The FaceSwap Engine is a software library designed to detect and swap faces within an image. It uses computer vision techniques to identify facial features, allowing for the exchange of faces between individuals. The engine's logic adapts based on the number of faces detected: if two faces are found, they are swapped. If more than two are present, the largest face is copied onto all others.

Key Components

Core Classes

• FaceSwapEngine: The primary class that orchestrates the face-swapping process. It inherits from EngineBase and implements the main processing logic.
• EngineBase: A foundational class that provides the main framework for image processing, including initialization, pre-processing, and post-processing steps.
• core::Detection: A data structure that holds all relevant information about a detected face, such as its bounding box and the precise location of its facial landmarks (shape points).
• core::NeuralBase: Serves as the base class for neural network models used within the engine, primarily for facial landmark detection.

Face Detection and Landmark Mapping

core::face::FaceDetectorYunet: This class is responsible for the initial step of detecting the presence and location of faces in an image, utilizing OpenCV's robust DNN-based model.

core::face::FaceMesh: Once faces are detected, this class generates a detailed mesh of facial landmarks for each one. These landmarks are critical for accurately mapping the geometry of the source face onto the target face.

Swapping and Rendering

visualization::FaceDrawer: This utility class contains the core rendering logic. Its drawFaceSwap method takes the landmarks from a source and target face, calculates the necessary transformations to warp the source face's texture, and seamlessly blends it onto the target face's position and orientation.

Temporal Smoothing for Video

core::filter::LandmarkFlowFilter: To improve stability in video sequences, this class implements a filter to track facial landmarks smoothly across consecutive frames. This helps reduce jitter and ensures the swapped face remains properly aligned during motion.

Function Highlights

• Engine Initialization: FaceSwapEngine::init(resourcePath) sets up the engine by loading all required models and configuration files.
• Image Processing: FaceSwapEngine::process(input, output) executes the full pipeline: it detects all faces, generates their landmark meshes, performs the swap, and writes the result to the output image.
• Face Detection: FaceDetectorYunet::process(frame) returns a list of all faces found in the current frame.
• Landmark Generation: FaceMesh::process(frame, detections) populates the detected face objects with detailed landmark data.
• Face Swapping: FaceDrawer::drawFaceSwap(...) performs the visual swapping by warping the source face's texture based on the target's landmarks and blending it onto the target image.

File and Dependency Structure

The project files are organized in the C:/Projects/Engine/AntalEngine/Engine directory. Key header files include:

• FaceSwapEngine.h: Contains the declaration of the main engine class.
• FaceDrawer.h: Provides the rendering functions, including the core face-swapping logic.
• FaceMesh.h: Defines the class responsible for detailed facial landmark detection.
• FaceDetectorYunet.h: Defines the class for initial face detection.
• LandmarkFlowFilter.h: Declares the filter used for smoothing landmark tracking in videos.
• EngineBase.h: Contains the definition for the base engine class.