Face recognition algorithm is the basic component of any face detection and recognition system or software. Experts divide these algorithms into two core methods: the geometric method focuses on distinguishing features, and the photometric statistical method is used to extract values from images. These values are then compared with the template to eliminate differences. These algorithms can also be divided into two more general categories — feature-based models and global models. The former focuses on facial markers and analyzes their spatial parameters and correlations with other features, while the holistic approach treats faces as a whole.

Artificial neural network is the most popular and successful method in image recognition. Face recognition algorithm is based on mathematical calculation, neural network simultaneously performs a large number of mathematical operations.

These algorithms perform three main tasks: detect faces in images, videos, or live streams; Mathematical models for calculating human faces; Compare the model to a training set or database to identify or verify a person.

This article covers the most well-known face recognition algorithms and key features. Because each method has its task-specific advantages, researchers are actively trying to combine methods and develop new technologies.

Convolutional Neural Network (CNN)

Convolutional Neural network (CNN) is one of the breakthroughs in artificial neural network (ANN) and artificial intelligence development. It is one of the most popular algorithms in deep learning, a type of machine learning in which model learning performs categorizing tasks directly on images, video, text or sound. The model shows impressive results in several areas: computer vision, natural language processing (NLP), and the largest Image classification data set (Image Net). CNN is a common neural network with new layers — convolution layer and pooling layer. CNN can have dozens and hundreds of such layers, each learning to detect different imaging features.

Characteristics of the face

Eigenfaces is a face detection and recognition method used to determine face variance in an image dataset. It uses these differences to encode and decode faces through machine learning. A set of feature faces is a collection of “standardized face components” determined by statistical analysis of a large number of face images. Facial features are assigned mathematical values because this method does not use digital images, but a statistical database. Any human face is a combination of these values in different percentages.

Fisherfaces

Fisherfaces is one of the most popular facial recognition algorithms; It is considered superior to many of its alternatives. As an improvement of the Eeigenfaces algorithm, it is often compared to Eigenfaces and is considered to be more successful at category discrimination during training. The main advantage of the algorithm is its ability to interpolate and extrapolate light and facial expression changes. The Fisherfaces algorithm was reported to be 93% accurate when combined with PCA in the pretreatment stage.

Kernel methods: PCA and SVM

Principal component analysis (PCA) is a general statistical method with many practical applications. When used in face recognition, PCA aims to reduce the size of the source data while retaining the most relevant information. It generates a weighted set of feature vectors that in turn build feature faces — a large number of different images of human faces. The linear combination of feature faces represents each image in the training set. PCA is used to receive these feature vectors from the covariance matrix of the training image set. For each image, calculate its main components (from 5 to 200). Other components encode subtle differences between faces and noise. The recognition process involves comparing the main components of the unknown image with those of all other images.

A support vector machine (SVM) is a machine learning algorithm that uses two sets of classification principles to distinguish between faces and “non-faces”. For each category, the SVM model receives a labeled training data set to classify the new test data. The researchers applied linear and nonlinear SVM training models to face recognition. Recent results show that nonlinear training machines have greater margin and better recognition and classification results.

Haar Cascade

Haar Cascade is an object detection method for locating objects on an image. The algorithm learns from a large number of positive and negative samples — the former containing objects of interest, and the latter containing anything other than the object you are looking for. After training, the classifier can find the object of interest on the new image. This method combines with local binary pattern algorithm for face recognition and is used in criminal identification. The Haar cascade classifier uses 200 features (out of 6,000) to ensure an 85-95% recognition rate even if the expression changes.

The three dimensional recognition

The basic idea behind 3D face recognition is the unique structure of the human skull. Each person’s skull structure is unique and can be described by dozens of parameters. The facial recognition method is based on comparing 3D facial scans with database patterns. It has an important advantage — makeup, facial hair, glasses and similar factors do not interfere with the detection and recognition process. The latest research uses a technique to map 3D geometric information onto a regular 2D grid. It allows the descriptive nature of 3D data to be combined with the computational efficiency of 2D data and displays the highest performance of FRGC V2 (Face Recognition Grand Challenges 3D Face Database) reports.

Skin texture analysis

Skin recognition technology has many applications — face detection algorithms, bad image filtering, gesture analysis, etc. It usually uses high-resolution images. Skin texture analysis for special cases uses different unique parameters, such as nevus, skin tone, skin tone, etc. Recent studies based on the combination of texture features and skin color have shown interesting results. The researchers used neural networks to develop and test skin recognition systems. The feedforward neural network used in the project classifies the input texture images into “skin” and “non-skin” and shows impressive performance.

Thermal imager

Thermal imager is a device used to monitor the temperature distribution of the inspected surface. The temperature distribution is shown as a color picture of different colors corresponding to the temperature. The technology already has several practical applications for adapting to global change — smartphone-based immunization certificates, remote fever detection and hot face recognition. The thermal imager face recognition model is based on the unique temperature pattern of face. The consistent human temperature “signature” is measured using the imaginary part of thermal infrared (IR). There is an undeniable benefit to using thermal methods in face recognition – makeup, beards, hats and glasses do not affect their accuracy. In addition, it can distinguish between twin brothers and sisters.

ANFIS

Adaptive neural fuzzy interference system (ANFIS) is an artificial neural network. The method combines the principles of neural network and fuzzy logic and combines their advantages in a single structure. ANFIS is used to classify the image features extracted from the dataset during the pre-processing stage. Data scientists combine this approach with various feature extraction algorithms. As a result, some studies have reported ANFIS classification accuracy of an incredible 97.1% after feature extraction using two-dimensional principal component analysis.

Local binary mode histogram (LBPH)

The method uses local binary mode (LBP), a simple and effective texture operator in computer vision that marks pixels in an image by setting neighborhood thresholds for each pixel and treating the result as a binary number. In the learning phase, LBPH algorithm creates histograms for each labeled and classified image. Each histogram represents each image in the training set. Thus, the actual recognition process means comparing the histograms of any two images.

FaceNet

FaceNet, a facial recognition system developed by Google researchers in 2015, is based on a benchmark dataset for facial recognition. The available pre-training models and various open source third-party implementations make the system very extensive. FaceNet showed excellent results in research surveys, test performance and accuracy compared to other algorithms developed earlier. FaceNet accurately extracted face embedding, high-quality features for the later training of face recognition system.

NEC

The solution developed by Japanese technology company NEC can identify people with high accuracy while recognizing age changes. The solution uses adaptive region mix matching, a model that focuses on mapping highly similar segments. NEC technology divides the input and registered images into small segments and focuses only on the segments with greater similarity. It allows the system to show higher recognition accuracy, even when the face is covered with a mask or glasses. As its underlying algorithm, the NEC solution uses generalized Learning Vector quantization (GLVQ).

Kuang (FACE++)

The open-view algorithm is based on image detection and fuzzy image search techniques. The technology solution uses MegEngine, the company’s proprietary deep learning framework built on big data. The company’s technology has successfully performed face information extraction, including several key features: face and human detection and tracking, face recognition and clustering, key point detection, face attribute estimation and face search engine.

Facial recognition: Combining different technologies

Each facial recognition technology has its own characteristics of effectiveness. However, recent research has proved that the best results are achieved through a combination of different algorithms and methods. These combinations aim to solve many of the usual problems in facial recognition — differences in facial expression, posture, lighting conditions, image noise, and so on. The latest experiments combine LBP algorithm with advanced image processing techniques: bilateral filtering, histogram equalization, contrast adjustment, and image blending. Such a technique shows a significant improvement over the LBP code and looks very promising for further study.

conclusion

There are a variety of face recognition algorithms and methods. Although they all have a primary goal, they can be task-specific and problem-specific. They range from neural networks and mathematical models to technical solutions from private companies, depending on their purpose and implementation.

This article covers the most extensive algorithms and methods. However, more studies and scientific experiments have shown that there are undeniable benefits to combining different algorithms in the face recognition process to get better results. It leads to new technologies and purpose-specific approaches.

Face recognition algorithm FAQ

How do face detection algorithms work?

Typically, specific neural networks are trained to detect landmarks of faces and distinguish them from other objects in the image. Landmarks are universal human facial features such as eyes, nose, mouth, eyebrows, etc. How this is actually implemented varies from algorithm to algorithm.

What is face recognition algorithm?

Face recognition algorithm is a method to build biometric face model for further analysis and face recognition process.

How to do face detection in OpenCV?

Open Source Computer Vision Library (OpenCV) is a popular open source library for computer vision algorithms, image processing, and numerical algorithms. Using OpenCV, face recognition process can be divided into three steps:

• Face detection
• Face embedding is used for feature extraction
• Facial recognition

How to train face recognition algorithms?

Before facial recognition algorithms are ready to perform the necessary tasks, they need to process large amounts of data — sets of precisely labelled images. These collections are used to develop machine learning models.

Develop reading

Face recognition technology is a kind of high precision, easy to use, high stability, difficult to counterfeit biometric technology, has extremely broad market application prospects. In the public security, national defense, customs, transportation, finance, social security, medical and other civil security control industries and departments there is a wide range of demand.

Our R&d staff in TSINGSEE Qingxi Video are also actively developing AI technologies such as face detection, face recognition, traffic statistics and helmet detection, and actively integrating them into existing video platforms. Typical examples are EasyCVR video fusion cloud service, which is capable of AI face recognition, license plate recognition, voice intercom, pso control, sound and light alarm, surveillance video analysis and data summary. It is widely used in intelligent access control of residential areas and buildings, perimeter suspicious wandering detection, scenic spot traffic statistics and other scenes.