Author | Jaime Duranx compile | Flin source | medium

I am currently working on a computer vision problem involving face classification. This usually means applying deep learning, so a special pre-processing phase is required before the image is injected into our neural network.

In order to improve the accuracy of our model, this is a very important task that can be easily accomplished by following a few simple steps. For this article, we can use OpenCV: a highly optimized open source library for computer vision, available in C++, Java, and Python.

This is a short article that contains some basic guidelines, examples, and code that you may need to apply to each facial classification or recognition problem.

Note: All still images used in this article are from imgflip.com/memetemplat…

Images load

We will load the image using the imread() function, specifying the path and mode of the file. The second parameter is important for running base channels and depth conversions dynamically.

  • Mode:docs.opencv.org/4.1.0/d4/da…
img = cv2.imread('path/image.jpg', cv2.IMREAD_COLOR)
Copy the code

To view the image, we have the imshow() function:

cv2.imshow(img)
Copy the code

If you write type (img), you’ll see dimensions (height, weight, channels).

Our color image has three channels: blue, green, and red (in that order in OpenCV).

We can easily view a single channel:

# Example for green channel
img[:, :, 0]; img[:, :, 2]; cv2.imshow(img)
Copy the code

Grayscale version

It’s a good idea (or not!) to use black and white images in order to avoid distractions in face image classification. You can try both). To get the grayscale version, we just need to specify in the image loading function, passing the appropriate value as the second argument:

img = cv2.imread('path/image.jpg', cv2.IMREAD_GRAYSCALE)
Copy the code

Now our image has a separate channel!

Face and eye testing

When dealing with face categorization, we may want to do face testing to verify (face?) Cropped and straightened our images. We will use the haar-based cascade classifier included in OpenCV for object detection. (docs.opencv.org/2.4/modules…

First, we select a pre-trained face and eye detection classifier. There is a list of available XML files that we can use:

  • Github.com/opencv/open…

1) For face detection, OpenCV provides the following (from the most loose prior to the most strict prior):

  • haarcascade_frontalface_default.xml

  • haarcascade_frontalface_alt.xml

  • haarcascade_frontalface_alt2.xml

  • haarcascade_frontalface_alt_tree.xml

2) For eye detection, we can choose two methods:

  • haarcascade_eye.xml

  • haarcascade_eye_tree_eyegasses.xml

We load the pre-trained classifier in this way:

Face_cascade = cv2.cascadeclassifier (cv2.data.haarcascades + 'haarCascade_frontalface_default.xml') eyes_CASCADE = cv2.cascadeclassifier (cv2.data.haarcascades + 'HaarCascade_frontalface_default.xml') eyes_CASCADE = Cv2. CascadeClassifier (cv2. Data. Haarcascades + 'haarcascade_eye. XML')Copy the code

You can test several combinations. Remember that none of them is optimal in all cases (if the first classifier fails, you can try a second classifier, or even try all of them).

For face detection, we use the following code:

faces_detected = face_cascade.detectMultiScale(img, scaleFactor=1.1, minNeighbors=5)
Copy the code

The result is an array containing all the detected faces. We can easily draw rectangles:

(x, y, w, h) = faces_detected[0]
cv2.rectangle(img, (x, y), (x+w, y+h), (0.255.0), 1);
cv2.imshow(img)
Copy the code

For eyes, we searched in a similar way, but narrowed the search down to the face rectangle:

eyes = eyes_cascade.detectMultiScale(img[y:y+h, x:x+w])
for (ex, ey, ew, eh) in eyes:
    cv2.rectangle(img, (x+ex, y+ey), (x+ex+ew, y+ey+eh), 
                  (255.255.255), 1)
Copy the code

Into!

While this is the expected result, we have a lot of other problems. Most of the time, we don’t have a positive and clear human face, or even…

Without eyes:

The eye is a black stain surrounded by white:

There are 4 eyes here, and only 3 eyes were detected:

Straight face

By calculating the Angle between the eyes, we can straighten the image of the face (which is easy). After calculation, we only need two steps to rotate the image:

rows, cols = img.shape[:2]
M = cv2.getRotationMatrix2D((cols/2, rows/2), <angle>, 1)
img_rotated = cv2.warpAffine(face_orig, M, (cols,rows))
Copy the code

Cut out the face

In order to help our neural network with the task of classifying faces, it is best to remove external distractions such as background, clothing or accessories. In this case, cutting out the face is a good option.

The first thing we need to do is get the face rectangle again from the straightened image. Then we need to make a decision: we can crop the rectangular area as is, or add an extra fill so we can get more space.

It depends on the specific problem to be addressed (broken down by age, sex, race, etc.); Maybe you want more hair; Probably not.

Finally, clipping (p for fill) :

cv2.imwrite('crop.jpg', img_rotated[y-p+1:y+h+p, x-p+1:x+w+p])
Copy the code

Look! The face is isolated, almost ready for deep learning

Image zooming

Neural networks require all input images to be of the same shape and size, because the GPU applies the same instructions to a batch of images at the same time to achieve super-fast speeds. We can resize them dynamically, but this might not be a good idea, as multiple conversions will be performed on each file during training.

Therefore, if our data set has many images, we should consider implementing a batch resizing process prior to the training phase.

In OpenCV, we can scale down and up using the resize() function, and there are several interpolation methods available. Example of specifying the final size:

cv2.resize(img, (<width>, <height>), interpolation=cv2.INTER_LINEAR)
Copy the code

To shrink the image, OpenCV recommends INTER_AREA interpolation, while to magnify the image, you can use INTER_CUBIC (slow) or INTER_LINEAR (fast, still good).

Finally, there is the tradeoff between quality and time.

I did a quick upgrade comparison:

The first two images seem to be of higher quality (but you can observe some compressed artifacts).

The linear method results in significantly smoother and less noisy results.

The last one is pixelated.

The normalized

We can use the normalize() function to apply visual normalization to fix very dark/light images (and even low contrast).

The normalized type (docs.opencv.org/3.4/d2/de8/… Specify in function arguments:

norm_img = np.zeros((300.300))
norm_img = cv2.normalize(img, norm_img, 0.255, cv2.NORM_MINMAX)
Copy the code

Example:

This normalization is not required when images are used as inputs to deep convolutional neural networks.

In practice, we will normalize each channel appropriately, such as subtracting the mean and then dividing by the standard deviation at the pixel level (so we get mean 0 and deviation 1). If we use transfer learning, the best approach is always to use pre-trained model statistics.

conclusion

When dealing with face classification/recognition problems, it is a common task to detect and isolate faces if the input image is not a passport image.

OpenCV is a great library for image preprocessing, but not just that. It is also a powerful tool for many computer vision applications…

Look at the documentation!

  • docs.opencv.org/master/

Hope you enjoyed this article!

Original link: medium.com/yottabytes/…

Welcome to panchuangai blog: panchuang.net/

Sklearn123.com/

Welcome to docs.panchuang.net/