A list,
1. Overview of template matching
When we compare two images, the first basic question we face is: when are two images the same or relatively similar, and how similar are they to be measured? Of course, the general method is that when the gray values of all the pixels of two images are the same, we consider the two images to be the same. This comparison method is feasible in some specific applications, such as detecting changes in two consecutive frames of images under constant lighting and camera internal environments. Simply comparing the differences between pixels is not appropriate in most applications. Noise, quantization errors, small light changes, and minimal translation or rotation will produce large pixel differences when the two images are simply different, but the two images will still look the same to the human eye. Obviously, human perception can sense a wider range of similar content, and can use such things as structure and image content to identify the similarity of two images even when the difference of single pixels is large. Comparing images at the level of structure or semantics is not only a difficult problem but also an interesting research field.
Here we introduce a relatively simple method of comparing images at the pixel level. That is to locate a given sub-image in a larger image —– template image, which is commonly referred to as template matching. This happens a lot, such as locating a specific object in an image scene, or tracking down the most specific patterns in a sequence of images. The basic principle of template matching is simple: in the image to be searched, the template image is moved, and the difference between the sub-image of the image to be searched and the template image is measured at each position. When the similarity reaches the maximum, the corresponding position is recorded. But the actual situation is not so simple, how to choose the appropriate distance measurement method? What happens when the brightness or contrast changes? What is the total distance difference in the match to be considered high similarity? These questions need to be considered on a practical basis.
Are the above images “the same”? In the original image (a) and the other five images (B-F), a simple comparison of the difference between the image pixels will result in a large distance value.
Basic principles of template matching. With the origin of the two images as the reference point, the reference image R is translated by (R,s) units in the image I to be searched, and the size of the image to be searched and the size of the template image determine the maximum search area.
2. Template matching in grayscale image
Template matching in grayscale image is mainly to find the same or the most similar position between the template image R and the sub-images in the search image I. The following formula represents the offset (r,s) of the template in the original image,
The reference image R is translated by R and S units in the horizontal and vertical directions respectively. Then the problem of template matching can be summarized as follows:
Given the search image I and the template image R. Find the translated reference image and the position with the largest similarity of the corresponding sub-image in the search image.
2.1 Distance function in image matching
The most important thing in template matching is to find a similarity measurement function with good robustness to gray scale and contrast changes.
In order to measure the degree of similarity between images, we calculate the “distance” D (r,s) between the reference image after each translation (r,s) and the corresponding sub-image in the search image (as shown in the figure below). There are several basic measurement functions in two-dimensional gray image as follows:
Representation of distance measurement functions in two-dimensional images
Sum ofabsolute differences:
Maximumdifference:
Sum ofsquared differences:
Because of the characteristics of the SSD function, it is often used in the field of statistics and optimization. In order to find the best matching position of the reference image in the search image, it is necessary to minimize the SSD function. namely
In the above formula, B is the sum of squares of gray values of all pixels in the reference image, and it is a constant (independent of r and S), so it can be ignored when calculating its minimum value. A(r,s) represents the sum of squares of gray values of all pixels of the search image in the sub-images of (r,s), and C(r,s) is called the linear cross-correlation function of the search image and the reference image, which can usually be expressed as:
When R and I exceed the boundary, their value is zero, so the above equation can also be expressed as:
If we assume that A(r,s) is A constant in the search image, then its value can be ignored when calculating its best matching position in SSD, and when C(r,s) reaches its maximum value, the reference image is most similar to the sub-images in the search image. In this case, the minimum value of SSD can be obtained by calculating the maximum value of C(r,s).
2.2 Normalized cross-correlation
In practice, the assumption that A(r,s) is A constant is not always true. Therefore, the above cross-correlation results will change greatly with the change of pixel gray value in the search image. Normalized cross-correlation takes into account the energy of the reference image and the current subimage:
When the gray values of both the reference image and the sub-image of the search image are positive, the value of Cn(r,s) is always within the range of [0,1], independent of the gray values of other pixels in the image. When Cn(r,s) is equal to 1, it indicates that at the translational position (r,s), the reference image and the sub-image reach the maximum similarity; Conversely, when Cn(r,s) is equal to 0, it indicates that at the translational position (r,s), the reference image and the subimage do not match at all. When all the gray values in the subimage change, the normalized cross-correlation Cn(r,s) will also change greatly.
2.2 Correlation coefficient
In order to solve the above problem, we introduce the grayscale average of the sub-image and the template image. The above normalized function can be rewritten as:
Average values of sub-images and reference images are respectively defined as:
Where, K represents the number of pixels in the reference image. In statistics, these expressions are known as correlation coefficients. However, instead of being a global measure in statistics, CL (r,s) is defined as a local measure function. The range of CL (r,s) is [-1,1]. When CL(r,s) is equal to 1, it indicates that at the translational position (r,s), the reference image and the subimage reach the maximum similarity; Conversely, when CL(r,s) is equal to 0, it indicates that at the translational position (r,s), the reference image and the subimage do not match at all. Expressions:
Represents the variance of pixel gray value in K times template image R, which is a constant and only needs to be calculated once. Because the variance exists the following relationship :(please refer to imagej.net/Integral_Im…)
The details are as follows:
So we can rewrite SR as
Substituted into CL(r,s), we can get:
Thus, an efficient method for calculating the correlation coefficient is obtained.
Ii. Source code
% clear
addpath(genpath(pwd)); %添加子文件夹下的所有文件
%%加载图像
start=6212;
% state=num+1;
% start=6540;
state=1;
% for num = state:294
close all
num = 1 ; %读取文件夹下的第num张图片
% 20
%
fname=['样本库\IMG_',num2str(start+num),'.jpg'];
% 有三个车牌文件夹：'PlateImages/%d.jpg' 或者'PlateImages/Image1/%d.jpg' 或'PlateImages/Image2/%d.jpg'
filename = fullfile(pwd, fname);
Img = imread(filename);
%{
figure(5);
subplot(2,2,1);imshow(Img);title('原图');I1=rgb2gray(Img);
subplot(2,2,2);imshow(I1);title('灰度图');
subplot(2,2,3);imhist(I1);title('灰度图直方图');I2=edge(I1,'roberts',0.15,'both');
subplot(2,2,4);imshow(I2);title('roberts算子边缘检测')
se=[1;1;1];
I3=imerode(I2,se);
figure(6);
subplot(2,2,1);imshow(I3);title('腐蚀后图像');se=strel('rectangle',[25,25]);I4=imclose(I3,se);
subplot(2,2,2);imshow(I4);title('平滑图像的轮廓');I5=bwareaopen(I4,2000);
subplot(2,2,3);imshow(I5);title('从对象中移除小对象');
%}
GetDB;
%% 定位车牌
% 定位车牌：找出车牌在原始图片中的位置
% 输入参数Img： 读取的原始真彩色图像信息
% 输出参数plate：经过定位处理后，从原始真彩色图像(会进行压缩处理)截取到的车牌位置处的真彩色图像信息
plate = Pre_Process(Img);
%% 倾斜校正
% 倾斜校正：对待识别数字的倾斜校正，分别使用Radon变换和仿射函数处理水平倾斜校正和垂直倾斜校正
% 输入参数：plate为定位截取的真彩色车牌图像信息
% 输出参数：
plate = radon_repair(plate);
%% 车牌滤波
% 车牌滤波：剔除(像素值置0)车牌图片的边界图像信息
% 输入参数：plate为定位截取的真彩色车牌图像信息
% 输出参数：d为原始车牌图像滤波(即剔除上下边界外(和使用多边形区域剔除))后的图片，p为真彩色原始车牌图像plate逆时针旋转后的图片
[d, p] = Plate_Process(plate,fname);
%% 分割车牌
% 分割车牌：裁掉(删除)车牌图像信息的边界
% 输入参数：d为原始车牌图像滤波(即剔除上下边界外(和使用多边形区域剔除))后的图片，p为真彩色原始车牌图像plate逆时针旋转后的图片
% 输出参数：根据图像d的非0边界，裁剪出的输入图片：输入图片d裁剪后输出图片e，输入图片p裁剪后输出图片p
[e, p] = Segmation(d, p);
%% 去除噪声
function [result, plate] = Plate_Process(plate, fname, flag)
% 分割步骤
if nargin < 3
flag = 1;
end
% n = ndims(A) returns the number of dimensions in the array A.
if ndims(plate) == 3
% I = rgb2gray(RGB) converts the truecolor image RGB to the grayscale intensity image I.
% rgb2gray converts RGB images to grayscale by eliminating the hue and saturation information while retaining the luminance.
plate1 = rgb2gray(plate); % 将车牌原始图片信息转换成grayscale intensity image(灰度图像？为方便，下面都称为灰度图像)
else
plate1 = plate;
end
Im = plate1; % Im为灰度图像
plate = double(plate);
% B = mean2(A) computes the mean of the values in A.
% b = std2(A) computes the standard deviation of the values in A.
% 求出当前图片的[均值 标准差]矩阵，用于和数据库的[均值 标准差]矩阵进行计算，然后找出最适合用于处理当前图片的数据库参数信息
m = [mean2(plate(:,:,1)) mean2(plate(:,:,2)) mean2(plate(:,:,3)) std2(plate(:,:,1)) std2(plate(:,:,2)) std2(plate(:,:,3))];
% f = fullfile(filepart1,...,filepartN) builds a full file specification, f, from the folders and file names specified.
% f = fullfile('myfolder','mysubfolder','myfile.m') ===> f = myfolder\mysubfolder\myfile.m
load('model.mat');
ms = cat(1, M.m); % 数据库中的[均值 标准差]矩阵
% B = repmat(A,m,n) creates a large matrix B consisting of an m-by-n tiling of copies of A.
% The size of B is [size(A,1)*m, (size(A,2)*n]. The statement repmat(A,n) creates an n-by-n tiling.
m = repmat(m, size(ms, 1), 1); % 将当前图片的[均值 标准差]矩阵进行拓展，以便进行与数据库[均值 标准差]矩阵运算
% B = sum(A,dim) sums along the dimension of A specified by scalar dim. The dim input is an integer value from 1 to N,
% where N is the number of dimensions in A. Set dim to 1 to compute the sum of each column, 2 to sum rows, etc.
% 按行求和
dis = sum((m - ms).^2, 2); % 当前图片[均值 标准差]矩阵与数据库[均值 标准差]矩阵的方差
[~, id] = min(dis); % 找出方差最小的那个，也就是说，找出最适合用于处理当前图片的数据库参数
if fname(6)=='s'
ro = M(id).ro; % 图片旋转的角度参数，单位为度
else
ro=0;
end
th = M(id).th; % 将灰度图像转换为二进制图像的门限，灰度图像中的值大于该门限则转换为1，否则转换为0
pts = M(id).pts; % 定义图片的顶点
% B = imrotate(A,angle,method) rotates image A by angle degrees in a counterclockwise direction around its
% center point, using the interpolation method specified by method.
Im = imrotate(Im, ro, 'bilinear'); % 将灰度图片Im按照逆时针方向，旋转ro度，插值方法选择双线性插值
plate = imrotate(plate, ro, 'bilinear'); % 将车牌原始图像，也按照逆时针方向，旋转ro度，插值方法选择双线性插值
% BW = im2bw(I, level) converts the grayscale image I to a binary image.
% The output image BW replaces all pixels in the input image with luminance(亮度) greater than level with the value 1 (white)
% and replaces all other pixels with the value 0 (black). Specify level in the range [0,1]. This range is relative to
% the signal levels possible for the image's class. Therefore, a level value of 0.5 is midway between black and white, regardless of class.
bw = im2bw(Im, th); % 将灰度图像Im转换成二进制图像，转换门限为th
% h = fspecial('average', hsize) returns an averaging filter h of size hsize.
% The argument hsize can be a vector specifying the number of rows and columns in h, or it can be a scalar, in which case h is a square matrix.
h = fspecial('average', 2); % 定义一个多维均值滤波器h，维度为2 x 2
% B = imfilter(A,h) filters the multidimensional array A with the multidimensional filter h.
% The array A can be logical or a nonsparse numeric array of any class and dimension. The result B has the same size and class as A.
% ___= imfilter(___,options,...) performs multidimensional filtering according to the specified options.
% 'replicate' ===> Input array values outside the bounds of the array are assumed to equal the nearest array border value.
bw1 = imfilter(bw, h, 'replicate'); % 使用多维均值滤波器h，对二进制图像bw进行滤波，滤波选项为replicate
% mask = Mask_Process(bw1); % 此方法功能是去除bw1图像的上下杂线，即使用特定算法，剔除二进制图像bw1的上下边界处的信息(这些行的像素值全设为0)
% 此处的mask可理解为：如果图像像素点在杂线上边界与车牌上边界之间或杂线下边界与车牌下边界之间，则这个像素点的值为0，
% 否则，如果图像像素点在杂线上边界和杂线下边界之间，则这个像素点的值为1
% bw2 = bw1 .* mask; % 这个.*操作可理解为：与运算，这样bw2就是利用mask矩阵剔除二进制图像bw1中干扰后的结果
bw2 = bw1;
% 除了通过Mask_Process()函数剔除杂线外，如果有定义车牌边界信息，则可进一步使用车牌边界信息进行图像处理
% if ~isempty(pts) % 如果pts不空(即，有外部边界顶点定义)，则执行
% % BW = roipoly(I, c, r) returns the region of interest(ROI) specified by the polygon(多边形) described by vectors c and r,
% % which specify the column and row indices of each vertex(顶点), respectively. c and r must be the same size.
% % 根据二进制图片bw2，以及多边形的顶点pts(:, 1), pts(:, 2)，得到二进制图像mask，顶点围起来的区域的值全为1，
% % 其它区域的值全为0，这样做的目的是剔除车牌边界的干扰
% mask = roipoly(bw2, pts(:, 1), pts(:, 2)); % 得到一个由pts顶点(可组成多边形)指定的二进制边界图像
% bw1 = bw1 .* mask; % bw1是只使用多边形剔除干扰后的二进制图像
% bw2 = bw2 .* mask; % bw2是使用Mask_Process()算法和多边形两种方法剔除干扰后的二进制图像
% end
Copy the code
