To map the system overview

Graph search refers to the search of similar images based on their content.

Two key problems need to be solved in constructing a pick-search system: first, extracting image features; Secondly, feature data search engine, that is, feature data is built into a database and provides the function of similarity search.

Image feature representation

Three ways are introduced.

Image hash

Image through a series of transformation and processing of the final set of hash values known as the image of the hash value, and the middle of the transformation and processing process is known as the hash algorithm.

The hash value of the image is an abstract representation of the whole image.

For example, the calculation process of Average Hash algorithm:

  1. Reduce size: Reduce the original image to 8 x 8, or 64 pixels, ignoring details.
  2. Reduce color: 64 grayscale images are obtained by grayscale processing.
  3. Average the colors: Calculates the grayscale mean of 64 levels.
  4. Compute the bits by comparing each pixel to the previous step’s mean to 0 or 1, respectively.
  5. Construct the Hash: Construct the Hash from the matrix from the previous step to Construct a 64-bit integer, for example, from left to right and from top to bottom. And what you end up with is the average hash of the graph.

Reference: http://www.hackerfactor.com/b…

There are many kinds of image hashing algorithms, including but not limited to:

  • AverageHash: Also called Different Hash
  • PHash : Perceptual Hash
  • MarrHildrethHash : Marr-Hildreth Operator Based Hash
  • RadialVarianceHash : Image hash based on Radon transform
  • BlockMeanHash : Image hash based on block mean
  • ColorMomentHash : Image hash based on color moments

Probably the most common thing we see is PHash.

Image hashing can resist watermarking, compression, noise and other influences to a certain extent, that is, by comparing the Hamming distance of image hashing value, we can judge whether the contents of two images are similar or not.

The hash value of the image is the overall abstract representation of the image, and its limitations are also obvious. Because it is the processing of the image as a whole, once we destroy the integrity, such as adding a black edge to the original image, it is almost impossible to judge the similarity.

Traditional feature

In the early days of computer vision, many classic hand-designed feature algorithms were created, such asSIFT

As shown in the figure above, a series of feature points are extracted by the SIFT algorithm.

An image extracted feature points have multiple, and each feature point is a local vector, for similarity calculation, often need to incorporate this a series of characteristic points first encoded as a global characteristics, namely global fusion feature vector coding for a local features vector (using the global feature vector said an image), Algorithms related to fusion coding include but are not limited to:

  • BOW
  • Fisher vector
  • VLAD

CNN features

After the rise of artificial intelligence, image feature extraction based on CNN Convolutional Neural Network is becoming more and more mainstream.

Image features extracted by CNN is a multi-dimensional vector, such as using VGG16 model to extract features may refer to: https://keras.io/applications…

Search engine

Because the image is transformed into a feature vector, so what the search engine has to do is actually vector retrieval.

Milvus is directly recommended here, just open source, can be very convenient to use in engineering projects, specific related content can be directly refer to the official documentation.