0. Write first

This time to share a classic ARTICLE on NLP text matching, the main idea is to use the idea of image recognition to solve the PROBLEM of NLP text matching, after all, the development of computer vision technology at that time was extremely rapid.

Personal experience:

  1. The text matching problem can be transformed into a binary classification problem, and the output probability is similarity
  2. In this paper, the idea of image recognition is applied to text matching task, and text features are extracted by convolution

Thesis Address:

Arxiv.org/pdf/1602.06…

1. The background

Text matching is widely used in search engines, question answering systems, information flow related article recommendations and other scenarios, in order to find the text similar to or related to the target text in the text library. The TextMathcing model introduced in this paper is an end-to-end text matching method, which introduces the solution idea of image recognition task in the field of computer vision into the text matching model. Let’s take a look at how the author designed and implemented it.

2. Model architecture

The TextMatching model architecture is shown below.

We broke down the model architecture into the following steps:

  1. Text embedded in this paper, the method first, word vector model are used to get the text embedding vector of each word or sentence, this step is also very general steps, the current mainstream word vector model for word2vec/fastText/glove/Bert, etc., this paper selects the glove.

  2. The similarity matrix is calculated for the cross product of the matched text (specifically, it is a two-dimensional array, and each dimension is the embedding vector of each word in the text), and the similarity score between each word is obtained. If two texts are composed of M and N words respectively, then the size of the similarity matrix is M*N. The author provides three methods to calculate the similarity score between words:

    A. 0-1: The corresponding word is 1. Otherwise, the value is 0. The disadvantage of this method is that it cannot capture semantic matching relation between similar words.

    ​

    There are many similarities between cosine and cosine

    C. the dot product

    Through the experiment, scheme 3 has the best effect.

  3. CNN feature extraction

    Why the idea of this paper is to do text matching like image recognition? The key step is that the author uses CNN convolution layer commonly used in image recognition to extract text features. TextMatching model uses two layers of CNN to extract features from similarity matrix. It should be noted that the size of text similarity matrix generated by different text pairs is not consistent. In order to keep the size of feature map extracted by CNN consistent when it is input into the full connection layer, the author uses a layer of dynamic pooling layer after the first layer CNN to dynamically adjust the size of feature map.

  4. The connection layer

    After extracting features by CNN, TextMatching sends features into two full-connection layers, and then softMax gets model inference results. The output probability value can be considered as the matching degree of text pairs.

3. Model evaluation

The author compares the model in this paper with other mainstream TextMatching models, and it can be seen that the effect of TextMatching model proposed in this paper is better than the mainstream models at that time.

4. To summarize

This paper introduces a classic work of text matching. The idea is to introduce the image recognition method based on CNN into the text matching model to improve the text matching effect.