Human-machine dialogue system and natural language processing

preface

One or two years ago, I focused on natural language processing and human-machine dialogue system due to my work needs. This paper will list their knowledge points and my own thinking.

NLP talks to man – machine

For ordinary enterprises, the main application of man-machine dialogue is task-based man-machine dialogue system. Both NLP and Chatbot are implemented in different ways academically and engineering. The research results and the latest research trend of human-machine dialogue system in the academic circle are seldom applied in engineering, and engineering has its own way to realize human-machine dialogue. NLP is the foundation of ChatBot, as a lot of NLP technology is used in the implementation process.

Involved in NLP

• Words: grammar, semantics, pragmatics.

• Phrase (sentence) : grammar, semantics, pragmatics.

• Discourse: grammar, semantics, pragmatics.

The study of words and phrases has been relatively mature, and the main research has been concentrated after 2000. Chapter research is not mature, and the main research was done after 2010.

The syntax tree:

• DG

• CFG

• PCFG

• LPCFG

Common algorithms:

• Search the S

• Dynamic programming D

• Classification algorithm C

• Sequence label S

• Optimization algorithm O

Mainstream framework of human-machine dialogue

• NLU: Understands user input and converts natural language into structured representation.

• DM: System decision.

• NLG: Natural language generation, transforming structured representations into natural language.

Therefore, according to the mainstream framework, it is divided into three main modules, namely, NLU, DM and NLG.

Natural language understanding

Natural language understanding consists of three parts:

• Domain identification is mainly to judge the types of tasks, such as conference room reservation, train ticket purchase, meal reservation and so on.

• Intent recognition, which is used for intent, such as user confirmation and rejection.

• Slots are filled to extract important information related to tasks, such as meeting reservation. Slots can be defined as meeting place and meeting time.

The user enters “I will have a meeting in the company tomorrow”, and after natural language comprehension processing (sentence classification and sequence labeling), the result is:

Domain: Intent: Provide information Slots:{Time: tomorrow; Location: the company}Copy the code

Research Status of Natural Language Understanding:

• Rule-based method, regular expression, more labor – consuming, poor flexibility, poor portability.

• Statistics-based approach, single modeling and joint modeling. Domain recognition, intention recognition and slot filling are studied separately by separate modeling, which are implemented by support vector machine, convolutional neural network, decision tree, conditional random field and cyclic neural network respectively, resulting in error accumulation.

Research and development steps:

1. Define labels based on task requirements.

2. Prepare data, observe data patterns if using a rules-based approach, and extract templates. If the statistics-based method is used, the data should be annotated, which is divided into training set, verification set and test set. The typical amount of data is hundreds of thousands.

3. Write regular expression or build model training model.

4. Complete NLU research and development.

5. Iterative optimization.

Dialogue management

The main framework of the dialogue management module is shown below.

• The user enters “wudaokou nearby restaurants”. After NLU processing, the intent = ask,slots={location: Wudaokou}.

• Dialog state tracking module output slots={location: five channel}.

• Action candidates for restaurant retrieval and requirements clarification.

• Policy sorts candidate actions.

• Action execution Executes actions in order to update the interaction state

• NLG production responded, “Any other requirements?”

• The user continues typing.

Natural language generation

Regarding the natural language generation module NLG, the typical concept to text generation consists of three parts,

1. Content selection, selecting the appropriate content from the input and determining the structure of the output text.

2. Sentence planning determines the lexical content of individual sentences.

3. Content expression, render selected sentence to output.

Traditional practices involve:

• Probabilistic context-free grammar

• n-gram

• Properties test

• Threshold filtering

• Text filtering

Methods based on deep learning:

• RNN

• LSTM

• seq2seq

• encoder-decoder

Hundreds of thousands of training samples are needed.

Questions and Reflections

• Human-computer dialogue is currently only effective in a small area, and it is unlikely to achieve a wide range of universal chatbots.

• At present, the main body frame of the mainstream dialogue system is almost the same, but it still needs to be effective. The only way to be effective is to make it very small, you can make it very small, just a small piece of business.

• Conversational systems involving machine learning or deep learning require a large number of training samples, and there is no way to skip the manual tagging step.

• NLU, DM and NLG modules are mainly called in the form of libraries in engineering. If they are called in the form of services, it will be heavier and more troublesome to upgrade, involving multiple project teams.

• Data is the most important thing, you can’t do anything without it, and it needs to be constantly updated, closed loop with the online system.

• Currently, the ones with good engineering results are mainly based on traditional rules, which will introduce machine learning or deep learning in some links rather than directly end to end.

• Whether it is artificial intelligence or deep learning, it is only by returning to the essence of the problem that artificial intelligence can be truly utilized, rather than blindly boasting about lofty ARTIFICIAL intelligence.

• In terms of sample labeling, a user-friendly labeling system is needed to facilitate the use of business personnel or labeling personnel.