Visit Flyai. club to create your AI project with one click



These days, I have been flooded by this BERT paper. What is the model of “reading comprehension exceeds human” that has swept 11 records?


BERT stands for Bidirectional Encoder Representation from Transformers. Transformer’s Bidirectional Encoder Representation improves architecture fine-tuning methods. Language Representation Model (SOTA) is a language representation model trained by large data, large model, and large computational overhead. SOTA has achieved optimal results in 11 natural language processing tasks.


In short, BERT’s authors argue that neither unidirectional nor bi-directional can fully understand the meaning of the entire statement, and that a better way is to use context-wide prediction so that multiple points can be used at once, not just from the front to the back, Or from the back to the front. Instead, it uses two novel unsupervised forecasting tasks:


1. Cover 15% of the words randomly in an article. The task of the model is to correctly predict the covered words according to the context, so as to preliminarily train the parameters of the model.


2. Two sentences are randomly selected from a certain number of articles, one is two consecutive sentences, the other is discontinuous, so that the model can judge whether it is continuous or not to further modify parameters.


BERT has taken first place. Source: Gluebenchmark.com


Source: reddit


We’ll be watching as Google releases all pre-trained models and code.


After all, Google is Still Google,


Google is all you need.


The new era of NLP just began a few days ago, says Thang Luong of the Google team


If you want to get a PDF of the original paper, please reply to the official account
BERT
Can be




Tutorial | Jupyter Notebook primer – the foundation of the fan dead person operation

Tutorial | Jupyter Notebook based tutorial – the use of shortcuts

Tutorial | of the Python Numpy ndarray basic introduction 1

Tutorial | of the Python Numpy ndarray basic introduction 2

Tutorial | of the Python Numpy ndarray basic introduction to 3

Tutorial | of the Python Numpy ndarray introduce four basic

Tutorial | generic functions: fast elements series set of functions

Tutorial | using arrays to programming (on)

Tutorial | using arrays to programming (under)


Click on theRead the original, to learn more


— the End —