Copyright notice: This article was originally published by Noogel’s Notes. Please apply to my mailbox for reprinting.

This article is a study of artificial intelligence notes, first of all, ask the following question.

The relationship between artificial intelligence, machine learning and deep learning?

As can be seen from the figure, the concept of ARTIFICIAL intelligence is broader. Machine learning is a subset of it, and deep learning is a subset of machine learning.

The application fields of deep learning are computer vision, speech recognition, natural language processing and man-machine game with higher accuracy than machine learning based on mathematical statistics. This is mainly about taking notes on deep learning.

The development of deep learning

Deep learning is a synonym for deep neural networks, which originated in the last century and only became popular in the past few years.

Early neural network models are similar to free machine learning, which evolved from neurons in the human brain, as shown in the figure below (taken from TensorFlow Practical Google Deep Learning Framework) :

Then came perceptron model, distributed knowledge representation and neural network back propagation algorithm, and then convolutional neural network, cyclic neural network and LSTM model.

At the same time of the development of neural network, the research of traditional machine learning algorithm is also developing constantly. In the late 1990s, it gradually surpassed the neural network. At that time, the traditional machine learning algorithm has better recognition accuracy, but the development of neural network is difficult due to the limitation of data volume and computing capacity. In recent years, due to the emergence of cloud computing, GPU, big data, etc., which has paved the way for the development of neural network, AI has begun a new era. From my own perspective, I set up a vague dream in high school to participate in the research and development of artificial intelligence, so I learned computer in college. When my dream intersects with the development of The Times, the realization of my dream becomes more accessible.

The following chart compares the leading open source tools for deep learning (from TensorFlow’s Google Deep Learning Framework)



Caffe and TensorFlow are the two open source tools I’ve heard a lot about. The former is 20.1K and the latter 69.3K. TensorFlow is the most popular deep learning framework. This article and the following articles will record the learning process of TensorFlow. The following content is to introduce the basic concepts of TensorFlow. I try to avoid adding codes in the introduction, because the current TensorFlow updates quickly, and I find that many writing methods are no longer supported in the new version.

TensorFlow learning

TensorFlow has two important concepts: Tensor and Flow, Tensor is the data structure, Flow is the calculation model.

PS: TensorFlow is referred to as TF below

Tf Calculation model — Calculation diagram

Tf expresses the programming system of computation through computation diagrams, in which each node represents a computation and connections between different points express dependencies. The following lines of code express this:

Import tensorflow as tf a = tf.constant([1.0, 2.0], name='a') b = tf.constant([2.0, 3.0], name='b') result = a + bCopy the code

The code above will have nodes A, B, and A + B in the calculation diagram. Tf has a global graph by default and can also generate new graphs. Different graphs do not share tensors and operations.

Different resources, such as tensors, variables, or queue resources, can be managed in a collection in a computational graph.

Tf data model – tensors

Tensors are the form of tf to manage and represent data, which can be simply understood as multidimensional arrays. A zero-order tensor is a scalar, it’s a number; The first-order tensor is a one-dimensional array; A tensor of order n is an n-dimensional array. Tf.add (a, b, name=’add’) does not get a result at runtime, but a reference to the result, which is a tensor structure containing three attributes name, shape, and dtype. Name is just the name of a node. Shape is a dimension of a tensor, this is one-dimensional, length 2, that’s a very important property; Dtype is a data type. Each tensor has a unique data type, and the calculation between different tensors needs to ensure the same type.

The above example is a reference to adding two constants and then producing the computed result.

Tf run model – Sessions

The Session of tf is used to perform the defined operation. The Session is used to manage all the resources during the operation. When the calculation is complete, it helps recycle the resources, with tf.session () as sess: this method automatically closes the recycle resource when with ends.

Using tf.ConfigProto, you can configure parameters such as the number of parallel threads, GPU allocation policy, and calculation timeout period, and add the configuration to tF. Session to create a Session.

Forward propagation algorithm

Through the introduction of fully connected network structure, a neuron has multiple inputs and one output, and the output is the weighted sum of different inputs. Different input weights are the parameters of neural network, and the optimization of neural network is the process of optimizing parameter values. The fully connected network structure means that there are connections between any two nodes between two adjacent layers.

The forward propagation algorithm requires three pieces of information:

  • The first part takes the eigenvectors from the entity as input.
  • The second part is the connection structure of neural network, the connection relation between input and output of different neurons.
  • The third part is the parameters in each neuron.

In TF, parameters in neural network are saved and updated by tF.variable.

Import tensorflow as tf # w1 W1 = tf.variable (tf.random_normal([2, 3], stddev=1, seed=1)) # w2 W2 = tf.variable (tf.random_normal([3, 1], stddev=1, seed=1)) # Placeholder (tf.float32, Shape =(1, 2), name='input') # w1) y = tf.matmul(a, w2) with tf.Session() as sess: Sess.run (tf.global_variables_initializer()) sess.run(tf.global_variables_initializer()) Print sess.run(y, feed_dict={x: [[0.7, 0.9]]}) print sess.run(y, feed_dict={x: [[0.7, 0.9]]})Copy the code
[[3.95757794]]Copy the code

WeChat exceptional

Alipay rewards