Chapter 1 Basic concepts of TensorFlow

TensorFlow Tutorial: Understanding Deep Learning

TensorFlow tutorial: Tensor Basics

TensorFlow tutorial: Session basics

TensorFlow TensorFlow Graph tutorial

TensorFlow installation

The TensorFlow installation website has a detailed tutorial. I won’t repeat it here. We focused more on understanding the three basic concepts of TensorFlow. So that we can learn to be more silky when we learn neural networks later.

Tensor
Session
Graph

Tensor

Tensors are TensorFlow’s form of managed data, which can be understood simply as arrays or multi-dimensional arrays.

So if you don’t have any idea, let’s look at one of the simplest tensors.

# define tensor
v1 = tf.constant(1,name='v1',shape=(),dtype=tf.float32)
v2 = tf.constant(2,name='v2',shape=(),dtype=tf.float32)

Define a tensor
add = v1 + v2

# create session, run operation, more on this later
with tf.Session() as sess:
    print(sess.run(add))   Output: 3
Copy the code

We can think of tensor as a constant in TensorFlow, so at the first step, we define two constants v1 and v2 on both sides of the tensor, and then we define a add constant, and it’s going to be the sum of v1 and v2. Then we perform the calculation through the Session and get the result of the calculation.

Through the above visualizations produced by TensorFlow, I can intuitively feel the process.

Three properties of tensors

Tensor has three important properties: name, Shape, and Type. Let’s introduce them one by one.

name

Name is the tensor’s unique identifier.

Define tensor, the first parameter is the value; The second parameter is the tensor's name;
Number three is tensor's shape; The fourth parameter is the tensor type
v1 = tf.constant(1,name='v1',shape=(),dtype=tf.float32)
v2 = tf.constant(2,shape=(),dtype=tf.float32)

Define a tensor
add = v1 + v2

print(v1) #Tensor("v1:0", shape=(), dtype=float32)
print(v2) #Tensor("Const:0", shape=(), dtype=float32)

Copy the code

TensorFlow will automatically define the name property for us when we do not specify the name property.

Tensor is shown in the form of node:src_output, where Node is the tensor name, and src_output is the output of the current tensor. For example: v1:0 is the first output of the v1 node (src_output starts at 0)

shape

Shape is latitude, and that describes latitude for tensor, so let’s do a simple example:

# define two two-dimensional arrays
v1 = tf.constant(1,name='v1',shape=(2.2),dtype=tf.float32)
v2 = tf.constant(2,name='v2',shape=(2.2),dtype=tf.float32)

add = v1 + v2

with tf.Session() as sess:
    [[# 1. 1.]
    # [1. 1.]]
    print(sess.run(v1)) 
    # [[2. 2.]
    # [2. 2.]]
    print(sess.run(v2))
    [[# 3. 3.]
    # [3. 3.]]
    print(sess.run(add))

Copy the code

So in this example, we’ve created two tensor shapes, and if you think two-dimensional is hard to understand, you can just think of it as a matrix with two rows and two columns, and in deep learning, you’ll always have a tensor as a matrix.

And finally, we compute v1+v2 through Session, which actually represents matrix addition. If you are not familiar with the operation of the matrix, you can take a look at ruan Yifeng teacher’s blog about the matrix.

And the tensor has a lot of dimensions, and we’re going to get to them as we go along.

type

Every tensor has a unique type. TensorFlow performs type detection on all tensors involved in the calculation and generates an error when a type mismatch occurs.

#TypeError: Input 'y' of 'Add' Op has type float32
#that does not match type int32 of argument 'x'
v1 = tf.constant(1,name='v1', shape = (1, 2), dtype = tf. Int32) v2 = tf. Constant (name = 2'v2',shape=(2,1),dtype=tf.float32)
add = v1 + v2

Copy the code

You have to add the tensor of v1 to int32 and v2 to float32.

TensorFlow supports 14 different types: It mainly includes real numbers (tf.foat32, tF.float64), integers (tf.int8, tf.intl6, tf.int32, tf.int64, tf.uint8), Boolean types (tf.bool), and complex numbers (tf.plex64, Tf.com plex128).

A couple of ways to create a tensor

tf.constant

Generate the specified tensor

We can specify tensor values and dimensions by using value and shape
v1 = tf.constant(value=1,name='v1',shape=(1.2),dtype=tf.float32)
[[# 1. 1.]]

You can also write a value with a dimension directly in value
v2 = tf.constant(value=[[1.2.3], [4.5.6]],dtype=tf.float32)
[[# 1. 2. 3.]
# [4. 5. 6.]]
Copy the code

tf.zeros

Generate a tensor that’s filled with all zeros

v3 = tf.zeros((1.2),tf.float32,name='zeros')
# [[0. 0.]
# [0. 0.]]

Copy the code

tf.ones

Create a tensor that’s all filled with ones

V4 = tf. 'ones ((2, 2), tf float32, name ='ones')
[[# 1. 1.]
# [1. 1.]]
Copy the code

tf.diag

Generates a diagonal matrix with the diagonal being the specified value and the other values being 0

V8 = tf. Diag (name = [1, 2, 3]'diag')
1 0 # [[0]
2 0 # [0]
# [0, 0, 3]]
Copy the code

tf.truncated_normal

Generate a normally distributed matrix

V9 = tf. Truncated_normal ((2, 3))# [[0.18460223 0.11237118 0.61098295]
#[0.02013025 0.3336802-1.0556936]]
Copy the code

Variable variables

We use Variable to define variables that represent tensors whose values can be changed by running operations on them.

Tf. Variable Creates a Variable

V1 = tf. Variable (tf. 'ones (shape = [1, 2], dtype = tf. Float32), name ='v1')
with tf.Session() as sess:
    Initialize variables
    sess.run(tf.global_variables_initializer())
    print(v1)
    
      
Copy the code

We initialize a Variable with tf.variable (), the first parameter is our tensor, the second parameter is the name of the Variable, and the difference with tensor is: Variables must be initialized by calling tf.global_variables_initializer() via session before they can be used. Otherwise, an error is reported.

Tf.get_variable Creates a variable

Tf.get_variable can also create a variable. The specific creation method is as follows:

v1 = tf.get_variable(name='v1',shape=(1,2),dtype=tf.float32,initializer=tf.ones_initializer()) with tf.session () as sess: sess.run(tf.global_variables_initializer())print(v1)
    
      
Copy the code

The arguments to tf.get_variable are similar, except that we use the initializer argument to initialize the value of the variable.

The difference between the two creation methods

V1 = tf. Variable (tf. 'ones (shape = [1, 2], dtype = tf. Float32), name ='same') v2 = tf. Variable (tf) ones (shape = [1, 2], dtype = tf. Float32), name ='same')
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print(v1)
    print(v2)

      

      
Copy the code

We saw that we defined two variables with the same name, and we initialized them without an error, and TensorFlow changed the name of our duplicate variable to same_1 to avoid the two variables having the same name.

v1 = tf.get_variable(name='same', shape = (1, 2), dtype = tf. Float32,, initializer. = tf ones_initializer ()) v2 = tf. Get_variable (name ='same',shape=(1,2),dtype=tf.float32,initializer=tf.ones_initializer()) with tf.session () as sess: sess.run(tf.global_variables_initializer())print(v1)
    print(v2)
# error: Variable same already exists, disallowed.
#Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? 
Copy the code

When we use get_variable to create two variables with the same name, we will find that the compilation error will alert us to the variable name conflict. What if we want to reuse the previous variable?

with tf.variable_scope('my_scope',reuse=tf.AUTO_REUSE):
    v1 = tf.get_variable(name='same', shape=(1, 2), dtype=tf.float32, initializer=tf.ones_initializer())
    v1 = tf.get_variable(name='same', shape=(1, 2), dtype=tf.float32, initializer=tf.zeros_initializer())
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print(v3)
    print(v4)

      

      
Copy the code

By defining a scope, my_scope, the variable name becomes scope_name/name:src_output. Reuse Specifies whether a variable in this scope can be reused. If its name is the same, reuse a variable with the same name and redefine a new initializer for it.

analyse

This section is just about the most important tensor and Variable in TensorFlow, and tensor and Variable are used a lot in deep learning. In this case, we can think of tensor as a constant definition, and Variable is a Variable, it can change its value as it goes along, but it has to be initialized by Session before it can be used.

In the above article, we have a vague sense of the effect of Session (Session). After we define the add calculation, it will not be executed immediately to get the result, but need the help of Session to calculate and get the result. We’ll talk more about this in the next section.