Look directly at the code:

A, tensorflow

#tensorflow import tensorflow as tf import random import numpy as np x_data = np.random.randn(100).astype(np.float32) Y_data = x_data * 0.1 + 0.3 weights = tf.Variable(tf.random_uniform([1], -1.0, 0)) allow biases = tf.variable (tf.zeros([1])) Y = weights * x_data + heuristic = tf.reduce_mean(tf.square(Y-y_data)) Optimizer = tf. Train. GradientDescentOptimizer (0.5) "train" = optimizer. Minimize the init = (losses) tf.initialize_all_variables() with tf.Session() as sess: sess.run(init) for step in range(10): sess.run(train) print(step, sess.run(weights), sess.run(biases))Copy the code

2, pytorch

# PyTorch Import Torch Import numpy as NP x_data = NP.random.randn (100).astype(NP.float32) y_data = x_Data * 0.3 + 0.1 x_data = torch.from_numpy(x_data) y_data = torch.from_numpy(y_data) weights = torch.rand(1,requires_grad=True) biases = Torch. Zeros (1,requires_grad=True) print(" weights:{}, biases:{}". Format (weights. biases.data)) parameters = [weights, biases] criterion = torch.nn() optimizer = torch.optim.SGD(parameters, For I in range(10): y = weights * x_data + biases losses = criterion(y_data, y) print(losses.data, weights.data, biases.data) optimizer.zero_grad() losses.backward() optimizer.step()Copy the code

Third, keras

#keras from keras.models import Sequential from keras.layers import Dense from keras.optimizers import SGD import numpy As np x_data = Np.random.randn (100).astype(Np.float32) y_data = x_data * 0.3 + 0.1 model = Sequential() Model.add (Dense(input_dim=1, units=1)) Model.compile (loss=" MSE ", Optimizer =SGD(LR =0.5)) for I in range(10): losses = model.train_on_batch(x_data, y_data) w, b = model.get_weights() print(losses, w, b)Copy the code

Let’s talk about some of their similarities and differences:

Difference:

  • Pytorch requires tensor, tensorflow and Keras can be Numpy;
  • Tensorflow1.x is a static graph. We can define the related operations and execute them in the session. Pytorch uses a dynamic graph where we calculate the associated losses as we go through the loop; Keras package is more advanced, just like the loss function and optimization method in model.compile(), we do not display calculation;
  • Tensorflow requires that after the graph is defined, Session() performs the computation on the graph;
  • Tensorflow initializes parameters by defining a tF.initialize_all_variables (), and then performs initialization operations in the session first: sess.run(init); Pytorch encapsulates the related parameters into a list and passes it to the optimizer; As for Keras (I know that Sequential() is used to build the model, I don’t know if there is a custom parameter implementation, no classes);
  • Tensorflow uses optimizer. Minimize losses (losses), Pytorch uses loss. Backward () and optimizer.step(). Keras can use model.train_on_batch() directly;

Similarities:

  • The general idea is the same: input data — define parameters — calculate losses — define optimizer — iterate to minimize losses.

Conclusion: This is just a simple comparison, but such a set of procedures can be applied to various neural networks, but the data processing, network structure construction and other differences.