If we realize a CNN network, classification recognition can be accomplished through two convolution layers on MNIST. However, in the process of code debugging, we often want to know the effect changes during our network training, such as the curve of loss and accuracy. \

Of course, we can print out the data during the training process, but one is not intuitive enough, and the other is not as expressive as graphics. So this note introduces tensorBoard to perform visual operations.

1. TensorBoard is introduced

Tensorboard started as a visual tool in TensorFlow, which can be used to show network diagrams, data processing flow, and indicator changes during execution. Especially when training the network, different Settings of network parameters (such as weight, bias, number of convolution layers, number of full connection layers, etc.).

The popular point is that the changes of various parameters and indicators in the process of network training can be displayed in the form of charts. In addition, the structure of network model and photos of training data can also be displayed.

The excellent performance in TensorFlow has made PyTorch officially shipped with TensorBoard since version 1.2.0. This makes it easy to visualize during PyTorch training.

2. Run an example first

According to our custom, first find a simple example to run, and then step by step to learn the principle of it. Let’s start with an example of an official document:

from torch.utils.tensorboard import SummaryWriter
writer = SummaryWriter()
x = range(100)
for i in x:
    writer.add_scalar('y=2x', i * 2, i)
writer.close()
Copy the code

After executing this code, tensorBoard saves a log of the training in the ‘./runs’ folder. Let’s take a closer look at this example to help understand the implementation: \

  • First, import the SummaryWriter, which is a class that needs to instantiate an object, which we’ll call Writer. One of the initialization parameters that we care about most is the address of the log file. Default to the runs folder in the current file if there is no input, or create one if there is no runs folder.

  • Next, we see that one of writer’s methods is called in the for loop: add_scalar(), whose parameters are defined as follows:

    def add_scalar(self, tag, scalar_value, global_step=None, walltime=None):

  • Let’s go over some of the more important parameters of this method. The function of this method is to store the data we want in the log file, so the tag of the data (the title of the drawing) is the parameter tag, and the scalar_value is the value currently stored (the Y-axis of the drawing). Global_step is where we store the data once (the X-axis of the drawing).

  • So in this loop, we can see that we’re storing data labeled ‘y=2x’, and we’re storing 2* I at step I, so by going through this loop, we’re storing the y value at each step, or we’re storing the coordinates of each point of a graph in log.

  • Finally, use write-.close () to terminate the object.

Then we call the following command in terminal:

The idea behind this step is to use tensorboard to call the data in the target log file for visualization. You can also specify the port by adding the parameter – port. We don’t specify it here, so we’ll choose the default port 6006.

We then copy the displayed address: http://localhost:6006/ into the browser and see the following:

Here we can see that we have drawn a graph where step goes from 0 to 99, and the vertical axis is the value of the corresponding step, and the final graph is exactly a diagonal line y=2x.

Through this example, we saw how to complete a visual process using TensorBoard. Knowing the meaning of several key steps, we will modify the code in the second note to complete the visualization of data in a CNN training process.

3. Visualize CNN training data

The previous code is used for modification here. At that time, we printed loss in the code training process, calculated accuracy on the test set every 50 steps, and then printed loss on the training set and accuracy on the test set.

Here we mainly look at the code during the training process:

writer = SummaryWriter('tb_mnist')
for epoch in range(EPOCH) :for step, (b_x, b_y) in enumerate(train_loader) : #print(b_x.shape); break


        if cuda_gpu:
            b_x = b_x.cuda()
            b_y = b_y.cuda()


        output = cnn(b_x)
        loss = loss_func(output, b_y)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()


        if step % 50= =0:
            test_output = cnn(test_x)
            pred_y = torch.max(test_output, 1) [1].data


            if cuda_gpu:
                pred_y = pred_y.cpu().numpy()
            else:
                pred_y = pred_y.numpy()


            accuracy = float((pred_y == test_y.data.numpy()).astype(int).sum()) / float(test_y.size(0))
            print('Epoch: ', epoch, '| train loss: %.4f' % loss.data, '| test accuracy: %.2f' % accuracy)


            writer.add_scalar("Train/Accuracy", accuracy, step)


        writer.add_scalar("Train/Loss", loss.item(), step)
Copy the code

You can see that the first line also instantiates a Writer, and everything else is the same, until the last two lines below are modified. The first is to add the corresponding step’s accuracy into Train/ accuracy when printing each time. The second is outside the printed for loop, where the current Loss information is saved for each step.

Tensorboard = tensorboard = tensorboard = tensorboard = tensorboard = tensorboard

It can be seen that the two pictures are accuracy on the left and loss on the right. And you can see the accuracy on the left because we save every 50 steps, so the data is obviously creased. While Loss is printed for each step, it can be seen that the change curve of Loss in the training process is described in great detail.

During the training of our model, add_Scalar () can be used to describe the change process of some target parameters.

4. Visualization of pictures and models

After the introduction to diagram drawing, let’s show an example from the official documentation to further illustrate the wide range of uses of TensorBoard. The examples here are using Tensorboard to display images and network structures respectively. Take a look at the official code first:

import torch
import torchvision
from torch.utils.tensorboard import SummaryWriter
from torchvision import datasets, transforms


# Writer will output to ./runs/ directory by default
writer = SummaryWriter()


transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))])
trainset = datasets.MNIST('mnist_train', train=True, download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True)
model = torchvision.models.resnet50(False)
# Have ResNet model take in grayscale rather than RGB
model.conv1 = torch.nn.Conv2d(1.64, kernel_size=7, stride=2, padding=3, bias=False)
images, labels = next(iter(trainloader))


grid = torchvision.utils.make_grid(images)
writer.add_image('images', grid, 0)
writer.add_graph(model, images)
writer.close()
Copy the code

With the previous example, we can easily understand the meaning of this code. First data to choose the mnist data set, we use the previous network structure directly call the pytorch resnet cabin model, through this command, to be obtained torchvision. Models. Resnet50 ().

The focus is on the last section of code, grid, which uses the tools integrated in TorchVision to read a batch of photos from the data set and arrange them in a grid. We can see the number of photos, the previous trainLoader set batCH_size = 64, that is to say, 64 photos.

Then we call the corresponding method using Writer, using add_image for the image and add_graph for the model graph. Let’s take a look at the effect:

The image of sample

Graph example

The two pictures show the effects of image and graph respectively. One is 64 photos during training, which are splice into 8*8 grid. One is the network structure of resnet50, which also supports further expansion by double-clicking on it. This is what it looks like when I have expanded it properly.

conclusion

In today’s article, we introduced the tensorBoard, and then, through a few examples, stepped into the use of TensorBoard. Of course, tensorBoard does much more than what we’ve covered in this article, but it’s enough to get started. If you want to learn more about TensorBoard, you can visit the official documentation: pytorch.org/docs/stable…

There are many other ways to use TensorBoard in the official documentation, with clear interfaces and examples.

I hope today’s article can let you know how to use TensorBoard. In the process of network training, effective visualization can help us quickly locate bugs and better understand the training effect of network. Give it a try

Recommended reading:

Read common cache problems in high concurrency scenarios \

Using Django to develop DApp\ based on Ethereum smart contract

Let’s read Python tasks like celery\

5 minutes on chain calls in Python

Create a Bitcoin price alert application in Python

▼ clickBecome a community member and click on itIn the see