“This is the ninth day of my participation in the First Challenge 2022. For details: First Challenge 2022”
preface
This is mainly divided into two parts, one is how to intuitively show the effect of our prediction, or error, the other is how to build the network, and understand the neural network again.
The neural network
We gave an example earlier of deriving the workflow of a neural network from linear regression. We’re going to use the loss function to propagate back, so we’re going to use the loss function to do gradient descent, and we’re going to assume that y is equal to x2 plus 2 and we’re going to put in x and y but we’re going to guess 2 and 2 so y is equal to ax plus b of a and b. And then for nonlinear fitting, we have an activation function, so what is the activation function? It’s a nonlinear transformation of our linear fit. (learning neural networks, machine learning, intelligent optimization algorithm, you will find that most of the mathematical modeling can use these things to do, such as the 2021 mathematical modeling of A question I don’t understand, actually not difficult to C questions but too lazy to analysis, B directly, first to A fitting, and then I directly use Fourier fitting, neural network prediction relations (this approach can have, But fitting is absolute, because the data is too little, so want to have a lot of error, the sensitivity analysis to filling holes (however goose I forgot the process), the final optimization direct optimization algorithm, and then there is a problem to also want to evaluate analysis, the final step according to your analysis with experiments, it is to analyze the insufficiency, also is your model is not enough, Those numbers are still needed to correct. Although the practice is very violent, but much better than language modeling, and the analysis in place is able to win the national award).
In fact, the neural network can be divided into three parts in the operation process. The first part is initialization, which is what I said: first guess a few parameters randomly; the second part is to optimize our unknown parameters by reverse transfer (namely gradient descent) through the loss function; the last step is various optimizations
OK, let’s look at a basic neural network model
And that network model we had last time
Here’s what the whole process actually looks like
Build the linear regression model by hand
Now, it’s time to build a simple linear model using PyTorch, and let’s briefly talk about our APILinearThis is our linear layer, compared to the picture above
OK, now let’s go straight to the code
from torch import nn
from torch.optim import SGD
import torch
class LineBack(nn.Module) :
def __init__(self) :
super().__init__()
self.line = nn.Linear(1.1)
def forward(self,x) :
x = self.line(x)
return x
myline = LineBack()
optim = SGD(myline.parameters(),lr=0.01)
lossfunction = nn.MSELoss()
# Data generation
x = [i for i in range(10)]
y = [(i*2 + 2) for i in range(10)]
# Data conversion
x = torch.tensor(x,dtype=torch.float)
x = x.reshape(-1.1) # becomes 10 rows in a column
y = torch.tensor(y,dtype=torch.float)
y = y.reshape(-1.1)
for epoch in range(1.1001):
optim.zero_grad()# Clear gradients to prevent impact
outputs = myline(x)
loss = lossfunction(outputs,y)
loss.backward()
optim.step()
if(epoch % 50= =0) :print("Training: the total error of {} times is {}".format(epoch,loss.item()))
Copy the codeThen, let’s see what happens
Overall, it’s pretty simple.
Our y is equal to 2 times x plus 2
Nonlinear model building
So we built the linear model, so now let’s look at the nonlinear model.
Here we have to use the activation function because it’s nonlinear. Rule is used here
from torch import nn from torch.optim import SGD import torch import torch.nn.functional as F class LineBack(nn.Module): Def __init__(self): super().__init__() self.line1 = nn.linear (1,10) self.line2 = nn.linear (10,1) def forward(self): X = f.relu (self.line1(x)) # this is the same as nn.relu () Return x myline = LineBack() optim = SGD(myline.parameters(),lr=0.01) Lossfunction = nn.mseloss () # data generation x = torch. Linspace (-1, 1, 1) X = x.row (-1,1) x = x.row (-1,1) x = x.row (-1,1) y = y.row (-1,1) for Epoch in range (1100 1) : Outputs = myline(x) Loss = lossfunction(outputs,y) loss. Backward () optim.step() if(epoch % 50) = = 0) : print (" training: general error was {} {} time ". The format (epoch, loss. The item ()))Copy the code
TensorBoard intuitive display
We didn’t look very good on the console, so we had to do vago diagrams, we could have just used Matplotlib but in our TensorFlow or PyTorch we have tensorBoard which is a very powerful thing to draw diagrams.
But before you can use it, you need to download it
pip install tensorboard
Or use Conda, depending on your environment, I am directly in the pyTorch environment native environment, I dual system, and previously had the VirtualEnv to create the development environment.
from torch import nn
from torch.optim import SGD
import torch
import torch.nn.functional as F
from torch.utils.tensorboard import SummaryWriter
class LineBack(nn.Module) :
def __init__(self) :
super().__init__()
self.line1 = nn.Linear(1.10)
self.line2 = nn.Linear(10.1)
def forward(self,x) :
x = F.relu(self.line1(x)) # this is the same as nn.relu (), except that this function can be used in forward
x = self.line2(x)
return x
wirter = SummaryWriter("logs") # Graph tool,logs save path
myline = LineBack()
optim = SGD(myline.parameters(),lr=0.01)
lossfunction = nn.MSELoss()
# Data generation
x = torch.linspace(-1.1.100) # Generate 100 numbers from -1 to 1
y = x.pow(2) + 0.2*torch.rand(x.size())
# Data conversion
x = x.reshape(-1.1) # becomes 10 rows in a column
y = y.reshape(-1.1)
for epoch in range(1.1001):
optim.zero_grad()# Clear gradients to prevent impact
outputs = myline(x)
loss = lossfunction(outputs,y)
loss.backward()
optim.step()
if(epoch % 50= =0):
percent =(1- loss.item())*100
wirter.add_scalar("Proximity",percent,epoch)
print("Training: the total error of {} times is {}".format(epoch,loss.item()))
Copy the codeCode as above, then type in our console
