Easy to understand machine learning algorithms

video

Select data randomly from the source data to form several subsets

S matrix is the source data, with 1-N pieces of data. A, B and C are features, and the last column C is the category

Generate M submatrices randomly from S

These M subsets are going to give you M decision trees and if you put new data into these M trees, you’re going to get M classification results, and you’re going to count the number of predicted categories, and you’re going to use that category as your final prediction

When the prediction target is probability, the range needs to be greater than or equal to 0, less than or equal to 1. At this time, the simple linear model cannot do this, because when the range is not within a certain range, the range also exceeds the specified interval.

So it would be good to have a model with this shape

So how do you get this model?

So this model has to satisfy two conditions: greater than or equal to 0, less than or equal to 1 and if it’s greater than or equal to 0 you can choose the absolute value, the square value, the exponential function here, it must be greater than or equal to 0 and less than or equal to 1 if you divide, the numerator is itself, and the denominator is itself plus 1, so it must be less than 1

And if you do another transformation, you get a Logistic regression model

You can calculate the coefficients from the source data

And you end up with the logistic graph

support vector machine

To separate the two types and obtain a hyperplane, the optimal hyperplane is to maximize the margin of the two types. Margin is the distance between the hyperplane and its nearest point, as shown in the figure below, Z2>Z1, so the green hyperplane is better

I’m going to represent this hyperplane as a linear equation, one class above the line, all greater than or equal to 1, and one class less than or equal to minus 1

The distance from point to surface is calculated according to the formula in the figure

Therefore, the expression of the total margin is as follows. The goal is to maximize the margin, so the denominator needs to be minimized, which becomes an optimization problem

For example, three points, find the optimal hyperplane and define the weight vector = (2,3) – (1,1).

The weight vector is obtained as (a, 2a), the two points are substituted into the equation, (2,3) and its value = 1, and (1, 1) and its value = -1, then the values of a and the moment w0 are solved, and the expression of the hyperplane is obtained.

After a is solved and substituted into (a, 2a), the support vector is obtained

The equation of a and W0 into the hyperplane is the support Vector machine

Take an example of an application in NLP

Give a paragraph of text, return to the emotional classification, the attitude of the paragraph is positive or negative

To solve this problem, just look at some of the words

This text will only be represented by a few words and their count

The original question was: To give you a word, which category does it belong to is turned into a relatively simple and easily solved question by Bayes rules

The question becomes, what is the probability of this statement in this category, and of course, remember the other two probabilities in this formula

The probability of the word “love” occurring in positive situations is 0.1, and in negative situations 0.001

k nearest neighbours

When you give a new number, the number of k points closest to it, the number of k points closest to it, that number belongs to that category

To distinguish the cat and dog, according to the claws and sound features, circle and triangle are categorized, so what category does the star represent

When k = 3, the points connected by these three lines are the nearest three points, so there are more circles, so this star belongs to the cat

I want to divide a set of data into three categories, the pink value is high, the yellow value is low, and I want to initialize it first, so I choose the simplest 3,2,1 as the initial value of each category and then I calculate the distance between each of the rest of the data and the three initial values, and then I put it into the category of the nearest initial value

After sorting the categories, calculate the average of each category as the center point of the new round

After a few rounds, the group has stopped changing, and you can stop

Adaboost was one of Bosting’s methods

Bosting is to synthesize several classifiers with poor classification effect and get a classifier with good effect.

In the figure below, the left and right decision trees, individually, are not very good, but putting the same data into them, considering the two results together, will increase the credibility, right

Adaboost chestnut, handwriting recognition, can capture a lot of features on the drawing board, such as the direction of the beginning point, the distance between the beginning point and the end point and so on

During training, the weight of each feature will be obtained. For example, the beginning parts of 2 and 3 are very similar. This feature plays a small role in classification, so its weight will be small

This alpha Angle has strong identification, and the weight of this feature will be large. The final prediction result is the comprehensive consideration of the results of these features

Neural Networks are suitable for an input that can fall into at least two categories

NN consists of several layers of neurons and the connections between them. The first layer is input layer, and the last layer is Output layer

Each layer has its own classifier in the hidden layer and output layer

Input is input into the network, activated, and the calculated scores are transferred to the next layer, which activates the neural layer behind. Finally, the scores on the nodes of the output layer represent the scores belonging to various types. In the example below, the classification result is class 1

The same input is transmitted to different nodes. The result is different because each node has different weights and bias

This is also called forward Propagation

Markov Chains are composed of State and Transitions

The sentence: “The quick brown fox jumps over the lazy dog” : to get markov chain

Step, first set each word into a state, and then calculate the probability of switching between the states

This is the probability calculated in one sentence. When you use a large amount of text to do statistics, you will get a larger state transition matrix, such as the words that can be connected after the, and the corresponding probability

In life, the alternative result of keyboard input method is the same principle, the model will be more advanced