Neural network – support vector machine
Support Vector Machine (SVM) was first proposed by Cortes and Vapnik in 1995. It shows many unique advantages in solving small sample size, nonlinear and high-dimensional pattern recognition, and can be generalized to other Machine learning problems such as function fitting. 1 Mathematics section 1.1 Two-dimensional space
Genetic algorithm
• Genetic Algorithm (GA) is an evolutionary Algorithm whose basic principle is to imitate the evolutionary law of “natural selection and survival of the fittest” in biology. It was first proposed by Professor J. Holland of University of Michigan in 1967. • Genetic algorithms start with a population that represents the set of possible potential solutions to a problem. A population consists of a number of individuals encoded by a gene. Therefore, the first step is to achieve the mapping from phenotype to genotype, that is, coding. After the generation of the initial population, according to the principle of survival of the fittest and survival of the fittest, generation by generation evolution produces better and better approximate solutions. In each generation, individuals are selected according to their fitness in the problem domain. By virtue of genetic operators of natural genetics, a population representing the new solution set is generated. This process leads to the evolution of the population like natural evolution, in which the population in the later generation is more adaptable to the environment than the previous generation, and the optimal individual in the last generation is through decoding, which can be regarded as the approximate optimal solution of the problem.
• Genetic algorithms have three basic operations: Selection, Crossover and Mutation. • (1) Choice. The purpose of selection is to select good individuals from the current population so that they have a chance to father the next generation. According to the fitness value of each individual, according to certain rules or methods, some excellent individuals from the previous generation of population were selected and passed on to the next generation of population. Selection is based on the probability that highly adaptable individuals will contribute one or more offspring to the next generation. • (2) cross. A new generation of individuals can be obtained through crossover operation, and the new individuals combine the characteristics of the parent individuals. Individuals in the population are randomly paired, and for each individual, some chromosomes between them are exchanged with crossover probability. • (3) variation. To change the value of one or more loci to the other alleles with the probability of variation for each individual in the population. As in biology, variation is rare and provides opportunities for the creation of new individuals.
The basic steps of genetic algorithm:
1) Coding: GA first represents the solution data of the solution space as the genotype string structure data of the genetic space before searching, and the different combinations of these string structure data constitute the different points. 2) Generation of initial group: N initial string structure data are randomly generated, and each string structure data is called an individual, and N individuals constitute a group. GA starts to evolve with these N string structure data as the initial point. 3) Fitness evaluation: fitness indicates the advantages and disadvantages of an individual or solution. The definition of adaptive function is also different.
4) Selection: The purpose of selection is to select good individuals from the current population so that they have a chance to reproduce as fathers for the next generation. Genetic algorithm embodies this idea through the selection process, the principle of selection is that adaptable individuals have a high probability of contributing one or more offspring to the next generation. Selection embodies Darwinian principle of survival of the fittest. 5) Crossover: crossover operation is the most important genetic operation in genetic algorithm. A new generation of individuals can be obtained through crossover operation, and the new individuals combine the characteristics of their parents. Crossover embodies the idea of information exchange. 6) Variation: firstly, an individual is randomly selected in the population, and the value of a certain string in the string structure data is randomly changed with a certain probability for the selected individual. As in biology, the probability of variation in GA is very low, and the value is usually very small.
Genetic Algorithm Toolbox:
• MATLAB embedded genetic algorithm toolbox: Gadst • Sheffield Genetic algorithm toolbox: Gatbx • UnC Genetic algorithm toolbox: GAOT
Initializega function:
The ga functions:
Optimization of initial weights and thresholds of BP neural network by genetic algorithm:
Three, code,
%Name:genmain05.m clear CLF popsize=20; % population size chromlength=10; % String length (individual length) PC =0.6; % cross probability PM =0.001; Pop =initpop(popsize,chromlength); % randomly generate initial population for I = 1:20% 20 is the number of iterations [objValue]= calobjValue (pop); Fitvalue = calFitValue (objValue); % Calculate the fitness of each individual in the population [newPOP]=selection(POP, fitValue); Copy [newpop] % = crossover (pop, PC); Cross [newpop] = % mutation (pop, PC); Variation [bestindividual bestfit] % = best (pop, fitvalue); % The individual with the maximum fitness in the population and its fitness y(I)=-max(bestfit); n(i)=i; pop5=bestindividual; X (I) = decodechrom (pop5, 1, chromlength) * 10/1023; pop=newpop; end fplot('x^2-4*x+20',[0 10]) hold on plot(x,y,'r*') hold off [z index]=min(y); % to calculate the maximum value and its position, here is the maximum value in y vector, if the minimum value should be min, at the same time modify the fitness function x5=x(index)% to calculate the x value corresponding to the maximum value y=zCopy the code
5. References:
The book “MATLAB Neural Network 43 Case Analysis”