background

The reference of ARTIFICIAL intelligence has always been human beings. The lifelong, incremental ability to learn is one of the most important human abilities. If a robot can learn its environment and tasks incrementally like a human, it will make lifelong learning possible. The importance of incremental learning is reflected in two aspects: (1) In the actual perceptual data, the amount of data tends to increase gradually. Therefore, in the face of new data, the learning method should be able to make some changes to the trained system to learn the knowledge contained in the new data; (2) The time cost of modifying a trained system is usually lower than the cost of retraining a system. Incremental learning can be described as: whenever new data is added, it does not need to rebuild all the knowledge base, but only updates the changes caused by the new data on the basis of the original knowledge base. We find that incremental learning is more in line with the principles of human thought. There are many incremental learning frameworks, the core of which is to evaluate the similarity between new data and stored knowledge. Because this method determines how new knowledge is perceived and added to the knowledge base, it affects the growth of knowledge. The decision mechanism of new knowledge is the core component of incremental learning. The following is a brief introduction of several common incremental learning frameworks, including SOINN, a self-organizing incremental learning neural network, and EM-MDP, a Markov decision process of episodic memory.

Methods classification

1. SOINN

SOINN is a two-layer neural network based on competitive learning. SOINN’s incremental nature allows it to discover and learn new patterns as they emerge from data flows without affecting the results of previous learning. Therefore, SOINN can be applied to all kinds of unsupervised learning problems as a general learning algorithm. SOINN is a competitive neural network with a two-layer structure (excluding the input layer). It conducts online clustering and topological representation of input data in a self-organizing way, and its working process is shown in Figure 1.

  • The layer 1 network accepts the input of raw data and adaptively generates prototype neurons to represent the input data in an online manner. These nodes and the connections between them reflect the distribution of the raw data;

  • The second layer estimates the distance between classes and the distance within classes of the original data according to the results of the network of the first layer, and takes this as parameter. The neurons generated at the first layer are used as input to run SOINN algorithm again to stabilize the learning results.

As shown in Figure 2: When the input data has multiple clusters and there is noise, SOINN can still generate reliable neuron nodes to represent each cluster in the input data. Meanwhile, the topological structure of subgraph reflects the distribution of original data. Dynamic adjustment is the key for SOINN to realize self-organization and incremental learning. It enables the weight vector of neurons and the topology of the network to be dynamically adjusted with the arrival of the input mode to optimize the expression accuracy of the input data. In addition, by increasing the number of neurons at the right time, not only can the number of neurons be determined adaptively to meet certain quantitative error constraints, but also can adapt to previously unlearned input patterns without affecting the previous learning results. SOINN defines in-class node insertion and interclass node insertion operations to achieve both purposes, respectively.

  • The node insertion operation within the class is mainly to adaptively reduce the quantization error of neurons and approximate the distribution of original data as accurately as possible. Specifically,SOINN will record the cumulative quantization error of each neuron during its operation. After a fixed period of learning, it will find out the two nodes with the largest cumulative quantization error among all nodes, and then insert a new node between them to update their cumulative quantization error value by interpolation. Considering that not every insert operation is necessary, the number of nodes will increase as the algorithm progresses without some restriction. Therefore,SOINN will determine whether the insertion significantly reduces the quantization error after each node insertion operation within the class: if not, the insertion operation will be cancelled.

  • Interclass node insertion occurs when the newly entered data differs greatly from the previously learned data. SOINN sets a similarity threshold parameter for each neuron I TiT_iTi determines whether the new data sample is likely to belong to a new category: if the data point is quite different from the previously learned neuron, a new node is generated at the location of the data point to represent the possible pattern. As shown in Figure 3,ξ is the newly input data point. SOINN first finds the two most similar neurons S1s_1S1 and S2s_2S2, if D (s1,ξ)>T(s1) D (s1,ξ)>T(s1) or D (s2, factor) > T (s2) d (s_2, factor) > T_ (s_2) d (s2, factor) > T (s2), think of the differences between data is deduced. Where, d(∙)d(∙) D (∙) is the similarity measurement function (usually The Euclidean distance function). The newly generated node does not mean that it will eventually belong to a new cluster, but under the current similarity threshold, the input is quite different from the previously learned pattern. As more and more input patterns are learned, the similarity threshold and connections between neurons are constantly changing.

It can be seen that inter-class node insertion is the key for SOINN to realize incremental learning. The timing of node insertion has a great influence on the final result, and the similarity threshold parameter TTT of each node is the key to determine the insertion operation. If the TTT value is too small, each data is considered a new schema and a node is generated. If the TTT value is too large, the number of nodes will be too small, and the quantization error will increase, and the data distribution cannot be accurately reflected. Ideally, this parameter should be greater than the average intra-class distance and less than the average inter-class distance. SOINN takes an adaptive approach to this problem by constantly updating the value of tit_ti to accommodate changing input patterns. Assume that NNN is the set of all nodes and NiN_iNi is the set of neighbor nodes of node I. If NiN_iNi is not empty, i.e., there are other nodes connected to it by an edge, let: Ti = maxj ∈ Ni ∣ ∣ Wi – Wj ∣ ∣ T_i = max_ {j \ in N_i} | | W_i – W_j | | Ti = maxj ∈ Ni ∣ ∣ Wi – Wj ∣ ∣ otherwise, Ti = minj ∖ I ∈ N ∣ ∣ Wi – Wj ∣ ∣ T_i = min_ {j \ N \ setminus in {I}} | | W_i – W_j | | Ti = minj ∖ I ∈ N ∣ ∣ Wi – Wj ∣ ∣. As you can see, these two definitions are actually current estimates of the maximum intra-class distance and the minimum inter-class distance. The practical application shows that this dynamic adjustment method is effective.

2. EM-MDP

Em-mdp is exactly a complete set of artificial intelligence scheme (simplified version), which includes cognition of the situation, incremental learning, short and long term memory models. We focus on the incremental learning part of the framework. This framework is based on adaptive resonance theory (ART) and sparse distributed memory (SDM) to implement incremental learning of episodic memory sequences. SDM is a computer scientist named Penty. A method proposed by Kanerva in 1974 that can store any perception possessed by the mind into a finite memory mechanism. Thus, in the learning process, more than one state neuron can be activated at the same time, and each neuron can be regarded as a representative of a kind of similar perception. Compared with SOINN network which can only have one output node at most, this method has the advantage of good adaptability to environment. The construction of episodic memory network learning model is based on em-MDP model framework, which consists of perceptual input (O layer), perceptual similarity measure (U layer), state neuron (S layer) and output episodic memory (E layer). Its structural model is shown in Figure 3. Both U layer and S layer in the framework have the ability of incremental learning. The structure of U layer and S layer is shown in Figure 4. The number of nodes in U layer is equal to the dimension of input perception, and the output of each node is determined by three signals :(1) the perception input of the current environment
o c o_c ; (2) Control signal
C 1 C1 ; (3) Mapping perception of winning state neurons fed back by S-layer.FIG. 3 Output of node at U layer U root layer These three signals are calculated using the principle of “majority voting 2/3”. when
C 1 = 1 C1=1 , when the feedback mapping perception signal is 0, the node output of U layer is determined by the input perception, namely
u = o c u=o_c . When the feedback mapping perception signal is not 0,
C 1 = 0 C1=0 , node output of U layer depends on the comparison between input perception and feedback mapping perception. If the similarity measure is greater than the threshold, the perception vector learning should be adjusted; otherwise, a new perception should be added
o m + 1 = o c o_{m+1}=o_c . S layer has m nodes to represent M state neurons, and the state neuron space can be dynamically increased by adding new neuron nodes. There are weights between state neurons, which represent the connection relation of episodic memory.FIG. 4 Situational network accepts the perceptual input from the environment. By checking the matching degree between the current perceptual input and all the stored perceptual vectors, it can determine whether the new perception and related events already exist in the robot’s episodic memory. The similarity measure is examined against a preset activation threshold to determine how to treat the perception of new input. Each time the network receives a new perceptual input, it needs to go through a matching process. There are two cases of similarity measurement:

  • If the similarity exceeds the set threshold, the neighboring state set is selected as the representative state neuron set of the current input perception. Perception adjusts through learning so that it can obtain greater similarity when it encounters a perception that is close to the current input perception. For non-adjacent state sets, the perception vector does not make any adjustment. In effect, the mapped perceptions in episodic memory are recoded to stabilize familiar events that have already been learned.

  • If the similarity does not exceed the set threshold, a state neuron representing the new input perception should be added to the S layer and the mapping perception of the current perception as the new state should be stored to participate in the subsequent matching process. At the same time, the weights connected with the state neurons are established to store this kind of perception and participate in the subsequent matching process. In effect, it creates new expressive codes for unfamiliar events.

conclusion

By referring to relevant literature, SOINN is relatively widely used. Some are used for incremental road map construction of the environment, and some are used for cognitive map construction with processed visual information as feature input. However, em-MDP junction has not effectively separated the incremental learning part of the frame, so the structure is not very clear compared with SOINN proposed as a separate frame. The core component of incremental learning is the processing of old and new knowledge, and the “distance” between new input and old knowledge needs to be measured to determine whether an input is new knowledge. Filtering noise in knowledge network is also an essential part of maintaining an effective knowledge network. ** Finally, it is worth noting that SOINN’s evaluation threshold is automatically and dynamically adjusted, whereas the threshold in EM-MDP is a predetermined fixed value. ** Of course, efficient auto-tuning of parameters is something everyone likes.

END