Adaptive Resonance Theory

Introduction:-Unsupervised learning is considered to be more psychologically plausible, possibly because humans tend to learn more about nature and life through their own experience, rather than by listening to a teacher. For example, imagine that there is nobody to tell us which instance to classify into which class, that is, what we should derive. The patterns in this case are not labeled with class-labels. There is evidence that unsupervised learning is plausible from a biological point of view too, that is, the way neurons change their connection weights, but we shall look at this paradigm here from an engineering point of view.

Unsupervised Learning in Neural Networks

Two principle learning rules are implemented in the contemporary unsupervised learning algorithms for neural networks:

 (1)Noncompetitive learning, that is, many neurons may be activated at a time; and

(2)Competitive learning, that is, the neurons compete and after that, only one is activated at one time, e.g. only one wins after the competition. This principle is also called "winner takes all."

The differential Hebbian learning law:-

The differential Hebbian law introduces the first derivatives of the activation signals to the Hebbian law.

1.Grossberg's competitive law (Grossberg 1982), expressed as:

2.The differential competitive learning law (Kosko 1990):

The differential competitive learning law introduces the first derivative of neuronal output values to thecompetitive learning law.

3.Adaptive vector quantization (Kohonen 1982, 1990); the learning law in a vector form is:

where c is a learning rate, x(t) is the input vector at moment t, and wj is the vector of the weights from the input neurons to the neuron j.

Unsupervised learning is also applicable when noise is present. For example, the random signal Hebbian law looks like:

wherenij is a noise introduced to the connection i-j.

In the major competitive learning models, the neural networks are organized in layers. There are excitatory connections between neurons from different layers, and inhibitory connections between neurons in one layer. The neurons in the output layer compete, each one inhibiting the others with its current activation level. The winner takes all in the end, that is, the neuron that gets the highest activation level is activated. Then a"reward" follows; this neuron strengthens its weights according to the input pattern and possibly suppresses the other neurons' weights.

Unsupervised learning is applicable for conceptualization that is, discovering and creating new concepts and categories from data examples. Unsupervised learning in neural networks can be used to learn structures, similarities, and relations.

Another major application for unsupervised learning networks is vector quantization. N-dimensional data vectors are represented as k-dimensional, when k < n. This is important for signal compression (images, speech, etc.), clustering, reducing dimensionality, etc.