Boltzmann Machines

Introduction:-In order to overcome the local minima problem, a model based on using a variable called temperature was developed. The activation value of a neuron is calculated as a statistical probability:

where u_iis the net input to the ith neuron, calculated as in the Hopfield network. This model is called a Boltzmann machine after its resemblance to processes in thermodynamics. During the recall process the temperature T decreases, which is similar to the process of annealing in metallurgy. The physical meaning is that by changing the temperature T we "shake" the neural network "box" and if it has happened that it has rested at a wrong local minimum attractor, it will "jump out" and continue to "move'' until its eventual relaxation at a global minimum attractor (which represents a correct pattern).

Kosko's Bidirectional Associative Memories

A bidirectional associative memory (Kosko 1988B) is a heteroassociative memory. It can memorize pattern associations of the type (a(p), b(p)) where a(p) is an n-dimensional vector (pattern) and b(p) is its corresponding m-dimensional pattern (figure 4.32).

The neurons are of the same type as in the Hopfield network-either bivalent or binary inputs and outputs.A threshold activation function isused. The training process of a BAM is simply calculating the weights with the use of the formula

A recall in both directions is possible, that is, either a(p) or b(p) or both, complete or corrupted, can be used to recall corresponding associated patterns. A severe limitation of BAM is its small capacity, which is less than the minimum number between n and m. In order to overcome this limitation, BAM systems consisting of many BAM modules, BAM₁, BAM₂. . . BAM_k,have been developed (Simpson 1990). Every association (a,b) is tried to be memorized in a BAM module. If it is not possible to memorize it in the BAM_jmodule for example, then it is tried in the next one-the BAM_{j i}module, and so on, until all the pattern associations are memorized successfully. BAM networks have been applied to pattern recognition.