Hierarchical And Modular Connectionist Systems

Introduction:-The flexibility of intelligence comes from the enormous number of different information-processing rules, modules, and the levels of operation of such modules. Hierarchical models are biologically and psychologically plausible.

Modular systems are systems consisting of several modules linked together for solving a given problem. Representing a system for solving a problem as a modular system may be justified for the following reasons:-

• The whole task may be represented as a collection of simpler subtasks, each being solved in one submodel of the whole system, so each module solves a different part of the whole problem, for example, one module is used for feature extraction, and another for pattern classification (which is usually the case in speech recognition systems).
   
• Different modules may provide alternative solutions, the final one being the best of them or a combination of them; different modules may imitate different experts on the same problem, the final solution being a weighted compromise between the outputs from the modules.

Different modules in a multimodular system may specialize during training to give a good approximation of the solution for a subspace of the whole problem space. They become local experts.

If each of the modules in a modular system is realized in a connectionist way, the system is called aconnectionist modular system. A connectionist modular system can be flat, if all the modules have the same priority for information processing; or hierarchical, if some modules have higher priority than other modules in the system. According to the types of neural networks used in a modular neural network, the latter can be classified as homogeneous, that is, all the neural networks are of the sametype; or heterogeneous, that is, different types of neural networks are used in one system; this is the casein the example below.

According to the type of connectivity between neural networks in modular neural network systems, the latter can be classified as fully connected, where every neural network is connected to every other one by at least one link, or partially connected, where only selected neural networks are linked together.

According to the way the neural networks in a modular system are used for solving a task, there are three types of operating modes in a system:

1. Sequential mode. Neural networks are used sequentially when different subtasks of the global task are performed; different neural networks are trained either separately or in conjunction with one another.

2. Parallel mode. All neural networks work in parallel, either on alternative solutions of one subtask, after which a final solution is worked out based on a given criterion,or on different subtasks if the problem allows a parallel solution.

3. Mixed mode. Both of the above modes are implemented in one modular system.

Modular and hierarchical multinetwork systems have been used for:-

• Robot control;
• Time-series forecasting.
• Classification of remotely sensed satellite images.
• Geometrical transformation and theorem-proving.