4 min read

Introduction:-A neural network is a massively parallel distributed processor made up of simple processing units, which has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects: 1. Knowledge is acquired by the network from its environment through a learning process. 2. Interneuron connection strengths, known as synaptic weights, are used to store the acquired knowledge. Neural networks are also referred to in literature as neurocomputers, connectionist networks. Parallel distributed processors, etc.

3 min read

Introduction:-The manner in which the neurons of a neural network are structured is intimately linked with the learning algorithm used to train the network. We may therefore speak of learning algorithms (rules) used in the design of neural networks as being structured. In general, we may identify three fundamentally different classes of network architectures:-

6 min read

Introduction:-The history of neural networks begins in the early 1940’s and thus nearly simultaneously with the history of programmable electronic computers. The youth of this field of research, as with the field of computer science itself, can be easily recognized due to the fact that many of the cited persons are still with us. The beginning :-As soon as 1943 Warren McCullochand Walter Pitts introduced modelsof neurological networks, recreatedthreshold switches based on neuronsand showed that even simplenetworks of this kind are able tocalculate nearly any logic or arithmeticfunction. Further precursors ("electronic brains")were developed, among others supported by KonradZuse, who was tired of calculating ballistic trajectories by hand. 1947:Walter Pitts and Warren Mc-Culloch indicated a practical field of application (which was not mentionedin their work from 1943),namely the recognition of spacial patterns by neural networks .

3 min read

Introduction:-The goal of artificial intelligence (AI) is the development of paradigms or algorithms that require machines to perform cognitive tasks, at which humans are currently better. An AI system must be capable of doing three things: (1) store knowledge, (2) apply the knowledge stored to solve problems, and (3) acquire new knowledge through experience. An AI system has three key components: representation, reasoning, and learning

3 min read

Introduction:-A neuron is an information-processing unit that is fundamental to the operation of a neural network. The model of a neuron, which forms the basis for designing neural networks. Here we identify three basic elements of the neuronal model:- 1. A set of synapses or connecting links, each of which is characterized by a weight or strength of its own. Specifically, a signal Xj a t the input of synapse j connected to neuron k is multiplied by the synaptic weight wkj" It is important to make a note of the manner in which the subscripts of the synap tic weight wkj are written. The first subscript refers to the neuron in ques tion and the second subscrip t refers to the input end of the synapse to which the weight refers. Unlike a synapse in the brain, the synaptic weight of an artificial neuron may lie in a range that includes negative as well as positive values.

3 min read

Introduction:-A typical example of a heuristic method for problem solving is the genetic approach used in what is known as genetic algorithms. Genetic algorithms solve complex combinatorial and organizational problems with many variants, by employing analogy with nature's evolution. Genetic algorithms were introduced by John Holland (1975) and further developed by him and other researchers. Nature’s diversity of species is tremendous. How does mankind evolve into the enormous variety of variants—in other words, how does nature solve the optimization problem of perfecting mankind? One answer to this question may be found in Charles Darwin's theory of evolution. The most important terms used in the genetic algorithms are analogous to the terms used to explain the evolutionary processes. They are: Genetic algorithms are usually illustrated by game problems. Such is a version of the "mastermind" game, in which one of two players thinks up a number (e.g., 001010) and the other has to find it out with a minimal number of questions. Each question is a hypothesis (solution) to which the first player replies With another number indicating the number of correctly guessed figures. This number is the criterion for the selection of the most promising or prospective variant which will take the second player to eventual success. If there is no improvement after a certain number of steps, this is a hint that a change should be introduced. Such change is called mutation. In this game success is achieved after 16 questions, which is four times faster than checking all the possible combinations, as there are 2⁶ = 64 possible variants. There is no need for mutation in this example. If it were needed, it could be introduced by changing a bit (a gene) by random selection. Mutation would have been necessary if, for example, there was 0 in the third bit of all three initial individuals, because no matter how the most prospective individuals are combined, by copying a precise part of their code we can never change this bit into 1.Mutation takes evolution out of a "dead end."

3 min read

Introduction: -The human nervous system may be viewed as a three-stage system, as depicted in the block diagram. Central to the system is the brain, represented by the neural (nerve) net, which continually receives information, perceives it, and makes appropriate decisions. Those pointing from left to right indicate the forward transmission of information-bearing signals through the system. The arrows pointing from right to left signify the presence of feedback in the system. The receptors convert stimuli from the human body or the external environment into electrical impulses that convey information to the neural net (brain).The effectors convert electrical impulses generated by the neural net into discernible responses as system outputs. Synapses are elementary structural and functional units that mediate the interactions between neurons. The most common kind of synapse is a chemical synapse, which operates as follows. A presynaptic process liberates a transmitter substance that diffuses across the synaptic junction between neurons and then acts on a postsynaptic process. Thus a synapse converts a presynaptic electrical signal into a chemical signal and then back into postsynaptic electrical signal. In traditional descriptions of neural organization, it is assumed that a synapse is a simple connection that can impose excitation or inhibition, but not both on the receptive neuron. Axons, the transmission lines, and dendrites, the receptive zones, constitute two types of cell filaments that are distinguished on morphological grounds; an axon has a smoother surface, fewer branches, and greater length, whereas a dendrite has an irregular surface and more branches.

2 min read

Introduction:-The property that is of primary significance for a neural network is the ability of the network to learn from its environment and to improve its performance through learning. A neural network learns about its environment through an interactive process of adjustments applied to its synaptic weights and bias levels. The network becomes more knowledgeable about its environment after each iteration of the learning process. There are too many activities associated with the notion of "learning" to justify defining it in a precise manner. Moreover, the process of learning is a matter of viewpoint which makes it all the more difficult to agree on a precise definition of the term. We define learning in the context of neural networks as: Learning is a process by which the free parameters of a neural network are adapted through a process of stimulation by the environment in which the network is embedded. The type of learning is determined by the manner in which the parameter changes take place. This definition of the learning process implies the following sequence of events:

3 min read

Introduction:- The presence of nonlinearity in the model of a neuron limits the scope of their application to neural networks . Nevertheless, signal-flow graphs do provide a neat method for the portrayal of the flow of signals in a neural network. A signal-flow graph is a network of directed linksthat are interconnected at certain points called nodes. A typical node j has an associated node signal x_j" A typical directed link originates at node j and terminates on node k; it has an associated transfer function or transmittance that specifies the manner in which the signal Y_kat node k depends on the signal X_j at node j.The flow of signals in the various parts of the graph is dictated by three basic rules: Rule 1. A signal flows along a link only in the direction defined by the arrow on the link.

3 min read

Introduction:-Knowledge refers to stored information or models used by a person or machine to interpret, predict, and appropriately respond to the outside world.The primary characteristics of knowledge representation are twofold: (1) what information is actually made explicit; and (2) how the information is physically encoded for subsequent use. By the very nature of it, therefore, knowledge representation is goal directed. In real-world applications of "intelligent" machines, it can be said that a good solution depends on a good representation of knowledge. So it is with neural networks that represent a special class of intelligent machines. Typically, however, the possible forms of representation from the inputs to internal network parameters are highly diverse, which tends to make the development of a satisfactory solution by means of a neural network a real design challenge. A major task for a neural network is to learn a model of the world (environment) in which it is embedded and to maintain the model sufficiently consistent with the real world so as to achieve the specified goals of the application of interest. Knowledge of the world consists of two kinds of information:

3 min read

Introduction: -After successful learning it is particularly interesting whether the network has only memorized – i.e. whether it can use our training samples to quite exactly produce the right output but to provide wrong answers for all other problems of the same class. Suppose that we want the network to train a mapping and therefore use the training samples from fig. 4.1: Then there could be a chance that, finally, the network mwill exactly mark the colored areas around the training samples with the output and otherwise will output 0. Thus, it has sufficient storage capacity to concentrate on the six trainingsamples with the output 1. This impliesan oversized network with too much freestorage capacity. On the other hand a network could have insufficient capacity this rough presentation of input data does not correspond to the good generalization performance we desire. Thus, we have to find the balance.

3 min read

Introduction:-Gradient descent procedures are generally used where we want to maximize or minimize n-dimensional functions. The gradient is a vector g that is defined for any differentiable point of a function, that points from this point exactly towards the steepest ascent and indicates the gradient in this direction by means of its norm |g|. Thus, the gradient is a generalization of the derivative for multidimensional functions. Accordingly, the negative gradient −g exactly points towards the steepest descent. The gradient operator is referred to as Nabla operator, the overall notation of the the gradient g of the point (x, y) of a two dimensional function f being g(x, y) =f(x, y). Let g be a gradient. Then g is a vector with ncomponents that is defined for any point of a (differential) dimensional function f(x1, x2. . . xn). The gradient operator notation is defined as

3 min read

Introduction:-Fuzzy systems are rule-based expert systems based on fuzzy rules and fuzzy inference. Fuzzy rules represent in a straightforward way "commonsense" knowledge and skills, or knowledge that is subjective, ambiguous, vague, or contradictory. This knowledge might have come from many different sources. Commonsense knowledge may have been acquired from long-term experience, from the experience of many people, over many years. There are many applications of fuzzy logic on the market now. These include control of automatic washing machines, automatic camera focusing, control of transmission systems in new models of cars, automatic landing systems for aircraft, automatic helicopter control, automatic air-conditioning systems, automatic control of cement kilns, automatic control of subways, fuzzy decision making, fuzzy databases,etc. Many other applications of fuzzy logic in areas like control, decision-making and forecasting, human-computer interaction, medicine, agriculture, environmental pollution, cooperative robots, and so forth are in the research laboratories and are expected to enter the market.

1 min read

Time is divided into discrete time steps: discrete time steps The concept of time.The current time (present time) is referred to as (t), the next time step as (t 1), (t)I the preceding one as (t − 1). All other time steps are referred to analogously. If in the following chapters several mathematical

2 min read

Introduction:-The neurons are grouped in the following layers: One input layer, n hidden processing layers and one outputlayer. In a feedforward network each neuron in one layer has only directed connections to the neurons of the next layer. Feedforward network:-The neuron layers of a feedforward network are clearly separated: One input layer, one output layer and one or more processing layers which are invisible from the outside (also called hidden layers). Connections are only permitted to neurons of the following layer.Some feedforward networks permit the called shortcut connections. Recurrent networks have influence on themselves:-Recurrenceis defined as the process of aneuron influencing itself by any means orby any connection. Recurrent networks donot always have explicitly defined input oroutput neurons.

1 min read

Introduction:-A bias neuron is a neuron whose output value is always 1 and which is represented by It is used to represent neuron biases as connection weights, which enables any weight training algorithm to train the biases at the same time. Then the threshold value of the neurons j₁, j₂. . . j_nis set to 0.The threshold values are implemented as connection weights and can directly be trained together with the connection weights, which considerably facilitates the learning process. In other words: Instead of including the threshold value in the activation function, it includes the propagation function OrThe threshold value is subtracted from the network input.

5 min read

Introduction:-A technical neural network consists of simple processing units, the neurons, and directed, weighted connections between those neurons.The strength of a connection (or the connecting weight) be reactions of the neurons to the input values depend on this activation state. Activation state indicates the extent of a neuron’s activation and is often shortly referred to as activation. Definition 1 (Activation state / activationin general). Let j be a neuron. The activation state aj, in short activation, is explicitly assigned to j, indicates the extent of the neuron’s activity and results from the activation function. Components of neural networks:-A technical neural network consists of simpleprocessing units, the neurons, anddirected, weighted connections between those neurons. Here, the strength of a connection (or the connecting weight) be- tween two neurons i and j is referred to as wi,j.

3 min read

Introduction: -For a neural network it is very important in which order the individual neurons receive and process the input and output the results. We distinguish two model classes: 1. Synchronous activation:-All neurons change their values synchronously,i.e. they simultaneously calculate network inputs, activation and output, and pass them on. Synchronous activation corresponds closest to its biological counterpart, but it is to be implemented in hardware – only useful on certain parallel computers and especially not for feedforward networks. This order of activation is the most generic and can be used with networks of arbitrary topology. All neurons of a network calculate network inputs at the same time by means of the propagation function, activation by means of the activation function and output by means of the output function. After that the activation cycle is complete.

2 min read

Introduction:-Expert systems are knowledge-based systems that contain expert knowledge. For example, an expert system for diagnosing car faults has a knowledge base containing rules for checking a car and finding faults in the same way an engineer would do it. An expert system is a program that can provide expertise for solving problems in a defined application area in the way the experts do. Expert systems have facilities for representing existing expert knowledge, accommodating existing databases, learning and accumulating knowledge during operation, learning new pieces of knowledge from existing databases, making logical inferences, making decisions and giving recommendations, communicating with users in a friendly way (often in a restricted natural language), and explaining their"behaviour" and decisions. The explanation feature often helps users to understand and trust the decisions made by an expert system. Learning in expert systems can be achieved by using machine-learning methods and artificial neural networks. Expert systems have been used successfully in almost every field of human activity, including engineering, science, medicine, agriculture, manufacturing, education and training, business and finance, and design. By using existing information technologies, expert systems for performing difficult and important tasks can be developed quickly, maintained cheaply, used effectively at many sites, improved easily, and refined during operation to accommodate new situations and facts.

2 min read

Introduction:-In 1949, Donald O. Hebb formulated the Hebbian rule which is the basis for most of the complicated learning rules. Original rule:-Hebbian rule"If neuron j receives an input from neuron i and if both neurons are strongly active at the same time, then increase the weight w_i,j. Mathematically speaking, the rule is:

3 min read

Introduction:-In case of supervised learning we assume a training set consisting of training patterns and the corresponding correct output values we want to at the output neurons after the training. While the network has not finished training,i.e. as long as it is generating wrong outputs, these output values are referred to as teaching input and that for each neuron individually. Thus, for a neuron j with the incorrect output oj ,tj is the teaching input, which means it is the correct or desired output for a training pattern p. A training pattern is an input vector p with the components p1, p2. . . pn whose desired output is known. By entering the training pattern into the network we receive an output that can be compared with the teaching input, which is the desired output. The set of training patterns is called P. It contains a finite number of ordered pairs (p, t) of training patterns with corresponding desired output.Training patterns are often simply called patterns, that is why they are referred to as p. The teaching input tj is the desired and correct value j should output after the input of a certain training pattern. Analogously to the vector p the teaching inputs t1, t2, . . . , tn of the neurons can also be combined into a vector t. t always refers to a specific training pattern p and is, as already mentioned, contained in the set P of the training patterns.

2 min read

Description:-The Single Layer Feed-forward Network consists of a single layer of weights, where the inputs are directly connected to the outputs, via a series of weights. The synaptic links carrying weights connect every input to every output , but not other way. This way it is considered a network of feed-forward type. The sum of the products of the weights and the inputs is calculated in each neuron node, and if the value is above some threshold (typically 0) the neuron fires and takes the activated value (typically 1); otherwise it takes the deactivated value (typically -1).

3 min read

Introduction:-The learning curve indicates the progress of the error, which can be determined in various ways. The motivation to create a learning curve is that such a curve can indicate whether the network is progressing or not. For this, the error should be normalized i.e. represent a distance measure between the correct and the current output of the network. For example, we can take the same pattern specific, squared error with a prefactor, which we are also going to use to derive the backpropagation of error (let be output neurons and other set of output neurons:- Specific error Errp is based on a single Errp training sample, which means it is generated online. Additionally, the root mean square (abbreviated:RMS) and the Euclideandistance are often used. The Euclidean distance (generalization other theorem of Pythagoras) is useful for lower dimensions where we can still visualize its usefulness.

3 min read

Introduction:-The most interesting characteristic of neural networks is their capability to familiarize with problems by means of training and, after sufficient training, to be able to solve unknown problems of the same class. This approach is referred to as generalization. Learning is a comprehensive term. A learning system changes itself in order to adapt to e.g. environmental changes. A neural network could learn from many things but, of course, there will always be the question of how to implement it. Theoretically, a neural network could learn by

3 min read

Introduction:-In an RBF network the output neurons only contain the identity as activation function and one weighted sum as propagation function. Thus, they do little more than adding all input values and returning the sum.Hidden neurons are also called RBF neurons (as well as the layer in which they are located is referred to as RBF layer). As propagation function, each hidden neuron calculates a norm that represents the distance between the input to the network and the so-called position of the neuron. This is inserted into a radial activation function which calculates and outputs the activation of the neuron. The center C_h of an RBF neuron h is the point in the input space where the RBF neuron is located .In general the closer the input vector is to the center vector of an RBF neuron, the higher is its activation. RBF neurons h have a propagation function f_prop that determines the distance between the center C_h of a neuron and the input vector y. This distance represents the network input. Then the network input is sent through a radial basis function f_act which returns the activation or the output of the neuron. RBF neurons are represented by the symbol

3 min read

Description:-An artificial neural network is a biologically inspired computational model that consists of processing elements and connections between them. The structure of an artificial neuron is defined by inputs, having weights bound to them; an inputfunction, which calculates the aggregated net input signal to a neuron coming from all its inputs; anactivation (signal) function, which calculates the activation level of a neuron as a function of itsaggregated input signal and (possibly) of its previous state. An output signal equal to the activation valueis emitted through the output (the axon) of the neuron. Neural networks are also called connectionist models owing to the main role of the connections. Theweights bound to them are a result of the training process and represent the "long-term memory" of themodel. The main characteristics of a neural network are:

2 min read

Introduction:-Here we are comparing multilayer perceptrons and RBF networks with respect to different aspects. Input dimension: We must be careful with RBF networks in high dimensional functional spaces since the network could very quickly require huge memory storage and computational effort. Here, a multilayer perceptron would causeless problems because its number of neurons does not grow exponentially with the input dimension. Center selection: However, selecting the centers c for RBF networks is still a major problem. Such problems do not occur with the MLP.

2 min read

Introduction:-Looking at a few exemplary problems which we can use later to test implemented networks and learning rules. 1.Boolean functions:-A popular example is the one that did not work in the nineteen-sixties: the XOR function. We need a hidden neuron layer,. Thus, we need at least two neurons in the inner layer. Let the activation function in all layers be the hyperbolic tangent.

1 min read

Description: – The different types of neurons are explained as follows:-

2 min read

Introduction:-The Elman networks have context neurons, too, but one layer of context neurons per information processing neuron layer. Thus, the outputs of each hidden neuron or output neuron are led into the associated context layer (again exactly one context neuron per neuron) and from there it is reentered into the complete neuron layer during the next time step (i.e. again a complete link on the way back). So the complete information processing part1 of the MLP exists a second time as a "context version" – which once again considerably increases dynamics and state variety. Compared with Jordan networks the Elman networks often have the advantage to act more purposeful since every layer can access its own context. An Elman network is an MLP with one context neuron per information processing neuron. The set of context neurons is called K. This means that there exists one context layer per information processingneuron layer with exactly the same numberof context neurons. Every neuron hasa weighted connection to exactly one contextneuron while the context layer is completely linked towards its original layer. Now it is interesting to take a look at thetraining of recurrent networks since, for instance, ordinary back propagation of error cannot work on recurrent networks.

3 min read

Introduction:-Perceptron, most of the timeis used to describe a feed forward networkwith shortcut connections. This network has a layer of scanner neurons with statically weighted connections; theweights of the layers are allowed to be changed. All neurons subordinate to theretina are pattern detectors. Here we initiallyuse a binary perceptronwith everyoutput neuron having exactly two possibleoutput values (e.g. {0, 1} or {−1, 1}).Thus, a binary threshold function is usedas activation function, depending on the threshold value of the output neuron. In a way, the binary activation functionrepresents a query which can alsobe negated by means of negative weights. The perceptron can thus be used to accomplishtrue logical information processing, but it is not theeasiest way to achieve Boolean logic. An input neuron is an identity neuron. Itexactly forwards the information received. Thus, it represents the identity function, which should be indicated by the symbol/. Therefore the input neuron is representedby the symbol

2 min read

Introduction:-The aim is to generate minima on the energy surface, so that at an input the network can converge to them. As with many other network paradigms, a set P of training patterns p ϵ {1,−1}^|K|, representing the minima of our energy surface. Here we do not look for the minima of an unknown error function but define minima on such a function. The purpose is that the network shall automatically take the closest minimum when the input is presented. The training of a Hopfield network is done by training each training pattern exactly once. Single Shot Learning where p_i and p_j are the states of the neurons i and j under p ϵ P:

3 min read

Introduction:-In growing RBF networks, the number |H| of RBF neurons is not constant. A certain number |H| of neurons as well as their centers c_h and widths σ_hare previously selected (e.g. by means of a clustering method) and then extended or reduced. Neurons are added to places with large error values:-After generating this initial configuration the vector of the weights G is analytically calculated. Then all specific errors Err_pconcerning the set P of the training samples are calculated and the maximum specific error is sought. The extension of the network is simple: We replace this maximum error with a new RBF neuron. Of course, we have to exercisecare in doing this: IF the σ are small,the neurons will only influence each otherif the distance between them is short. Butif the σare large, the already existing neurons are considerably influenced by thenew neuron because of the overlapping ofthe Gaussian bells.So it is obvious that we will adjust the alreadyexisting RBF neurons when addingthe new neuron.To put it simply, this adjustment is madeby moving the centers c of the other neuronsaway from the new neuron and reducingtheir width σ a bit. Then thecurrent output vector y of the network iscompared to the teaching input t and theweight vector G is improved by means oftraining. Subsequently, a new neuron canbe inserted if necessary. This method is particularly suited for function approximations.

3 min read

Introduction:-Quantizationis also referred to as discretization. We know the sequence of discretenumbersN = {1, 2, 3. . .}, which contains the natural numbers. Discrete means, that this sequence consists of separated elements that are not interconnected. The elements of our example areexactly such numbers, because the natural numbers do not include, for example, numbers between 1 and 2. On the other hand, the sequence of real numbers R, for instance, iscontinuous: It does not matter how close two selected numbers are, there will always be a number between them. Quantization means that a continuous space is divided into discrete sections: By deleting, for example, all decimal places of the real number 2.71828, it could be assigned to the natural number 2. Here it is obvious that any other number having a 2 in front of the comma would also be assigned to the natural number 2, i.e.2 would be some kind of representativefor all real numbers within the interval. It must be noted that a sequence can be irregularly quantized, too: For instance, the timeline for a week could be quantized into working days and weekend. A special case of quantization is digitization: In case of digitization we alwaystalk about regular quantization of a continuous space into a number system with respect to a certain basis. (Quantization). Separation of a continuous space into discrete sections.

1 min read

When a known pattern p is entered, exactly this known pattern is returned. Thus, a(p) = p, With a being the associative this also works with inputs that are close to a pattern:

2 min read

Description:-Analogous to the MLP we perform a gradient descent to find the suitable weights by means of the already well known delta rule. Here, back propagation is unnecessary since we only have to train one single weight layer, which requires less computing time. We know that the delta rule is

3 min read

Introduction:-The LVQ is a supervised learning procedure. A teaching input that tells the learning procedure whether the classification of the input pattern is right or wrong: In other words, we have to know in advance the number of classes to be represented or the number of codebook vectors. The procedure of learning:-Learning works according to a simple scheme. We have (since learning is supervised)a set P of |P| training samples. Additionally, we already know that classes are predefined; too, i.e. we also have a set of classes C. A codebook vector is clearly assigned to each class. Thus, we can say that the set of classes |C| contains many codebook vectors C1, C2…C|C|. This leads to the structure of the training samples: They are of the form (p, c) and therefore contain the training input vector p and its class affiliation c. For the class affiliation c ϵ{1, 2, . . . , |C|} Holds, which means that it clearly assigns the training sample to a class or a codebook vector.

2 min read

Introduction:-Generally, recurrent networks are networks that are capable of influencing themselves by means of recurrences, e.g. by including the network output in the following computation steps. There are many types of recurrent networks of nearly arbitrary form, and nearly all of them are referred to as recurrent neural networks.Recurrent multilayer perceptrons such a recurrent network is capable to compute more than the ordinary MLP: If the recurrent weights are set to 0, the recurrent network will be reduced to an ordinary MLP. Jordan networks A Jordan network is a multilayer perceptron with a set K of so-called context neurons k₁, k₂. . . k_|K|.There is one context neuron per output neuron.A context neuron just memorizes an output until it can be processed in the next time step. Therefore, there are weighted connections between each output neuron and one context neuron. The stored values are returned to the actual network by means of complete links between the context neurons and the input layer.

5 min read

Number of layers: Two or three may often do the job, but more are also used A network should have one layer of input neurons and one layer of output neurons, which results in at least two layers. During the examination of linear separability at least one hidden layer of neurons, if our problem is not linearly separable (which is, as we have seen, very likely).It is possible, to mathematically prove that this MLP with one hidden neuron layer is already capable of approximating arbitrary functions with any accuracy but it is necessary not only to discuss the representability of a problem by means of a perceptron but also the learnability. Representability means that a perceptron can, in principle, realize a mapping - but learnability means that we are also able to teach it. In this respect, experience shows that two hidden neuron layers (or three trainable weight layers) can be very useful to solve a problem, since many problems can be represented by a hidden layer but are very difficult to learn. One should keep in mind that any additional layer generates additional sub minima of the error function in which we can get stuck. All these things considered, a promising way is to try it with one hidden layer at first and if that fails, retry with two layers. Only if that fails, one should consider more layers. However, given the increasing calculation power of current computers, deep networks with a lot of layers are also used with success. The number of neurons has to be tested

3 min read

Introduction:-Linear separability is a powerful technique which is used to learn complicated concepts that are considerably more complicated than just hyperplane separation. Let f be the XOR function which expects two binary inputs and generates a binary output .Let us try to represent the XOR function by means of an SLP with two input neurons i₁, i₂ and one output neuron

2 min read

Introduction:-A single layer perceptron (SLP) is a perceptron having only one layer of variable weights and one layer of output neurons Ω. Certainly, the existence of several output neurons Ω ₁, Ω₂. . . Ω _n does not considerably change the concept of the perceptron .A perceptron with several output neurons can also be regarded as several different perceptron with the same input

2 min read

Introduction: -A perceptron with two or more trainable weight layers is called multilayer perceptron or MLP.It is more powerful than an SLP. A single layer perceptron can divide the input space by means of a hyper plane (in a two-dimensional input space by means of a straight line). A two stage perceptron can classify convex polygons by further processing these straight lines, e.g. in the form "recognize patterns lying above straight line 1, below straight line 2 and below straight line3. A multilayer perceptron represents an universal function approximator. Perceptrons with more than one layer of variably weighted connections are referred to as multilayer perceptrons (MLP).An n-layer or n-stage perceptron has thereby exactly n variable weight layers and n 1 neuron layers () with neuron layer 1 being the input layer.

1 min read

The same applies for the first factor according to the definition of δ: -----------(8) Now we insert:

3 min read

Introduction: -Resilient backpropagation is an extension to backpropagation of error.On the one hand, users of backpropagation can choose a bad learning rate. On the other hand, the further the weights are from the output layer, the slower backpropagation learns. For this reason, MartinRiedmiller et al. enhanced backpropagation and called their version resilient backpropagation. Learning rates: Backpropagation uses by default a learning rate which is selectedby the user, and applies to the entire network.It remains static untilit is manually changed. Rprop pursues acompletely different approach: there is no global learning rate. First, each weight w_i,jhas its own learning rateɳi,j, and second, these learning rates are not chosen by the user, but are automatically set by R_prop itself. Third, the weight changes are not static butare adapted for each time step of R_prop. Weight change: When using backpropagation, weights are changed proportionally to the gradient of the error function. At first glance, this is really intuitive. However, we incorporate every jagged feature of the error surface into the weight changes. It is at least questionable, whether this is always useful. Here, Rprop takes other way as well: the amount of weight change Δw_i,jsimply directly corresponds to the automatically adjusted learning rate ɳ_i,j. Thus the change in weight isnot proportional to the gradient,it isonly influenced by the sign of the gradient. Until now we still do not know how exactly the ɳ_i,j are adapted atrun time, but let me anticipate that the resulting process looks considerably less rugged than an error function.

4 min read

Introduction: -Hopfield network is another example of supervised learning. Hopfield networks are inspired by particles in a magnetic field:-The idea for the Hopfield networks originated from the behavior of particles in a magnetic field: Every particle "communicates"(by means of magnetic forces) with every other particle (completely linked) with each particle trying to reach an energetically favorable state (i.e. a minimum of the energy function). As for the neurons this state is known as activation. Thus, all particles or neurons rotate and there by encourage each other to continue this rotation. As a manner of speaking, our neural network is a cloud of particles Based on the fact that the particles automatically detect the minima of the energy function; Hopfield had the idea to use the “spin" of the particles to process information. In a Hopfield network, all neurons influence each other symmetrically: -Hopfield network consists of a set K of completely linked neurons with binary activation use two spins), with the weights being symmetric between the individual and without any neuron being directly connected to itself. Thus, the state of |K| neurons with two possible states ϵ{−1, 1} can be described by a string x ϵ {−1, 1}^|K|.

4 min read

Introduction:-An RBF network receives the input by means of the unweight connections. Then the input vector is sent through a norm so that the result is a scalar. This scalar is processed by a radial basis function. The output values of the different neurons n of the RBF layer or of the different Gaussianbells are added within the third layer basically, in relation to the whole inputspace, Gaussian bells are added here. Suppose that we have a second, a thirdand a fourth RBF neuron and thereforefour differently located centers. Each of these neurons now measures another distancefrom the input to its own centerand de facto provides different values, evenif the Gaussian bell is the same. Since these values are finally simply accumulatedin the output layer, one can easilysee that any surface can be shaped by dragging, compressing and removing Gaussianbells and subsequently accumulating them. Here, the parameters for the superposition of the Gaussian bells are in the weights of the connections between the RBF layer and the output layer.

2 min read

Introduction:-The 8-3-8 encoding problem is a classic among the multilayer perceptron test training problems. In our MLP we have an input layer with eight neurons i1, i2. . . i8, an output layer with eightneurons Ω1,Ω, . . . ,Ω8 and one hiddenlayer with three neurons. Thus, this networkrepresents a function .The training task is that an input of a value1 into the neuron i_j should lead to an outputof a value 1 from the neuron Ωj. The network with the 3hidden neurons represents some kind of binary encoding. Thus, our network is a machine in which the input is first encoded and afterwards decoded again. Analogously, we can train a 1024-10-1024 encoding problem. But is it possible to improve the efficiency of this procedure? Could there be, for example, a 1024-9-1024- or an 8-2-8-encoding network? Yes, even that is possible, since the network does not depend on binary encodings: Thus, an 8-2-8 network is sufficient for our problem. But the encoding of the networks far more difficult to understand and the training of the networks requires a lot more time. An 8-1-8 network, however, does not work, since the possibility that the output of one neuron is compensated by another one is essential, and if there is only one hidden neuron, there is certainly no compensatory neuron.

4 min read

Introduction: -Jordan network without a hidden neuron layer for our training attempts so that the output neurons can directly provide input. This approach is a strong simplification because generally more complicated networks are used. But this does not change the learning principle. Unfolding in time:-The back propagation of error, which back propagates the delta values. So, in case of recurrent networks the delta values would back propagate cyclically through the network again and again, which makes the training more difficult. On the one hand we cannot know which of the many generated delta values for aweigh should be selected for training, i.e. which values are useful. On the other hand we cannot definitely know when learning should be stopped. The advantage of recurrent networks is great state dynamics within the network; the disadvantage of recurrent networks is that these dynamics are also granted to the training and therefore make it difficult. One learning approach would be the attempt to unfold the temporal states of the network: Recursions are deleted by putting a similar network above the context neurons, i.e. the context neurons are, as a manner of speaking, the output neurons of the attached network. More generally spoken, we have to backtrack the recurrences and place "‘earlier"’ instances of neurons in the network thus creating a larger, but forward-oriented network without recurrences. This enables training a recurrent network with any training strategy developed for non-recurrent ones. Here the input is entered as teaching input into every "copy" of the input neurons. This can be done for a discrete number of time steps. These training paradigms are calledunfolding in time.

1 min read

In continuous Hopfield networks, the activation is no longer calculated by the binary threshold function but by the Fermi function with temperature parameters here, the network is stable for symmetric weight matrices with zeros on the diagonal, too. Hopfield also stated that continuous Hopfield networks can be applied to find acceptable solutions for the NP-hard travelling salesman problem. According to some verification trials this statement can’t be kept up any more. But today there are faster algorithms for handling this problem and therefore the Hopfield network is no longer used here.

3 min read

Introduction:-It is obvious that the approximation accuracy of RBF networks can be increased by adapting the widths and positions of the Gaussian bells in the input space to the problem that needs to be approximated. There are several methods to deal with the centers c and the widths of the Gaussian bells: Fixed selection: The centers and widths can be selected in a fixed manner and regardless of the training . Conditional, fixed selection: Again, centers and widths are selected fixedly, but we have previous knowledge about the functions to be approximated and comply with it.

3 min read

Description:-Hetroassociation is the variant for a dynamic energy surface: Here, the appearance of the energy surface depends on the current state and we receive a heteroassociator instead of an autoassociator. For a heteroassociator a(p ϵ) = p Is no longer true, but rather

4 min read

Introduction:-Two possible ways of representing knowledge in neural networks are (1) local representation, where every neuron represents one concept or one variable, etc., and (2) distributed representation, where a concept or a value for a variable is represented by the collective activation of a group of neurons. Each way has alternatives. For example, we can represent an output variable that has the context of "percentage" as one neuron (activation values from 0 to 1, a value of 0.01 corresponding to 1%, etc.).Another possible neuronal encoding may use 10 neurons for a "thermometer" encoding (the first neuron represents percentage from 0% to 10%, the second, from 10% to 20%, etc., 100% being represented by the activation of all the output neurons).

3 min read

Introduction:-The problem of distinguishing input vectors that "fall" in the bordering area between two output neurons and the necessity for using known labels of the neurons for better topological mapping of the input vectors into the output space have led to the development of learning vector quantization (LVQ) algorithms LVQ1, LVQ2, and LVQ3 (Kohonen 1990). LVQ1 AlgorithmIn LVQ1 several codebook vectors are assigned to each class, and each is labeled with the corresponding class symbol (label). Initial values of the codebook vectors are learned by the SOM algorithm. The output neurons are then labeled according to the classes. Then a correction of the weights with the use of the known labels is performed by applying the following formulas to update the weights vectors:

3 min read

Introduction: -Diagnostic systems can be realized in neural networks in which the symptoms are represented by input nodes and the set of diseases by output nodes. The input nodes take input data, which, depending on the type of the symptoms, can be: On the output nodes, the network can produce, depending on the context of the disease, different values to indicate a diagnosis:

2 min read

Introduction:-The Hamming network performs the task of pattern association, or classification, based on measuring the Hamming distance. The network is apattern associate of n-dimensional binary vectors labeled with m-class labels (m-class patterns). A new vector x' is associated with class pattern xi with the minimum Hamming distance, a Hamming distance being the number of different bits in the two patterns. The Hamming network is similar to the Hopfield network, but it has two layers of connections. The first layer contains connections between all the n inputs, and all the m outputs. The second layer is a fully connected recurrent network with m neurons (similar to the Hopfield network). All nodes use linear threshold activation functions. The first layer of connections is finding the difference between the number n and the Hamming distance between the input pattern x' and each of the m-class patterns used to calculate the connection weights in advance.

3 min read

Neural networks have proved to be efficient at solving some optimization problems. Optimization problems are about finding a minimum of a goal function. A neural network, for example, the Hopfield network, can be represented by an energy function which tends to its minimum during convergence. Now, the problem of using a neural network for solving an optimization task is how to represent the optimization goal function as an energy function of a network. As an example, a solution to the Traveling Salesman Problem (TSP) is given here. One connectionist solution of the TSP was suggested by Hopfield and Tank (1985). They represented the problem as an N2-dimensional Hopfield network. The rows represent N towns and the columns the position of the town in the route.

2 min read

Speech processing, and speech recognition in particular, is one challenging problem where neural networks have a lot to offer. Neural networks have been widely used for solving many tasks and subtasks of speech processing. Speech Synthesis: NETtalk One of the first neural network applications of speech processing was NETtalk, developed by Sejnowskiand Rosenberg (1987). This is an MLP trained with a backpropagation algorithm to pronounce an English text. The input of the network consists of seven input modules, one for a letter, each consisting of 29 input nodes (26 for the letters in the alphabet plus three for punctuation), 80 hidden nodes, and 26 output nodes, encoding phonemes.

2 min read

Introduction: -This approach is based on building different layers of neural networks that perform different consecutivetasks in the process of finding the best next move. For example, layer 1, an input layer, receives datafrom the board; layer 2, an intermediate layer of receptor neurons, recognizes patterns, features, and situations on the game board; and layer 3, an output layer of effector neurons, selects the best next move. The patterns on the board are detected, and appropriate neurons in the intermediate layer are activated accordingly. This requires prewired connections. The weights are preset and no training is required. A connectionist model of this type was presented in Rumelhart et al. (1986a). Having established the links between the neurons from different layers in the network, the system will "play," performing the algorithm. The representation of game strategies in the suggested connectionist model is carried into effect in the following way. The operator, creating or perfecting the network, defines the most important configurations (characteristics) generated in the game process. The presence or absence of one such characteristic determines the need for performing, or refraining from performing, certain moves. For every major characteristic of the game a neural element is constructed in the second layer of the network, sensitive to the appearance (or absence) of the respective configuration characteristic on the game board.

3 min read

The traditional approach to building knowledge-based systems is to use rules articulated by experts and to build an expert system. If only this approach is used, some problems arise: One alternative to the standard approach is to build automatic learning machines which learn explicit rules from data, if enough data are available. Rules extracted from data can be subsequently used for the purpose of reasoning, explanation, understanding problems, and trying alternative techniques for solving problems, for example, fuzzy inference machines and AI rule-based techniques. Neural networks are appropriate candidates for the task of knowledge acquisition from data because of the following:

3 min read

Introduction:-Connectionist expert systems are expert systems which have their knowledge base represented in a connectionist structure. The neural network properties of learning and generalization, adaptability, robustness, associative storage of information, massive parallelism, and other characteristics make them a very powerful paradigm for building knowledge based expert systems. A general architecture of a connectionist expert system (CES):-It is distinguished from the symbolic AI and fuzzy expert systems in the way problem knowledge is used. Here problem knowledge need not only be a set of heuristic rules but can also be given as a set of past experience examples, for example, case studies of how a physician has previously treated patients. A CES may contain the following modules: A connectionist knowledge-based module, represented as connectionist architecture. Four different approaches to its building are:-

4 min read

The REFuNN algorithm first published in Kasabov (1993b) and further refined in Kasabov (1995c), is a simple connectionist method for extracting weighted fuzzy rules and simple fuzzy rules. This method is based on training MLP architecture with fuzzified data. The REFuNN algorithm, outlined below, is based on the following principles: 1. Simple operations are used and a low computational cost is achieved.

2 min read

A "yes"/"no" whole-word recognition MLP is shown below. It has 26 input nodes for the 26 Mel-scale coefficients, which in this case are averaged over several segments, to cover the whole words. The two words' speech signals are transformed into 26-element vectors. Samples of" yes"/ "no" spoken words are collected from different speakers-male, female, various age groups, etc. The network is trained and then tested on newly pronounced "yes''/"no" words. If the test words are pronounced by one of the speakers whose spoken words were used for training, and the system recognizes them, then the system is said to be of a multiple-speaker type. If the system recognizes a new speaker, then the system is called speaker independent. Phoneme recognition is a difficult problem because of the variation in the pronunciation of phonemes, the time alignment problem (the phonemes are not pronounced in and because of what is called the coarticulation effect, that is, the frequency characteristics of an allophonic realization of the same phoneme may differ depending on the context of the phoneme in different spoken.

2 min read

The decision-making tasks are usually characterized by a tremendous variety of data and available knowledge, different requirements and constraints, and a dynamically changing environment, which make the use of neural networks quite appropriate. For example, a decision-making system might require taking into account exact data, fuzzy data, and fuzzy rules and membership functions. Decision-making may require good explanatory facilities, for which purpose rules extraction from data might be a useful technique. SOM can well be used for visualization of a decision and placing it in regions of similar cases. The applicants Cl, C2, and C3 are "placed" by the network in the areas of "approve, “disapprove," and "not known," respectively.

2 min read

Time-delay neural networks (TDNNs) capture temporal speech information. Time-delay input frames allow the weights in the initial layers to account for time variations in speech. Like the MLP, they are feed forward networks. Figure shows a general TDNN scheme for the phoneme classification task. TDNNs are good at recognition of sub word units from continuous speech. For the recognition of continuous speech, a higher-level parsing framework is required in addition. It was discovered that a single monolithic TDNN was impractical for recognizing all the phonemes in Japanese. It was then suggested that a modular TDNN be used. Recurrent Neural Networks for Speech Recognition

3 min read

The feature vectors obtained after signal processing e.g. the Mel-scale cepstrum coefficients vectors, can be used as training examples for training a SOM. Vectors representing allophonic realizations of the same phoneme are taken from different windows (frames) over one signal sample and from different signals, that is, different realizations of this phoneme. After enough training, every phoneme is represented on the SOM by the activation of some output nodes. One node fires when an input vector representing a segment of the allophonic realization of this phoneme is fed into the network. The outputs, which react to the same phoneme pronounced differently, are positioned closely. The outputs that react to similar phonemes are positioned in proximity on the map. This is due to the ability of the SOM to activate topologically close output neurons when similar input vectors are presented. This approach has been used and phonemic maps have been created for Finnish, Japanese, English, Bulgarian (and other languages. Figure A is a two-dimensional drawing of the coordinates of the neurons in a SOM that was labeled with eight phonemes in English selected from digits pronounced by a small group of speakers. Figure B shows how allophonic segments of phonemes were segmented for training the SOM. It is clear from this drawing that the phonemes are well distinguished, and there are areas where the network cannot produce any meaningful classification. Instead of having a large, and therefore slow-to-process single SOM, hierarchical models of SOMs can be used. Every SOM at the second level is activated when a corresponding neuron from the first level becomes active. The asymptotic computational complexity of the recognition of the two-level hierarchical model is 0(2 × n × m) where n is the number of inputs and m is the size of a single SOM. This is much less than the computational complexity O(n · m²) of a single SOM with a size of m2 (m = 16). For a general r-level hierarchical model, the complexity is O(r · n · m). The first-level SOM is trained to recognize four classes of phonemes: a pause, a vocalized phoneme, a no vocalized phoneme, and a fricative segment.

2 min read

1. Memory capacity. The number m of training patterns should be about the number of neurons n, or less (according to Hopfield, it should be less than 0.15 n; according to some new results, m £ 0.5n/log n). This means that the memorizing capacity of a Hopfield network is severely limited. Catastrophic “forgetting" may occur if we try to memorize more patterns than the network is supposed to handle. 2. Discrepancy limitation. The new pattern to be recognized as one of the training patterns should not differ from any training pattern by more than about 25%. 3. Orthogonality between patterns. The more orthogonal (dissimilar) the training patterns, the better the recognition.

1 min read

Introduction:-Counter propagation network was developed by Hecht-Nielsen in 1987.It is a hybrid between two well-known connectionist models—competitive learning (performed by a first layer of connections) and supervised learning (performed by a following layer of connections).In this respect they are similar to the RBF networks. The idea behind this architecture is that competitive learning, which is fast, will cluster the input space, assigning one intermediate node to each group. After that, supervised learning is performed, but only on a single layer. The intermediate layer is of the "winner-takes-all” type. The major advantages of such a network are that

1 min read

Introduction:-There is a tremendous variety of neural network models. Some are variants or improvements of the types already presented-MLP, ART, SOM, associative memories. Other connectionist models emerge and evolve with the development of knowledge about the way real neurons work. Neural networkmodels may differ in the following points: 1.The type of neurons used and type of calculations.

2 min read

Monitoring problems may be solved by using a simple neural network model or a hierarchical and modular neural network model. In the former case, the input nodes take data for the current situation. The output activation signals represent values for the monitored parameters. The network has to be trained with enough examples of the monitoring process. Such a task could be the monitoring of a car for which a simple neural network can be trained subject to enough data available for the input parameters the cooling temperature, status of the gauge, status of the brakes and for the output variable level of risk for the driver. In the latter case, layers of neural networks may need to be used, each performing a different task. For example, bedside critical care monitoring of patients in intensive care may require a complex neural network to recognize dynamic states of several events occurring over a short or long period of time and predicting a dangerous state, such as heart failure, etc. Snapshots are not enough in this case. They contain local information only. It is necessary to observe two or more successive snapshots in their order of occurrence. In this case, two connectionist modules are required: (1) Short-term memory, to represent temporal information

3 min read

Introduction:-Representing space and time is an important issue in knowledge engineering. Space can be represented in a neural network by:- Representing time in a neural network can be achieved by: Different connectionist models for representing "time" and the way they encode time are explained below

2 min read

Introduction:-When problem knowledge is represented by a set of past data, a neural network can be used and trained with the data. The network should then be treated as a connectionist knowledge base. The data used can be exact or fuzzy or a combination of both. Using exact training data an MLP neural network, with 3 input neurons, 11 intermediate nodes, and 1 output neuron, to be trained with the use of the back propagation algorithm, is shown in figure 5.4. The number of hidden nodes is assigned to be equal to the number of supposeddifferent groups of applicants. Such a relatively large number of hidden nodes would facilitate effective training on a very large data set. Testing the validity of the neural network after training can be done with an use of a test set.

2 min read

Clusters in the input-output problem space represent "patches" of data, which can be represented asrules. Neurons in competitive learning neural networks learn to represent centers of clusters. A weight vector w_jmay be viewed as a geometrical center of a cluster of data. These characteristic of competitive learning neural networks can be used for the purpose of rules extraction, and finding fuzzy rules in particular. Figureshows a two-dimensional input output space for learning rules of the form of IF X is A, THEN Y is B, where A and B can be defined either as intervals for extracting interval rules or as fuzzylabels for extracting fuzzy rules. This clustering can be achieved in a competitive learning neuralnetwork.

1 min read

Description:-Pattern association is the process of memorizing input-output patterns in a heteroassociative network architecture, or input patterns only in anautoassociative network, in order to recall the patterns when a new input pattern is presented. It is not required that the new input pattern be exactly the same as one that is memorized. It can be different, but similar to it. Three exemplar patterns are shown in figure 4.27. After memorizing them in a system, a new pattern is presented, a corrupted variant of the pattern 3. An associative memory system should associate this pattern with one of the memorized patterns. This is a task for an autoassociative network architecture. Autoassociative neuronal architectures can be realized either by feedforward, or feedback networks.Hopfield networks and Boltzman machines are two examples of autoassociative networks. Other types of pattern associators are the heteroassociative network architectures. One of them is the bidirectional associative memory (BAM) (Kosko 1988).

2 min read

Three main tasks in image processing, where connectionist methods can be successfully applied are:- (1) Image compression and restoration, (2) Feature extraction, and

2 min read

The Paradigm:- The generic characteristics of neural networksmake possible their use for: Neural network models provide massive parallelism, robustness, and approximate reasoning, which areimportant for dealing with uncertain, inexact, and ambiguous data, with ill-defined problems and sparsedata sets. All the above-mentioned points are discussed here. The problem-solving process, when usingneural networks, comprises two phase:-

3 min read

Introduction:-The flexibility of intelligence comes from the enormous number of different information-processing rules, modules, and the levels of operation of such modules. Hierarchical models are biologically and psychologically plausible. Modular systems are systems consisting of several modules linked together for solving a given problem. Representing a system for solving a problem as a modular system may be justified for the following reasons:- Different modules in a multimodular system may specialize during training to give a good approximation of the solution for a subspace of the whole problem space. They become local experts.

3 min read

Introduction:-Both supervised and unsupervised learning in neural networks have been used for pattern recognition and classification. The steps to follow when creating a connectionist model for solving the problem are to define (1) The set of features to be used as input variables, and

2 min read

Introduction:-The strong development of digital electronic circuits and boolean algebra were influential in thedevelopment of connectionist models based on the idea of random-access memory (RAM). RAM is a device in which binary information can be stored and retrieved, each of the information elements having its own directly addressable space. The addresses are binary vectors. If n address lines are used, 2" addressable elements can be stored and retrieved, each consisting of m bits. Figure shows a RAM-based structure and its representation as a neural network, where n-inputs, one-output RAM elements are used. Using such a simple structure, different models have been developed and put into practice. They differ in their training algorithms, and application-oriented characteristics. Using the above idea, so-called weightless neural networks were developed by Igor Aleksander (1989).The connections between neurons do not have real weights; they are binary, {0,1}, that is, a connection either exists or does not exist.

2 min read

Introduction:-Neural networks are universal function approximators. They are "model-free estimators" in the sense that the type of function is not required to be known in order for the function to be approximated. One difficulty, though, is how to choose the best neural network architecture, that is, theneural network model with the smallest approximation error. When a multilayer perceptrons (MLPs) are used, this is the problem of finding the optimal number of hidden nodes. In addition to the heuristics given there, some other techniques are applicable, such as:- Growing neural networks. Training starts with a small number of hidden nodes and, subject to the error calculated, the number of the hidden nodes may increase during the training procedure.

3 min read

Introduction:-Unsupervised learning is considered to be more psychologically plausible, possibly because humans tend to learn more about nature and life through their own experience, rather than by listening to a teacher. For example, imagine that there is nobody to tell us which instance to classify into which class, that is, what we should derive. The patterns in this case are not labeled with class-labels. There is evidence that unsupervised learning is plausible from a biological point of view too, that is, the way neurons change their connection weights, but we shall look at this paradigm here from an engineering point of view. Unsupervised Learning in Neural Networks Two principle learning rules are implemented in the contemporary unsupervised learning algorithms for neural networks:

3 min read

Introduction:-One of the most used neural network models, used mainly for vector quantization and data analysis but also applicable to almost all the tasks where neural networks have been tried successfully, is the self organizing map introduced and developed by Teuvo Kohonen (1982-1993). The Philosophy of the Kohonen Network Self-organizing topological or feature maps became popular because they tend to be very powerful at solving problems based on vector quantization. A SOM consists of two layers, an input layer and an output layer, called a feature map, which represent the output vectors of the output .The weights of the connections of an output neuron j to all the n input neurons, form a vector w_j in an n dimensional space.

2 min read

Introduction:-The neural network models presented so far use variants of McCulloch and Pitt's neuron to build anetwork. New types of neurons have been introduced which use fuzzy membership functions asactivation functions or as functions attached to their connections. One of them is the so-called fuzzyneuron. A fuzzy neuron has the following features, which distinguish it from the ordinary types of neurons:- Excitatory connections are represented by MIN operation and inhibitory connections by fuzzy logic complements followed by MIN operation.

2 min read

Introduction:-In order to overcome the local minima problem, a model based on using a variable called temperature was developed. The activation value of a neuron is calculated as a statistical probability: where u_iis the net input to the ith neuron, calculated as in the Hopfield network. This model is called a Boltzmann machine after its resemblance to processes in thermodynamics. During the recall process the temperature T decreases, which is similar to the process of annealing in metallurgy. The physical meaning is that by changing the temperature T we "shake" the neural network "box" and if it has happened that it has rested at a wrong local minimum attractor, it will "jump out" and continue to "move'' until its eventual relaxation at a global minimum attractor (which represents a correct pattern).

2 min read

Destructive learning is a technique that destroys the initial neural network architecture for the purpose of better learning. One method in this class is structural learning with forgetting .The method is based on the following assumptions:- Forgetting can be selective, that is, only certain connections with small weights forget. Some advantages of using this approach are that better generalization can be achieved when compared with that of a fully connected trained neural network, and training is faster because unnecessary connections are deleted.

2 min read

The unsupervised algorithm for training a SOM, proposed by Teuvo Kohonen, is outlined in figure 4.22.After each input pattern is presented, the winner is found and the connection weights in its neighborhood area N_t increase, while the connection weights outside the area are kept as they are. a is a learning parameter. It is recommended that the training time moments t (cycles) are more than 500 times the output neurons. If the training set contains fewer instances than this number, then the whole training set is fed again and again into the network. SOMs learn statistical features. The synaptic weight vectors tend to approximate the density function of the input vectors in an orderly fashion (Kohonen 1990). Synaptic vectors w_jconverge exponentially to centers of groups of patterns and the whole map represents to a certain degree the probability distribution of the input data. This property of the SOM is illustrated by a simple example of a two input, 20 x 20-output SOM. The input vectors are generated randomly with a uniform probability distribution function as points in a two-dimensional plane, having values in the interval [0,1].

2 min read

Before starting to develop a solution to a given problem, questions must be answered. What is the point of using a neural network for solving that specific problem? Why should a neural network be used? What are the benefits of using a network? The generic properties of different neural network types, and connectionist models in general, must be known to answer this question. What properties are going to be useful for solving the problem? Problem identification also includes analysis of the existing problem knowledge. The problem knowledge may contain data and rules. If data are available, the independent variables (the input variables) should be clearly distinguished from the dependent variables (the output variables) in order to choose a proper neural network architecture and a learning method. These variables can be discrete, continuous, linguistic, boolean, etc. In some cases, only input variables are present. In this case, unsupervised learning algorithms and appropriate neural networks can be used to cluster, to conceptualize, and to recognize patterns in the domain data. If rules are available in the problem knowledge, they can be implemented in an appropriate neural network structure. Especially suitable for connectionist implementation are fuzzy rules. If both rules and data are available, then hybrid systems, which incorporate both neural networks and symbolic AI techniques, can be used, or a neural network can be trained by using both data and rules, the latter being treated as hints or input-output associations.

3 min read

Introduction:-Hopfield networks, named after their inventor John Hopfield (1982), are fully connected feedback Networks. The neurons in Hopfield networks are characterized by the following: binary or bivalent input signals, binary or bivalent output signals, simple summation function, and hard-limited threshold activation function. There are alternative variants of realizations of a Hopfield network. Every neuron j, j = 1, 2, . . .,n in the network is connected back to every other one, except itself. Input patterns x_j are supplied to the external inputs Ij and cause activation of the external outputs. The response of such a network, when an input vector is supplied during the recall procedure, is dynamic, that is, after supplying the new input pattern, the network calculates the outputs and then feeds them back to the neurons; new output values are then calculated, and so on, until an equilibrium is reached. Equilibrium is considered to be the state of the system when the output signals do not change for two consecutive cycles, or change within a small constant. The weights in a Hopfield network are symmetrical for reasons of stability in reaching equilibrium, that is, w_ij = w_ji. The network is of an additive type, that is,

2 min read

Description:-The way of playing games by revealing consequences through applying logic and heuristic rules is only one approach to game playing. Another approach, mainly applicable to simple board games, is to learn how to react to certain patterns on the board. Such a stimulus-reaction approach can be represented by associations in the form of:

3 min read

Introduction:-Similar to the way the fuzzy neuron and the neo-fuzzy neuron were created, different types of fuzzy neural networks have been developed and applied to different tasks. A fuzzy neural network (FNN) is a connectionist model for fuzzy rules implementation and inference. There is a great variety of architectures and functionalities of FNN. The FNNs developed so far differ mainly in the following parameters: 1. Type of fuzzy rules implemented; this affects the connectionist structure used. 2. Type of inference method implemented; this affects the selection of different neural network parameters and neuronal functions, such as summation, activation, and output function. It also influences the way the connection weights are initialized before training, and interpreted after training.

3 min read

Introduction:-A hybrid system is a mixture of paradigms in the philosophical sense. In a technical sense, it is a mixture of methods and tools used in one system. There are three major modes of functioning in which different knowledge-engineering paradigms can be coupled:- 1. Loosely coupled: Different paradigms are applied separately, either in parallel, or, more typically, in a sequence, on different subtasks of the problem. For example, a neural network can be used for feature extraction and rule extraction, and a rule-based system for inference. 2. Tightly coupled: The paradigms are used in different subsystems, which work in a close "collaboration" to reach the solution; they communicate intensively and possibly frequently.

2 min read

Introduction:-Auto-associative memory means patterns rather than associated pattern pairs, are stored in memory. Hopfield model is one-layer unidirectional auto-associative memory. The model consists, a single layer of processing elements where each unit is connected to every other unit in the network but not to itself. − Connection weight between or from neuron j to i is given by a number wij. The collection of all such numbers are represented by the weight matrix W which is square and symmetric, i.e, w i j = w j I for i, j = 1, 2. . . m.

1 min read

Introduction:-Fuzzy production systems use fuzzy logic inference techniques, with the use of multilayer perceptrons and the backpropagation learning algorithm, and with some other inference techniques. The main idea of realizing fuzzy production systems as an NPS is that activation values of nodes in the WM contextually represent membership degrees when dealing with fuzzy variables.

4 min read

Introduction:-The process of approximate reasoning in an NPS can be described as matching facts represented by their truth degrees against antecedents of rules. Then the truth values (certainty degrees) of the new facts (conclusions) are computed. This is repeated as a chain of inferring new facts, matching the newly inferred facts (and the old facts of course) to the productions again, and so forth. The reasoning process is no monotonic; processing of new facts may decrease the certainty degree of an already inferred fact. The main idea of controlling the approximate reasoning in an NPS is that by tuning the inference control parameters we can adjust the reasoning process for a particular production system to the requirements of the experts. Approximate reasoning in an NPS is a consequence of its partial match. For example, by using the noise tolerance coefficients Q_i, an NPS can separate facts that are relevant to the decision process from irrelevant facts. Rules with different sensitivity coefficients P_i react differently to the same set of relevant facts. An NPS can work with missing data. One rule may fire even when some facts are not known. By adjusting the degrees of importance DI_ij we declare that some condition elements are more important than others and rules can fire if only the important supporting facts are known. Adjustment of the inference control parameters facilitates the process of choosing an appropriate inference for a particular production system.

3 min read

Introduction: - An associative memory is a content-addressable structure that maps a set of input patterns to a set of output patterns. A content-addressable structure is a type of memory that allows the recall of data based on the degree of similarity between the input pattern and the patterns stored in memory. There are two types of associative memory: auto-associative and hetero-associative. • An auto-associative memory retrieves a previously stored pattern that most closely resembles the current pattern.

3 min read

Introduction:-An associative memory is a storehouse of associated patterns which are encoded in some form. When the storehouse is triggered or excited with a pattern, then the associated pattern pair is recalled or appears at the output. The input could be an exact or distorted or partial representation of a stored pattern. The associated pattern pairs

2 min read

Description:- Carpenter and Grossberg (1998) referred such occurrence as the stability-plasticity dilemma which is common in designing intelligent learning systems. In general, a learning system should be plastic, or adaptive in reacting to changing environments, and should be stable to preserve knowledge acquired previously. Stability-Plasticity Dilemma (SPD)

2 min read

Introduction:-Data communication between productions and neural networks take place through the working memory makes it possible to use the execution cycle of the production system as a control cycle for the whole hybrid connectionist production system. The overall control is done with the use of production rules. Through production rules different modules can be chained in a sequence, one module using the output from another, the wholechain solving one task.

2 min read

Introduction:-Hybrid systems employ more than one technology to solve a problem. Hybridization of technologies can have pitfalls and therefore need to be done with care. If one technology can solve a problem then hybrid technology ought to be used only if its application results in a better solution. Hybrid systems have been classified as Sequential, Auxiliary and Embedded. In Sequential hybrid system, the technologies are used in pipelining fashion. In Auxiliary hybrid system, one technology calls the other technology as subroutine. In Embedded hybrid system, the technologies participating appear to be fused totally. Sequential Hybrid System:-In Sequential hybrid system, the technologies are used in pipelining fashion. Thus, one technology's output becomes another technology’s input and it goes on. However, this is one of the weakest forms of hybridization since an integrated combination of technologies is not present.

3 min read

Introduction:-The idea of two-level knowledge processing, discussed in the previous section, is implemented here by using a standard production system for higher-level processing and neural networks for low-level processing. The main idea of incorporating neural networks into a production rule is to activate the networks as actions in the right-hand side of the production. Facts from the working memory are passed as input vectors to the neural networks. The output vectors, produced by the neural networks, are considered new facts. They may be either asserted in the working memory and possibly affect other productions during a next match-select-act cycle of the production system, or they can be used in the current production by activating other actions. Thefollowing example illustrates how neural networks are incorporated into productions. Example

3 min read

Introduction:-This model is based on the assumption that the whole process of speech recognition and language analysis is an integrated continuous process and the border between the two is not well-defined. This process can be viewed in two dimensions time and type of knowledge applied in order to recognize and understand a spoken sequence, where the speech recognition and language analysis phases overlap in both dimensions. For example, correctly recognizing a spoken phoneme may require understanding the context of the whole word, and also some words pronounced before and after it.

2 min read

Each of the paradigms has a contribution to make to the ultimate solution of a problem, as is pointed out below: Symbolic AI systems can contribute with: Fuzzy systems can contribute with:

4 min read

Description:-The speech recognition process is representable as a two-level hierarchical process, consisting of a low level sub words recognition, for example, phoneme recognition, and a high-level—words, sentences, contextual information recognition, language analysis; each of the levels being representable in a recursive manner as many other levels of processing. Different combinations of techniques for low-level and higher level processing uses the following techniques:- Template matching and dynamic time warping for low-level processing. Speech recognition using template-matching involves comparing an unclassified input pattern to a dictionary of patterns (templates) and deciding which pattern in the dictionary the input pattern is closest to. Distance measures are used to decide the best match. Before the matching is done it is necessary to perform some time alignment between the input pattern and each reference template. Owing to the variability of speech there will be local and global variations in the time scale of two spoken examples of the same word, regardless of whether the two examples were uttered by the same speaker or not. An effective technique utilized in computer speech recognition for time-aligning two patterns is a nonlinear time-normalizing technique, dynamic time warping. Speech recognition systems based on dynamic time warping have been used successfully for isolated word recognition. Usually the vocabulary is medium-sized or less (i.e., < 100 words) because the dictionary of reference templates takes up a lot of storage space. Dynamic time-warping systems are also usually speaker-dependent. For speaker-independent systems reference templates have to be collected from a large number of people; these are then clustered to form a representative pattern for each recognition unit (Owens 1993). Speech recognition systems utilizing dynamic time warping have also been used for connected speech recognition and recognizing strings of words such as a series of digits (e.g., a telephone number). A limitation of dynamic time warping, when the recognition units are words, is that its time-aligning Capabilities can lead to confusion between words when the principle distinguishing factor is the duration of a vowel, for example, "league" and "Leek.

5 min read

An NPS is a "pure" connectionist system. A concrete NPS system is constructed for each defined set of production rules. An NPS consists of three neural sub networks: (1) Production memory (PM); (2) Working memory (WM); and

4 min read

Introduction:- The neocognitron design evolved from an earlier model called the cognitron,and there are several versions of the neocognitron itself. The system was designed to recognize the numerals 0 through 9, regardless of where they are placed in the field of view of the retina. Moreover, the network has a high degree of tolerance to distortion of the character and is fairly insensitive to the size of the character. This first architecture contains only feed forward connections. Functional Description:-The PEs of the neocognitron are organized into modules that we shall refer to as levels. Each level consists of two layers: a layer of simple cells, or S-cells, followed by a layer of complex cells, or C-cells. Each layer, in turn, is divided into a number of planes, each of which consists of a rectangular array of PEs. On a given level, the S-layer and the C-layer may or may not have the same number of planes. All planes on a given layer will have the same number of PEs; however, the number of PEs on the S-planes can be different from the number of PEs on the C-planes at the same level. Moreover, the number of PEs per plane can vary from level to level. There are also PEs called V_s-cells and V_c-cells. Here We construct a complete network by combining an input layer, which we shall call the retina, with a number of levels in a hierarchical fashion. The interconnection strategy is unlike that of networks that are fully interconnectedbetween layers, such as the backpropagation network. Each layer of simple cells acts as a feature extraction system that uses the layer preceding it as its input layer. On the first S-layer, the cells on each plane are sensitive to simple features on the retina—in this case, line segments at different orientation angles. Each S-cell on a single plane is sensitive to the same feature, but at different locations on the input layer. S-cells on different planes respond to different features.

1 min read

Introduction: -Kosko (1988) extended the Hopfield model, which is single layer, by incorporating an additional layer to perform recurrent auto-associations as well as hetero-associations on the stored memories. The network structure of the bidirectional associative memory model is similar to that of the linear associator but the connections are bidirectional; i.e., w_ij = w_ji, for i = 1, 2, . . . , n and j = 1, 2, . . . , m.

2 min read

Introduction:-Kosko (1987) extended this network to two-layer bidirectional structure called Bidirectional Associative Memory (BAM) which can achieve hetero-association. The important performance attributes of the BAM is its ability to recall stored pairs particularly in the presence of noise. Definition : If the associated pattern pairs (X, Y) are different and if the model recalls a pattern Y given a pattern X or vice-versa, then it is termed as hetero-associative memory. Bidirectional Associative Memory (BAM) Operations

2 min read

Introduction:-An associative memory is a system which stores mappings of specific input representations to specific output representations. The simplest associative memory model is linear associator, which is a feed-forward type of network. It has very low memory capacity and therefore not much used. The popular models are Hopfield Model and Bi-directional AssociativeMemory (BAM) model. The simplest and among the first studied associative memory models is Linear associator. It is a feed-forward type of network where the output is produced in a single feed-forward computation. It can be used as an auto-associator as well as a hetero-associator, but it possesses a very low memory capacity and therefore not much used. The popular associative memory models are Hopfield Model and Bi-directional Associative Memory (BAM) model.

2 min read

Introduction:- The Architecture of ART1 neural network consist of two layers of neurons. ATR1 model consists an "Attentional" and an "Orienting" subsystem. The Attentional sub-system consists of : The pattern vector is input to comparison layer F1

2 min read

Introduction:-ART includes a wide variety of neural networks. ART networks follow both supervised and unsupervised algorithms. The unsupervised ARTs as ART1, ART2, ART3, are similar to many iterative clustering algorithms. The simplest ART network is a vector classifier. It accepts as input a vector and classifies it into a category depending on the stored pattern it most closely resembles. Once a pattern is found, it is modified (trained) to resemble the input vector. If the input vector does not match any stored pattern within a certain tolerance, then a new category is created by storing a new pattern similar to the input vector. Consequently, no stored pattern is ever modified unless it matches the input vector within a certain tolerance. This means that an ART network has

1 min read

Introduction:-The comparison layer F1 receives the binary external input, then F1 passes the external input to recognition layer F2 for matching it to a classification category. The result is passed back to F1 to find : If the category matches to that of input, then − If Yes (match) then a new input vector is read and the cycle starts again

3 min read

Introduction:-An adaptive clustering technique was developed by Carpenter and GrossBerg in 1987 and is called the Adaptive Resonance Theory (ART) .The Adaptive Resonance Theory (ART) networks are self-organizing competitive neural network. ART includes a wide variety of neural networks. ART networks follow both supervised and unsupervised algorithms. − The unsupervised ARTs named as ART1, ART2, ART3, . . and are similar to many iterative clustering algorithms where the terms "nearest “and "closer" are modified by introducing the concept of "resonance”. Resonance is just a matter of being within a certain threshold of a second similarity measure.

1 min read

Introduction:-The Adaptive Resonance Theory (ART) developed by Carpenter and Grossberg designed for clustering binary vectors, called ART1.They later developed ART2 for clustering continuous or real valued vectors. The capability of recognizing analog patterns is significant enhancement to the system. The differences between ART2 and ART1 are :

1 min read

Description:-The ART comes in several varieties. They belong to both unsupervised and supervised form of learning. Unsupervised ARTs are named as ART1, ART2, and ART3. . And are similar to many iterative clustering algorithms. − ART1 model (1987) designed to cluster binary input patterns. − ART2 model (1987) developed to cluster continuous input patterns.

3 min read

Introduction:- ART stands for "Adaptive Resonance Theory", invented by Stephen Grossberg in 1976. ART represents a family of neural networks. ART encompasses a wide variety of neural networks. The basic ART System is an unsupervised learning model. The term "resonance" refers to resonant state of a neural network in which a category prototype vector matches close enough to the current input vector. ART matching leads to this resonant state, which permits learning. The network learns only in its resonant state. ART neural networks are capable of developing stable clusters of arbitrary sequences of input patterns by self-organizing. ART systems are well suited to problems that require online learning of large and evolving databases.

3 min read

Introduction:-While no information is available about desired outputs the network updated weights only on the basis of input patterns. The Competitive Learning network is unsupervised learning for categorizing inputs. The neurons (or units) compete to fire for particular inputs and then learn to respond better for the inputs that activated them by adjusting their weights. An example of competitive learning network is shown below.

3 min read

Introduction: -The scenario is a simple pattern-matching operation during which an ART network tries to determine whether an input pattern is among the patterns previously stored in the network. A pattern of activation, X, is produced across F1. The processing done by the units on this layer is a somewhat more complicated form of that done by the input layer of the CPN .The same input pattern excites both the orienting subsystem, A, and the gain control, G (the connections to G are not shown on the drawings). The output pattern, S, results in an inhibitory signal that is also sent to A. The network is structured such that this inhibitory signal exactly cancels the excitatory effect of the signal from I, so that A remains inactive. Notice that G supplies an excitatory signal to F1. The same signal is applied to each node on the layer and is therefore known as a nonspecificsignal. The need for this signal will be made clear later.The appearance of X on F1 results in an output pattern, S, which is sent through connections to F2. Each FI unit receives the entire output vector, S. from F1. F2 units calculate their net-input values in the usual manner by summing the products of the input values and the connection weights. In response to inputs from F1, a pattern of activity, Y, develops across the nodes of F2. F2 is a competitive layer that performs a contrast enhancement on the input signal like the competitive layer.

3 min read

Introduction:-Human has ability to learn through classification. Human learn new concepts by relating them to existing knowledge and if unable to relate to something already known, then creates a new structure. The unsupervised ARTs named as ART1, ART2,ART3..Represent such human like learning ability. ART is similar to many iterative clustering algorithms where each pattern is processed by Iterative Clustering

2 min read

Description:-Implementation of AND function in the neural network. − There are 4 inequalities in the AND function and they must be satisfied.

1 min read

Introduction:-Back-propagation is simply a way to determine the error values in hidden layers. This needs be done in order to update the weights. The best example to explain where back-propagation can be used is the XOR problem. Consider a simple graph shown below.

3 min read

Introduction:-When problem knowledge includes explicit (fuzzy) rules, a connectionist system can be trained with them, as with input-output associations where the input patterns are the antecedent parts of the rules and the output patterns are the consequent parts. A fuzzy association A → B where A and B are fuzzy values, defined, for example, by their membership functions, can be memorized in an n-input, m-output neural network, where n is the cardinality of the universe U_x, m is the cardinality of the universe U_y, and x and y are fuzzy variables with corresponding fuzzy values A and B . This is the basis for using connectionist architectures for reasoning over fuzzy rules of the type IF x is A, THEN y is B. An MLP neural network can be trained with a set of fuzzy rules. The rules are treated as input-output training examples. When a new fuzzy set A' is supplied as a fuzzy input, the network will produce an output vector that is the desired fuzzy output B'. The generalized modus ponens law can be realized in a connectionist way. The following inference laws can also be satisfied by the same neural network, subject to a small error: A →B (modus ponens); Very A→Very B; More or less A →More or less B. A method for implementing multiantecedent fuzzy rules on a single neural network is introduced in Kosko (1992) and Kasabov (1993). The dimension of the input vector is the sum of the used discrete representation for the cardinality of the universes of all the fuzzy input variables. The dimension of the output vector is the sum of the corresponding discrete cardinality of all universes of the output fuzzy variables in the fuzzy rules. The fuzzy rules are assumed to have the same input and output variables but different combinations of their fuzzy values. If OR connectives are used in a rule, it may require that more training examples are generated based on combinations between antecedent parts of the rules for the Bank Loan Decision problem. The MLP consists of 33 input nodes, 11 intermediate nodes, and 11 output nodes. The inferred bank loan fuzzy decision values for the same three bank loan applicants as used in the example above, but here represented as fuzzy input values, are given as fuzzy membership functions in figure 5.8. The experiments show that the decisions inferred by the neural network for the three representative cases defined as fuzzy sets are correct. The ambiguity in the third solution vector clearly suggests 'not known decision' case.

3 min read

Introduction:- The hidden layer allows ANN to develop its own internal representation of input-output mapping. The complex internal representation capability allows the hierarchical network to learn any mapping and not just the linearly separable ones. Algorithm for Training Network • Basic algorithm loop structure

1 min read

The q^th neuron of the output layer is shown below. It has input from the output of the hidden neurons layers. If we consider transfer function as sigmoidal function then the output of the q^th output neuron is given by Where,

3 min read

Introduction:-Rosenblatt proposed learning networks called Perceptron. The task was to discover a set of connection weights which correctly classified a set of binary input vectors. The basic architecture of the perceptron is similar to the simple AND network in the previous example. A perceptron consists of a set of input units and a single output unit. As in the AND network, the output of the perceptron is calculated by comparing the net input and a threshold θ.If the net input is greater than the threshold θ , then the output unit is turned on , otherwise it is turned off. To address the learning question, Rosenblatt solved two problems.

1 min read

2 min read

The p^th neuron of the hidden layer is shown below. It has input from the output of the input neurons layers. If we consider transfer function as sigmoidal function then the output of the p^th hidden neuron is given by Where O_Hp is the output of the p^th hidden neuron,I_Hp is the input of the p^th hidden neuron, and θ_HP is the threshold of the p^th neuron;

2 min read

Properties Related to Intersection:-Absorption, Identity, Idempotence, Commutativity, Associativity. ■ Absorption by Empty Set : A ∩ Φ = Φ input = Equality [Small ∩ Empty , Empty]

3 min read

Introduction:-Fuzzy Relations describe the degree of association of the elements; Example : “x is approximately equal to y”. Fuzzy relations offer the capability to capture the uncertainty and vagueness in relations between sets and elements of a set. Fuzzy Relations make the description of a concept possible. Fuzzy Relations were introduced to supersede classical crisp relations; It describes the total presence or absence of association of elements. Fuzzy relation is a generalization of the definition of fuzzy set from 2-D space to 3-D space. Fuzzy relation :-Consider a Cartesian product

3 min read

Introduction: -The word "fuzzy" means "vagueness". Fuzziness occurs when the boundary of a piece of information is not clear-cut. Fuzzy sets have been introduced by Lotfi A. Zadeh (1965) as an extension of the classical notion of set. Classical set theory allows the membership of the elements in the se in binary terms, a bivalent condition - an element either belongs or does not belong to the set. Fuzzy set theory permits the gradual assessment of the membership of elements in a set, described with the aid of a membership function valued in the real unit interval [0, 1]. Example:

2 min read

Introduction:-A Fuzzy Set is any set that allows its members to have different degree of membership, called membership function, in the interval [0 , 1]. A fuzzy set A, defined in the universal space X, is a function defined in X which assumes values in the range [0, 1]. A fuzzy set A is written as a set of pairs {x, A(x)} as

2 min read

Description:- − If characteristic function µ_A(x) has only values 0 ('false') and 1 ('true''). Such sets are crisp sets.

3 min read

Example 2. The empty set Φ and the universal set X, as fuzzy sets, are complements of one another. Φ' = X , X' = Φ

2 min read

Introduction:-A fuzzy set A defined in the universal space X is a function defined in X which assumes values in the range [0, 1]. A fuzzy set A is written as a set of pairs {x, a(x)}. A = {{x , A(x)}} , x in the set X, where x is an element of the universal space X and A(x) is the value of the function A for this element. The value A(x) is the degree of membership of the element x in a fuzzy set A.

2 min read

Introduction:-Hybrid systems employ more than one technology to solve a problem. Hybridization of technologies can have pitfalls and therefore need to be done with care. If one technology can solve a problem then hybrid technology ought to be used only if its application results in a better solution. Hybrid systems have been classified as Sequential, Auxiliary and Embedded. In Sequential hybrid system, the technologies are used in pipelining fashion. In Auxiliary hybrid system, one technology calls the other technology as subroutine. In Embedded hybrid system, the technologies participating appear to be fused totally. Sequential Hybrid System:-In Sequential hybrid system, the technologies are used in pipelining fashion. Thus, one technology's output becomes another technology’s input and it goes on. However, this is one of the weakest forms of hybridization since an integrated combination of technologies is not present.

3 min read

Introduction: -Hybrid systems employ more than one technology to solve a problem. Hybridization of technologies can have pitfalls and therefore need to be done with care. • If one technology can solve a problem then a hybrid technology ought to be used only if its application results in a better solution. • Hybrid systems have been classified as:

2 min read

Introduction:-A fuzzy logic system contains the sets used to categorize input data (i.e., fuzzification), the decision rules that are applied to each set, and then a way of generating an output from the rule results (i.e., defuzzification). In the fuzzification stage, a data point is assigned a degree of membership (DOM) determined by a membership function. The membership function is often a triangular function centered at a given point. The Defuzzification is the name for a procedure to produce a real (non-fuzzy) output . Associative Memory is a type of memory with a generalized addressing method. The address is not the same as the data location, as in traditional memory. An associative memory system stores mappings of specific input representations to specific output representations.

2 min read

Motion in a crossed electric and magnetic fields: The force on the charged particle in the presence of both electric and magnetic fields is given by Let the electric and magnetic fields be at right angle to each other, so that,

2 min read

Neural networks (NNs) are the adaptive system that changes its structure based on external or internal information that flows through the network. Neural network solve problems by self-learning and self-organizing. Back Propagation Network (BPN) is a method of training multi-layer neural networks. Here learning occurs during this training phase. The steps involved are:

2 min read

The Neural Networks and Genetic Algorithms represent two distinct methodologies. Neural Networks: can learn various tasks from examples; classify phenomena and model nonlinear relationships. Genetic Algorithms: have offered themselves as potential candidates for the optimization of parameters of NN.

2 min read

Neural Networks and Fuzzy logic represents two distinct methodologies to deal with uncertainty. Each of these has its own merits and demerits. Neural Networks: − Merits: Neural Networks can model complex nonlinear relationships and are appropriately suited for classification phenomenon into predetermined classes.

3 min read

Introduction:-Fuzzy Systems include Fuzzy Logic and Fuzzy Set Theory. • Knowledge exists in two distinct forms : − The Objective knowledge that exists in mathematical form is used in engineering problems; and

3 min read

The Lorentz Force: We know that an electric field exerts a force on a charge . In the presence of a magnetic field , a charge q experiences an additional force where is the velocity of the charge. Note that • , which shows that the magnetic force does not do any work. In the case where both are present, the force on the charge q is given by

1 min read

Fuzzy systems have been integrated with GAs. The fuzzy systems like NNs (feed forward) are universal approximator in the sense that they exhibit the capability to approximate general nonlinear functions to any desired degree of accuracy. The adjustments of system parameters called for in the process, so that the system output matches the training data, have been tackled using GAs. Several parameters which a fuzzy system is involved with like input/output variables and the membership function that define the fuzzy systems have been optimized using GAs. Typical Hybrid Systems The Systems considered are listed below.

1 min read

Introduction:-The fuzzification is a process of transforming crisp values into grades of membership for linguistic terms of fuzzy sets. The purpose is to allow a fuzzy condition in a rule to be interpreted. • Fuzzification of the car speed

2 min read

Description:-Genetic algorithms work with population of individual strings. The steps involved in GAs are: − Each individual string represents a possible solution of the problem considered,

1 min read

Introduction:-For a system whose output is fuzzy, it is easier to take a crisp decision if the output is represented as a single quantity. This conversion of a single crisp value is called Defuzzification.It is the reverse process of fuzzification. The typical Defuzzification methods are − Centroid method,