Multi-layer Perceptrons

Multi-Layer Perceptrons: In contrast to perceptrons, multilayer networks can learn not only multiple decision boundaries, but the boundaries may be nonlinear. The typical architecture of a multi-layer perceptron (MLP) is shown below.

To make nonlinear partitions on the space we need to define each unit as a nonlinear function (unlike the perceptron). One solution is to use the sigmoid unit. Another reason for using sigmoids are that they are continuous unlike linear thresholds and are thus differentiable at all points.

Function σ is called the sigmoid or logistic function. It has the following property:
d σ(y) / dy = σ(y) (1 – σ(y))