Scribd
Upload
18 views

Lecture-4 Multi-Layer Perceptrons

The document discusses Multi-Layer Perceptrons (MLPs), which are advanced neural networks that consist of multiple layers, including input, hidden, and output layers, allowing them to classify complex and nonlinear problems. MLPs utilize supervised learning with backpropagation and employ nonlinear activation functions, such as tangent-sigmoid and pure linear, to enhance their learning capabilities. Key considerations for MLPs include the number of layers and neurons, as well as the amount of training data required for optimal performance.

Uploaded by

mohabfata2003
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
18 views

Lecture-4 Multi-Layer Perceptrons

The document discusses Multi-Layer Perceptrons (MLPs), which are advanced neural networks that consist of multiple layers, including input, hidden, and output layers, allowing them to classify complex and nonlinear problems. MLPs utilize supervised learning with backpropagation and employ nonlinear activation functions, such as tangent-sigmoid and pure linear, to enhance their learning capabilities. Key considerations for MLPs include the number of layers and neurons, as well as the amount of training data required for optimal performance.

Uploaded by

mohabfata2003
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 23

CS 363: Neural Networks

Lecture-4
Multi-Layer Perceptrons
Introduction

• The Single Layer Perceptron (SLP) classifier is the


simple form of pattern classifier with a single
perceptron.

• It can only classify between two cases that are


linearly separable.

• However, many problems in the real world are


nonlinear.
Introduction

• Because of the limitations by SLP, researchers have explored the


integration of multiple perceptrons into a larger neural network to
become the multilayer feed-forward network, also known as Multi-
Layer Perceptron (MLP).

• This configuration enables the MLP to classify difficult and


nonlinear cases well beyond the capabilities of the common SLP.

• MLPs usually use supervised learning based on the error


backpropagation algorithm
Remember XOR problem
Multilayer neural networks
A multilayer perceptron is a feedforward neural network with
one or more hidden layers.

The network consists of an input layer of source neurons, at


least one middle or hidden layer of computational neurons, and
an output layer of computational neurons.

The input signals are propagated in a forward direction on a


layer-by-layer basis.
Multilayer perceptron with two hidden layers

Out put S ignals


Input S ignals

First Second
Input hidden hidden Output
layer layer layer layer
Characteristics of MLP

• The network contains one or more layers of hidden units that


are not part of the input/output parts of the network.

• The hidden units enable the MLP to learn complex tasks and
meaningful features from the input/output relationship.

• The MLP is characterized by a nonlinear activation function in


its hidden layer.

• The MLP exhibits a high degree of connectivity between the


layers, determined by the weights of the network.
Characteristics of MLP

• MLPs typically consist of three or more layers.

• The input layer of the MLP accepts inputs from the


training/testing data.

• The output layer of the MLP presents the output of the MLP
to the outside.

• The hidden layer of the MLP is responsible to learn the


input/output relationship between the input layer and the
output layer through a process called training.
Multi-layer Networks: Issues
• How many layers are sufficient?

• How many PEs needed in (each) hidden layer?

• How much data needed for training?


Number of neurons

 Many neurons:
 Higher accuracy
 Slower
 Risk of over‐fitting
 Memorizing, rather than understanding
 The network will be useless with new problems.

 Few neurons:
 Lower accuracy
 Inability to learn at all

 Optimal number.
A Multilayer Feed-Forward Neural
Output Class Network
Ok
Output nodes
w jk
Oj
Hidden nodes

wij - weights

Input nodes
Network is fully connected
Input Record : xi
Examples of Multi-layer NNs

• Backpropagation
• Neocognitron
• Probabilistic NN (radial basis function NN)
• Boltzmann machine
• Cauchy machine
The MLP Activation Function

• Unlike SLP (which uses the hard limit activation function),


MLPs use a smooth continuous activation functions that are
differentiable.

• This is because the training algorithm requires activation


functions that are smooth and differentiable to work.

• Two most common activation functions are:

– tangent-sigmoid (tansig)

– pure linear (purelin).


The MLP Activation Function (TANSIG)

• The TANSIG activation function approximates the sigmoid


activation function.

• It is commonly used in the hidden layer.

• It can also be used for the output layer in pattern


classification problems.

• It has an input active range between -1 and +1. Values outside


this range will be squashed to within these limits.
The MLP Activation Function (TANSIG)
The MLP Activation Function (TANSIG)
The MLP Activation Function (PURELIN)

• The PURELIN activation function approximates the linear


activation function.

• It is commonly used in the output layer in function


approximation problems.

• It has an input active range between -1 and +1. It has no


squashing effect as the output is linear.
The MLP Activation Function (PURELIN)
How MLPs Work

• The MLP works by propagating the input signal towards the


output on a layer-to-layer basis.

• Each layer processes the signals given to it and outputs a


value to the next layer.

• Each interconnection has a weight value attached to it.

• The weights are used to adjust the strength of the signal


propagating through it.
Three-Layer MLP Example
Value of units in hidden layer
Value of units in hidden layer (simplified)
Value of unit in output layer

You might also like