Backpropagation Algorithm

Backpropagation algorithm was first formulated by Werbos and popularized by Rumelhart and Mc. Clelland to wear on ANN (Artificial Neural Nets), and is commonly abbreviated algorithm with BP. These algorithms include supervised learning method and designed for operation on the feed forward multi-layer nets.
Backpropagation method is used widely. An estimated 90% is used in many fields, among others in the financial sector, handwriting pattern recognition, and the introduction of sound and color.
This algorithm is widely used in applications settings because the learning process is based on a simple relationship, ie, if the output gives the wrong result, the weight is corrected so that errors can be reduced and the net response is expected to be closer to the true price. Backpropagation is also capable to handle weight in the hidden layer (hidden).
Broadly speaking this algorithm describes, when the nets are given input patterns as training patterns so that pattern to the nodes in the hidden layer to be forwarded to the output layer nodes. Then the output layer nodes is called the output response of the net. When the net output is not equal to the expected output then the output will be spread back on the hidden layer forwarded to the node in the input layer. Therefore, the mechanism is called the backpropagation training.

Training Phase

This training phase is the step how a neural net was trained, that is by making changes the connection weights. While the phase of problem solving will be done if the learning process is completed, phase is the process of testing or testing.
Backpropagation algorithm consists of two processes, namely the feed forward and backpropagation of error. To more clearly be described as follows :

1. Initialize weight factors with small random values.

2. Repeat steps 2 through 9 until the stop condition is met.

3. Perform steps 3 through 8 for each pair of training.

4. Each input unit (Xi, i = 1, ... n) receives input signals Xi and the signals are distributed to the upper unit of the hidden layer (hidden units).

5. Each hidden summing weighted factors :
and counted in accordance with the activation function:

Because the sigmoid function is used:

then sends a signal to all units on it (output units).

6. Each unit of output (Yk, k = 1,2,3, ..., m), summed the weighted factors :
Calculating according to the activation function :

7. Each unit of output (Yk, k = 1,2,3, ..., m) receives a target pattern in accordance with the input pattern during training and calculate error :because f'(Y_ink) = Yk by using the sigmoid function, then :
Calculating the weight factor correction (to correct Wjk)
Calculating the correction correction :
and deliver value to the unit k layer beneath.

8. Each hidden unit (Zj, j = 1,2,3, ..., p), summing the delta inputs (from units in the upper layer)
then multiplied by the activation function to compute the error.
Then calculate the weight correction (used to improve Vij)
then calculate the bias correction (to correct Voj)

9. Each output unit (Yk, k = 1,2,3, ..., m) fixed bias and weight for (j = 0,1,2, ..., p)
Each hidden unit (Zj, j = 1,2,3, ... p) fixed bias and weight for (j = 0,1,2, ..., n)

Fundamental of Neural Network

Artificial Neural Network
Neural nets (ANN) is known as connectionist models, parallel distributed processing models, or just written the neural network. Defining the neural nets views of the function or structure is a simplification of the model design of the human brain. The performance of the structure of biological neural nets in the human brain is by way of relay signals from one neuron to another neuron and the corresponding adjacent. The same thing continues to neurons that follows, until at last the desired neuron signals.
Artificial neurons in a neural net structure is a processing element that can function like a neuron. A collection of neurons made into a mesh that will serve as a computational tool based computers by way of a mathematical approach to math. Or ANN can also be viewed as "a system that consists of elements that are distributed in parallel with the ability to improve performance through a process of learning."

Basic Structure of Biological Nets
The human brain contains millions of nerve cells responsible for processing information. Each cell works like a simple processor. Each of these cells interact with each other so that supports the ability of the human brain works.


Each neuron will have a cell nucleus, this nucleus will be served to make the processing of information. The information received by the dendrites and then come out through the Axon and the results will be input for other neurons, which dendrites between the two cells are brought into contact with synapsis. More details on this can be obtained at the disciplines of molecular biology.
In general, neural nets are formed from millions (even more) the basic structure of neurons which are interconnected and integrated with each other so that they can carry out activities regularly and continuously in accordance with needs.

Learning Methods
Learning for the ANN is a process set the price of weight parameters to obtain the best price by exercising (training) nets according to the desired system performance. The definition of learning itself, according to Herbert Simon (1983): "Learning is to show the changes in a system which is adjusted based on sensing that allows the system to perform tasks more effectively and efficiently in the future."
The purpose of this process so that a collection of input patterns (input vector) is given to produce the output pattern (output vector) is desirable or at least close. This training sequence is formed by applying the input pattern and set nets close to the weight of output follows a pattern of a particular learning algorithm during the learning process, weight is slowly converging towards a certain price, so that the input pattern to produce the desired output pattern. Ability to learn is also the ability to approximate a function (approximation capability). This makes it flexible to be used in the process of identification of a plant.
There are two methods of learning the neural network (ANN), namely:

1. Supervised training
Backpropagation algorithm included in method (supervised training). This algorithm requires the couple to each input vector with the vector of the target (desired output). A trained neural network system by comparing the number of output pairs with the target vector.
Input pattern is inserted into the nets that are then processed to produce output, which is called the output of the net. The difference of the two output states an error (error) which will be used to change the connection weight. So the error will be smaller in the next training cycle.

2. Unsupervised Training
This algorithm does not require the target vector to output, so no comparisons to determine the ideal response. Collection of training patterns consist of only the input vector training algorithm and serves as a modifier or modifications to generate a net weight of the vector, so that the implementation of two training vectors of a vector of other similar enough to produce the same output pattern. In the training process, the net classifies the input patterns into similar groups. Introduction of a vector of a certain class of input vectors will produce a specific output, but there's no way to determine beforehand the training, which will produce a particular output pattern with an input vector of a particular class.