Foundations of Neural Networks

- Overview

An artificial neural network (ANN) is a simplified model of the human brain's process of processing information. ANNs are inspired by the brain's neurons, with each circular node representing an artificial neuron and an arrow representing a connection. 

ANNs work in the same way as the human brain's processing data. They can create algorithms that can predict problems and create complex patterns of a model. 

The three most common types of neural networks used in artificial intelligence (AI) are: 

- Three-Layered of Neural Network

The brain consists of hundreds of billion of cells called neurons. These neurons are connected together by synapses which are nothing but the connections across which a neuron can send an impulse to another neuron. 

When a neuron sends an excitatory signal to another neuron, then this signal will be added to all of the other inputs of that neuron. If it exceeds a given threshold then it will cause the target neuron to fire an action signal forward — this is how the thinking process works internally. 

An Artificial Neural Network (ANN) can be considered as a classification and as a forecasting technique. This technique tries to simulate how the human brain works. In this technique, there are three layers, Input, Hidden, and Output.

The input layer is mapped to the input attributes. For example, age, gender, number of children can be the inputs to the Input layer. The Hidden layer is an intermediate layer where every input with weightage is received to each node in the hidden layer. The Output layer is mapped to the predicted attributes. 

A neuron is a basic unit that combines multiple inputs and a single output. Combinations of inputs are done with different techniques, and the Microsoft Neural Network uses Weighted Sum. Maximum, Average, logical AND, logical OR are the other techniques used by the different implementation. 

After these inputs are calculated, then the activation function is used. In theory, sometimes, small input will have a large output, and on the other hand, large input might be insignificant to the output. 

Therefore, typically non-linear functions are used for activation. In Microsoft Neural Network uses tanh as the hidden layer activation function and sigmoid function for the output layer. 

- The Main Objective of A Neural Network

The main objective of a neural network is to learn by automatically modifying itself so that it can perform complex tasks. Neural networks can be used to model complex relationships between inputs and outputs or to find patterns in data. 

Neural networks are intricate networks of interconnected nodes, or neurons, that collaborate to tackle complicated problems. They can be written as compositions of elementary functions, typically affine transformations and nonlinear activation functions. The cost function's purpose is to calculate the error we get from our prediction. The smaller the output of the cost function, the closer the predicted value is to the actual value.

It is a type of machine learning process, called deep learning, that uses interconnected nodes or neurons in a layered structure that resembles the human brain. It creates an adaptive system that computers use to learn from their mistakes and improve continuously.

- Artificial Neural Networks (ANN) and MultiLayer Perceptron (MLP)

Artificial Neural Networks (ANN) and MultiLayer Perceptron (MLP) are both types of neural networks used in machine learning. The main difference between the two is that MLP is a type of ANN with specific architecture. ANN is a computational model inspired by the biological neural networks present in the human brain.

A fully connected multi-layer neural network is called a Multilayer Perceptron (MLP). It has 3 layers including one hidden layer. If it has more than 1 hidden layer, it is called a deep ANN. An MLP is a typical example of a feedforward artificial neural network.