Implementing Artificial Neural Networks (ANNs)

- Overview

An Artificial Neural Network (ANN) can be considered as a classification and as a forecasting technique. This technique tries to simulate how the human brain works. In this technique, there are three layers, Input, Hidden, and Output above.

The input layer is mapped to the input attributes. For example, age, gender, number of children can be the inputs to the Input layer. The Hidden layer is an intermediate layer where every input with weightage is received to each node in the hidden layer. The Output layer is mapped to the predicted attributes. 

A neuron is a basic unit that combines multiple inputs and a single output. Combinations of inputs are done with different techniques, and the Microsoft Neural Network uses Weighted Sum. Maximum, Average, logical AND, logical OR are the other techniques used by the different implementation. 

After these inputs are calculated, then the activation function is used. In theory, sometimes, small input will have a large output, and on the other hand, large input might be insignificant to the output. Therefore, typically non-linear functions are used for activation. In Microsoft Neural Network uses tanh as the hidden layer activation function and sigmoid function for the output layer.

In essence, ANNs are powerful tools that can learn complex patterns and make predictions by mimicking the way the human brain processes information.

- The Human Brain vs Artificial Neural Networks

The human brain consists of hundreds of billion of cells called neurons. These neurons are connected together by synapses which are nothing but the connections across which a neuron can send an impulse to another neuron. 

When a neuron sends an excitatory signal to another neuron, then this signal will be added to all of the other inputs of that neuron. If it exceeds a given threshold then it will cause the target neuron to fire an action signal forward — this is how the thinking process works internally.

Artificial Neural Networks (ANNs) are inspired by the human brain and function by processing information through interconnected nodes called neurons. These neurons are organized into layers, and data flows through the network, with each node calculating an output based on its inputs and associated weights. 

Through a learning process, the network adjusts these weights to improve its ability to make accurate predictions or decisions. 

1. Structure and Layers:

Neurons (Nodes): Each neuron receives input from other neurons, processes it, and produces an output, similar to how biological neurons work. 

Layers: ANNs are typically structured into layers:

• Input Layer: Receives the initial data.
• Hidden Layers: Perform intermediate calculations.
• Output Layer: Produces the final result or prediction.

Connections (Weights): Neurons are connected by weighted connections. These weights represent the strength of the connections, and they are adjusted during the learning process.

2. Processing Information:

• Input: Data is fed into the input layer.
• Forward Propagation: The data flows through the network, with each neuron calculating an output based on its inputs and weights.
• Activation Function: Each neuron applies an activation function to its output, which introduces non-linearity and allows the network to learn complex patterns.
• Output: The final layer produces an output, which could be a classification, prediction, or other desired result.

3. Learning and Iteration:

• Training Data: ANNs learn from training data, which is input-output pairs or other forms of data.
• Weight Adjustment: The network adjusts the weights of its connections to minimize the difference between its predicted output and the actual output in the training data.
• Iterative Process: This process of forward propagation, error calculation, and weight adjustment is repeated iteratively, allowing the network to learn and improve its performance.

4. Types of ANNs:

• Convolutional Neural Networks (CNNs): Used for image and video processing, with convolutional layers that learn spatial hierarchies.
• Recurrent Neural Networks (RNNs): Designed for sequential data, like text or time series, with a feedback loop that allows them to remember information.
• Generative Adversarial Networks (GANs): Two neural networks (generator and discriminator) that compete to generate realistic data.