BIDA Homework Chapter 4 - Machine Learning Question 1 a. Explain how a simple Artificial Neural Network (ANN) works. Your explanation should cover the following concepts: ● Structure: Describe the basic building blocks of an ANN, including neurons, layers (input, hidden, output), and connections (weights and biases). ● Forward Propagation: Explain how information flows through the network. Briefly mention the role of activation functions in this process. ● Learning: Describe the basic concept of how an ANN learns from data. (10 marks) b. Compare and contrast three different types of neural networks. Briefly discuss their strengths and weaknesses, and provide an example of a task where each network might be a good choice. (10 marks)

a. Artificial Neural Networks (ANNs) are computing systems inspired by the human brain's neural networks. They are designed to replicate the way in which the human brain processes information.

Structure: The basic building blocks of an ANN are neurons or nodes. These are organized into three types of layers: input, hidden, and output layers. The input layer receives the raw data or the input signals. The hidden layers perform computations and transfer information from the input nodes to the output nodes. The output layer produces the final result. Each neuron is connected to others through connections, where each connection has a weight and a bias. The weight represents the strength of the connection, while the bias allows for adjustments to the output along with the weighted sum of the inputs.

Forward Propagation: Information in an ANN flows from the input layer through the hidden layers to the output layer, a process known as forward propagation. Each neuron takes the weighted sum of its inputs, adds the bias, and then applies an activation function. The activation function, such as a sigmoid or ReLU, introduces non-linearity into the output of a neuron. This transformed output is then sent as input to neurons in the next layer.

Learning: ANNs learn from data through a process called backpropagation and an optimization technique, such as gradient descent. In backpropagation, the network calculates the error in the output (the difference between the predicted and actual output), and propagates this error back through the layers to update the weights and biases. This process is repeated multiple times, and the weights and biases are tweaked each time to minimize the error. This is how the network 'learns' from the data.

b.

1. Feedforward Neural Networks (FNNs): These are the simplest type of ANNs, where information moves in only one direction—forward—from the input layer, through the hidden layers, to the output layer. There are no loops in the network. FNNs are good for simple pattern recognition tasks. However, they might not perform well on complex tasks like image and speech recognition.

2. Convolutional Neural Networks (CNNs): These are designed to process data with a grid-like topology, such as an image, which has a spatial relationship in height and width. CNNs have their "neurons" arranged in three dimensions: width, height, and depth. They are excellent for image and video processing tasks but require a lot of computational power.

3. Recurrent Neural Networks (RNNs): These are used for sequential data where the order of the inputs matters, such as time series data or natural language. RNNs have loops that allow information to be passed from one step in the sequence to the next. They are great for natural language processing and speech recognition tasks. However, they can be difficult to train effectively due to problems like vanishing or exploding gradients.