Vanishing and Exploding
Vanishing and Exploding
& exploding
By
LOGESHWARI P
(CB.EN.P2BME23009)
GRADIENT DESCENT
• 2. **Output Range:**
• - **Softmax:** Produces a probability distribution over multiple classes, with each • 6. **Derivative:**
element in the range (0, 1). The sum of all elements in the output vector is 1.
• - **Softmax:** The derivative of the softmax function involves multiple terms, and it's
• - **Sigmoid:** Produces an output in the range (0, 1) and is suitable for binary often used in conjunction with the cross-entropy loss during backpropagation in
classification problems. It can be interpreted as the probability of belonging to the positive classification tasks.
class.
• - **Sigmoid:** The derivative of the sigmoid function has a simple and interpretable
form, making it computationally efficient during backpropagation.
• 3. **Application:**
• - **Softmax:** Typically used in the output layer of a neural network for multi-class • In summary, softmax is suitable for multi-class classification tasks, while sigmoid is
classification problems. It's especially useful when there are more than two classes. commonly used in binary classification problems. The choice between them depends on the
nature of the task and the number of classes involved.
• - **Sigmoid:** Commonly used in binary classification problems. It's also used in the
hidden layers of neural networks to model non-linear relationships in the data.