Search Search Any Topic from Any Website Search
1. What is a gradient in machine learning? A gradient is the vector of partial derivatives of a loss function with respect to model parameters. It points in the direction of the steepest increase of the function. Optimization algorithms update parameters in the negative gradient direction to minimize loss. 2. Difference between Gradient Descent, Stochastic Gradient Descent, and Mini-Batch Gradient Descent Gradient Descent: Uses the entire dataset to compute gradient (slow but stable). Stochastic Gradient Descent (SGD): Uses one sample at a time (fast but noisy). Mini-Batch Gradient Descent: Uses a subset (batch) → balance of speed and stability. Noise helps escape local minima and enables parallelism on GPUs. 3. What is the vanishing gradient problem? Gradients become extremely small when propagated backward through deep networks. This prevents early layers from learning effectively. Main causes: Sigmoid or tanh activation saturation ...