What is the difference between stochastic gradient descent, batch gradient descent, and gradient descent?

Question

Accepted Answer

Stochastic gradient descent, batch gradient descent, and gradient descent differ in the amount of data used to update the model weights.

Gradient descent updates the model weights using the entire dataset in each iteration.
Batch gradient descent updates the model weights using a subset of the dataset (batch) in each iteration.
Stochastic gradient descent updates the model weights using only one data point at a time in each iteration.
Stochastic gradient descent is faster but more noisy compared to batch gradient descent.
Batch gradient descent is slower but more stable compared to stochastic gradient descent.
Gradient descent can be computationally expensive for large datasets.

Popular interview questions of ml engineer