TorchedUp
LearnBetaProblemsSystem DesignSoonPremium
TorchedUp
LearnBetaProblemsSystem DesignSoonPremium
←

10. SGD with Momentum

Medium

Implement a single SGD with momentum parameter update.

Signature: def sgd_momentum_step(theta: np.ndarray, grad: np.ndarray, v: np.ndarray, lr: float = 0.01, mu: float = 0.9) -> np.ndarray

Return the updated theta_new.

  • Velocity update: v = mu*v - lr*grad
  • Parameter update: theta += v

Math

vt​=μvt−1​−α∇θ​L,θ←θ+vt​

Asked at

Python 30/10 runs today

Output

Anything you print() in your code will show up here after you click Run.

Test Results

○first step no prior velocity
○second step with velocity
○negative gradient🔒 Premium
○2D (N, D) parameter matrix with prior velocity🔒 Premium