Optimization Algorithms

Vanilla SGD is just the beginning. Adam, RMSprop, and momentum methods adapt learning rates and escape sharp minima faster.

Learning Objectives

• →Implement SGD with momentum
• →Understand adaptive learning rates
• →Implement RMSprop update rule
• →Implement Adam optimizer from scratch

Practice