Generalized Polyak Step Size for First Order Optimization with Momentum

Keyphrases

• Learning Rate 100%
• First-order Optimization Method 100%
• Popular 50%
• Oscillation 50%
• Effective Method 50%
• Ill-conditioned Problem 50%
• Stochastic Gradient Descent 50%
• Gradient Descent 50%
• Machine Learning Applications 50%
• Stochastic Gradient Algorithm 50%
• Stochastic Settings 50%
• Adaptive Step Size 50%
• First-order Optimization Algorithms 50%
• Machine Learning Tasks 50%
• Large-scale Machine Learning 50%
• Momentum Acceleration 50%
• Stochastic Gradient Descent with Momentum 50%
• Heavy Ball Method 50%
• Momentum Parameter 50%
• Adaptive Step-size Algorithm 50%
• Smoothness Parameter 50%

Engineering

• Step Size α 100%
• Gradient Descent 33%
• Learning System 22%
• Objective Function 11%
• Optimization Method 11%
• Learning Task 11%
• Conditioned Problem 11%

Mathematics

• Step Size 100%
• Stochastics 44%
• Objective Function 11%
• Learning Task 11%