Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent

Keyphrases

• Decentralized Algorithm 100%
• Centralized Algorithm 100%
• Parallel Stochastic Gradient Descent 100%
• Communication Cost 66%
• Network Configuration 66%
• Order of Magnitude 33%
• Computational Complexity 33%
• Decentralized Network 33%
• TORCH 33%
• High Latency 33%
• Low Bandwidth 33%
• Network Computation 33%
• TensorFlow 33%
• Distributed Machine Learning Systems 33%
• Distributed Stochastic Gradient Descent 33%

Computer Science

• Case Study 100%
• Communication Cost 100%
• Gradient Descent 100%
• Network Configuration 100%
• Machine Learning 50%
• Computational Complexity 50%
• Application Scenario 50%
• Distributed Machine Learning 50%

Chemical Engineering

• Learning System 100%