Decentralized Federated Averaging
National University of Defense Technology · University of Utah
Abstract
Federated averaging (FedAvg) is a communication-efficient algorithm for distributed training with an enormous number of clients. In FedAvg, clients keep their data locally for privacy protection; a central parameter server is used to communicate between clients. This central server distributes the parameters to each client and collects the updated parameters from clients. FedAvg is mostly studied in centralized fashions, requiring massive communications between the central server and clients, which leads to possible channel blocking. Moreover, attacking the central server can break the whole system's privacy. Indeed, decentralization can significantly reduce the communication of the busiest node (the central…
Citation impact
- FWCI
- 39.95
- Percentile
- 100%
- References
- 79
Authors
3Topics & keywords
- Computer science
- Sublinear function
- Quantization (signal processing)
- Stochastic gradient descent
- Distributed computing
- Rate of convergence
- Convergence (economics)
- Computer network