Communication-efficient federated learning via knowledge distillation

Federated learning is a privacy-preserving machine learning technique to train intelligent models from decentralized data, which enables exploiting private data by communicating local model updates in each iteration of model learning rather than the raw data. However, model updates can be extremely large if they contain numerous parameters, and many rounds of communication are needed for model training. The huge communication cost in federated learning leads to heavy overheads on clients and high environmental burdens. Here, we present a federated learning method named FedKD that is both communication-efficient and effective, based on adaptive mutual knowledge distillation and dynamic gradient compression…

• NN
  National Natural Science Foundation of China

  Awards: U1836204, 2021ZD0113902, U1936208, U1836204, U1936216, U1936216, U1936208