Slides

Youtube

Motivating Examples

What is federated learning

Federated learning ¹,² is a kind of distributed learning.

How does federated learning differ from traditional distributed learning?

1. Users have control over their device and data.
2. Worker nodes are unstable.
3. Communication cost is higher than computation cost.
4. Data stored on worker nodes are not IID.
5. The amount of data is severely imbalanced.

Let us recall parallel gradient descent

Federated Averaging Algorithm

Computation vs. Communication

References

¹. McMahan and others: Communication-efficient learning of deep networks from decentralized data. In AISTATS, 2017. ². Konevcny, McMahan, and Ramage: Federated optimization: distributed optimization beyond the datacenter. arXiv:1511.03575, 2015