Federated Learning
Federated Learning
What is federated learning?
- Collect results, not data
- Originals stay on the device
- Privacy-preserving training
Adopting federated learning shifts the project’s bottleneck. Before, you might spend months with the legal and information security teams over “can this data leave the premises?” Now the questions are whether each institution and device has the compute to run training, and whether the network can handle tens to hundreds of MB of weights moving back and forth every round. What you negotiate over shifts from the data to the model, and that is the real change. That is why months of schedule lost to export reviews can vanish altogether.
In practice, it splits into two main types. In cross-device setups, millions of smartphones are registered but only a few hundred to a few thousand are picked each round. In cross-silo setups, a fixed group of 2 to 20 institutions, such as hospitals or banks, take part. In the first type, training runs only while a device is charging and on Wi-Fi, so participation is uneven. In the second, there are fewer participants, but differences in data volume and labeling standards across institutions drive performance. Aggregation usually uses a weighted average proportional to data size (FedAvg), but when each institution has a different data distribution (non-IID), this simple average can actually damage the model.
The most common misconception is that 'if the raw data never leaves, it isn't personal data processing.' Model updates retain traces of the training data, and several studies have reported reconstructing a substantial part of the original images or sentences from gradients alone. In a consortium of just three participating institutions, re-aggregating without one institution's update can even reveal that institution's data characteristics. Unless secure aggregation or differential privacy is added on top, federated learning is not anonymization but personal data processing that has simply changed routes. And if you run rounds without aligning labeling standards, you can end up with a model that performs worse than one trained alone, which can sink the whole project.
Sungil KimA KAIST and Samsung Electronics alumnus and global pitching expert who coached the winning teams at the UN Tourism and Select USA IR pitch competitions
Jaryong KooA marketing PhD and leading authority on AI, data analytics, and personal brandingProposed by Google around 2016, this approach combines distributed on-device training without centralizing the data.
Consider a hypothetical case. Company A, a medical foundation with 1,400 employees, set out to build a chest X-ray reading-support model across its 3 affiliated general hospitals. Each hospital had 180,000 to 260,000 images, but pooling them in one place would have meant 5 months of data-export review alone. The service launch target was 6 months away.
- Kim, who leads the medical information office, scrapped the plan to collect the images. The IRB application was rewritten around a structure where the originals stay on each hospital’s GPU server and only the weights are exchanged once a week.
- In the first two rounds, only Hospital B’s model scored 12 percentage points lower in accuracy. On inspection, it was labeling readings on a 5-grade scale instead of the 3-grade scale.
- Two heads of radiology spent four weeks together standardizing the label definitions. The team also changed the server aggregation rule to a weighted average proportional to each hospital’s image count, which removed the skew toward the small Hospital C.
- The information security officer asked, “If you only send updates, can patient information really not be reconstructed?” So the team added secure aggregation and noise injection and locked the setup so that aggregation happens only when all 3 institutions take part in a round.
- Six months later, the reading-support model’s sensitivity was up 8.4 percentage points over single-hospital training. And because the 5-month data-export review disappeared from the schedule, the launch target was met two weeks early.