// report · 2026-05-04

The Real Limits of Distributed LLM Training: An Architectural Postmortem

Sterling Morrison · GDF Research · DOI 10.5281/zenodo.20018909

Abstract

A federated, peer-to-peer LLM training network — its architecture, mechanisms, and the quantitative reasons centralized training still wins for frontier models. Reproducible scripts for bandwidth, straggler, convergence, cost, and poisoning analysis included.

The question

Can a federated, peer-to-peer network of volunteer machines train a competitive language model?

What it covers

The report walks through the architecture and mechanisms of such a network, then works through bandwidth, stragglers, convergence, cost and data poisoning to show where it breaks down against centralized training for frontier-scale models. Reproducible scripts for each analysis are in the repository.

Limitations

  • An architectural analysis with modeled costs, not a training run at frontier scale.

Cite

Morrison, S. (2026). The Real Limits of Distributed LLM Training: An Architectural Postmortem. Zenodo. https://doi.org/10.5281/zenodo.20018909

@misc{morrison2026distributed,
  author    = {Morrison, Sterling},
  title     = {The Real Limits of Distributed LLM Training: An Architectural Postmortem},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.20018909},
  url       = {https://doi.org/10.5281/zenodo.20018909}
}