Segment
Model architecture and training: what the compute is actually building
The hardware branch explains what the machine is. This one explains what it is doing. Nearly every model in commercial use today shares one architectural ancestor, and most of the differences between them come down to how they were scaled and what was done to them after pre-training.
In one sentence
Model architecture and training covers the structure of a neural network, the recipe by which it is scaled and pre-trained on large corpora, and the post-training steps that make its outputs useful and safe to deploy.
Two facts do most of the explanatory work in this branch. First, one architecture — the transformer — turned out to scale predictably with data and compute in a way its predecessors did not, which is why the field consolidated around it. Second, a pre-trained model is not a product: it predicts continuations of text, and everything that makes it answer questions helpfully is added afterwards.
Those two facts split the spend. Pre-training is an enormous, concentrated capital expense that only a handful of organisations undertake at frontier scale. Post-training is comparatively cheap, is where most differentiation between similar models comes from, and is accessible to organisations that could never afford to pre-train.
Understanding the split is what makes claims about the field legible. A capability improvement can come from a bigger pre-training run, from a better post-training recipe, or from spending more compute at the moment of answering — and those three have very different cost structures.
How this breaks down
Split by the stage of a model's life — its structure, its pre-training, and what happens after.
- Model architecturethis page
- Model evaluationBenchmarks, judges, verifiers and the sandboxes that run generated code — how progress is actually established.Definition pageChokepoint
- TransformersAttention, why it replaced recurrence, and the cost that comes with it.Definition page
- Mixture of expertsRouting each token to a few sub-networks instead of all of them, and what that costs in memory and networking.Definition pageChokepoint
- Scaling and pre-trainingThe empirical relationships that let a training run be budgeted before it starts.Definition pageChokepoint
- Post-trainingSupervised fine-tuning, preference optimisation and reinforcement learning on verifiable tasks.Definition pageChokepoint
What this depends on
2 of these are marked as a chokepoint: a handful of qualified suppliers, a multi-year lead time, or a single geography.
- TechnologyChokepoint
Accelerated compute at cluster scale
Pre-training a frontier model is weeks of continuous work on thousands of accelerators.
AI compute - Supply chainChokepoint
Training corpora
Scaling laws demand tokens in proportion to parameters, and high-quality text is finite and increasingly licensed rather than scraped.
Training corpora - Technology
The training software stack
A model is expressed in a framework and executed through tuned kernels and a scheduler. None of the architectural work below can be run at scale without that layer underneath it.
Software stack - Resource
Research and systems staff
The recipes on these pages are held by a few thousand people worldwide, and a run that fails at week six fails on judgement rather than on hardware. Teams are bid for accordingly.
What depends on this
Other pages in this map that name Model architecture as something they cannot do without.
Companies across Model architecture
Every company named on a step below this page, ordered by how many of those steps it appears at. Compiled from the pages themselves rather than written separately, so the two cannot disagree. Not a ranking and not a recommendation.
- OpenAIPrivate5 steps
Model evaluation · Transformers · Mixture of experts · Scaling and pre-training · Post-training
- AnthropicPrivate4 steps
Model evaluation · Transformers · Scaling and pre-training · Post-training
- DeepSeekPrivate4 steps
Transformers · Mixture of experts · Scaling and pre-training · Post-training
- Mistral AIPrivate3 steps
- xAIPrivate3 steps
Transformers · Mixture of experts · Scaling and pre-training
15 more companies appear at a single step each; they are named on the pages for those steps.
How these pages are written
Each page explains one technology in plain language, states what it depends on, and names companies by what they supply at that step. Company roles are described qualitatively and deliberately carry no market shares, revenue figures or rankings — those change faster than an explainer can, and a stale number is worse than none. Ticker links point at company pages on this site and are provided for reference only.
Nothing here is investment advice, a recommendation, or a forecast. A company named on a page about a technology is not thereby a good investment, and the chokepoints described are structural facts about supply chains rather than predictions about prices. Technology moves; where a page describes something as unresolved or in development, that was true when it was written.
Plutux is not an investment adviser. Market data and AI-generated analysis are for information and education only, not investment advice. Disclaimer