Sub-system
A model trained in sixteen-bit numbers can usually be served in eight or fewer with little measurable loss. Since token generation is bandwidth-bound, halving the bits roughly halves the work of reading weights — which makes quantisation one of the highest-return optimisations in serving.
In one sentence
Quantisation is the process of representing a model's weights, and sometimes its activations, in fewer bits than they were trained in, reducing memory footprint and bandwidth demand at some cost to numerical fidelity.
The reason it works is that neural network weights are heavily redundant and the arithmetic is tolerant of small errors. The reason it needs care is that the tolerance is uneven: most weights quantise cleanly, and a small number of outliers dominate the error if handled naively.
Practical schemes therefore quantise in groups with their own scaling factors, and often keep the few sensitive parts of the network at higher precision. Hardware support matters as much as the algorithm — a format the accelerator implements natively is fast, one that must be unpacked in software often is not.
Weights are static and can be quantised offline with careful calibration, which is where most of the memory and bandwidth saving comes from. Activations change per request and are harder, because an unusual input can produce values outside the calibrated range. Many deployments quantise weights aggressively and activations conservatively.
Post-training quantisation converts an existing model using a small calibration set — cheap, fast, and adequate for most cases down to eight bits. Quantisation-aware training simulates the lower precision during fine-tuning so the model adapts to it, which costs a training run and is what makes very low precision viable.
Aggregate benchmark scores hide selective damage: a quantised model can score the same overall while getting materially worse at one thing, often long-context reasoning or a less-represented language. Evaluating on the workload that matters, rather than on a leaderboard, is the difference between a safe optimisation and a silent regression.
Technology dependencies are solved by engineering; supply dependencies are solved by building something, which takes years.
The saving is only realised if the accelerator executes the format directly rather than converting it back in software.
AI acceleratorsQuantisation damage is selective, so a general benchmark is not sufficient evidence that a deployment is safe.
Model evaluationA format is only faster if there are tuned kernels that read it and matrix units that consume it. Without both, a smaller model is unpacked back to full precision and nothing is gained but memory.
Kernel librariesEight-bit and narrower formats are only portable between vendors because their exponent, mantissa and scaling rules are written down. Without that a quantised checkpoint would be specific to the chip it was produced for.
Other pages in this map that name Quantisation as something they cannot do without.
What each company supplies at this step, and — where a public figure exists — its share of this specific market — with what that share measures, the period it covers and who published it. Some rows also show the company’s own reported revenue for the segment covering this step, which is a different thing: it says how much this business matters to that company, not how much of the market it holds. Not a ranking and not a recommendation.
Supplies the low-precision formats in hardware and the Model Optimizer toolchain that produces them.
Supplies competing low-precision hardware support and the Quark quantisation tooling.
Owns Neural Magic through Red Hat, whose compression work feeds the open serving stack.
Supplies the on-device inference hardware where aggressive quantisation is mandatory, not optional.
Distributes the quantised checkpoints and the libraries that produce them.
Maintains the runtime and the extreme low-bit research most on-device quantisation work references.
Maintains the compression toolkits and the runtime for quantised models on its own processors.
Maintains the quantisation library inside the framework, and ships quantised weights with its releases.
Releases quantisation-aware trained weights, so the small version is not simply a degraded copy of the large one.
Trained a frontier model natively in eight-bit floating point, moving the question from post-training compression to training precision.
Defines the low-precision instructions the phone and laptop inference paths execute.
How far usable precision falls before quality loss stops being acceptable for production workloads.
Whether models are increasingly trained in low precision from the start rather than converted afterwards.
Whether hardware and format support converge on a small number of standards or stay fragmented.
Each page explains one technology in plain language, states what it depends on, and names companies by what they supply at that step. Company roles are described qualitatively and deliberately carry no market shares, revenue figures or rankings — those change faster than an explainer can, and a stale number is worse than none. Ticker links point at company pages on this site and are provided for reference only.
Nothing here is investment advice, a recommendation, or a forecast. A company named on a page about a technology is not thereby a good investment, and the chokepoints described are structural facts about supply chains rather than predictions about prices. Technology moves; where a page describes something as unresolved or in development, that was true when it was written.
Plutux is not an investment adviser. Market data and AI-generated analysis are for information and education only, not investment advice. Disclaimer