New Technique Lets a Shrunken AI Model Beat Its Full-Size Original

Shrinking a large language model to run more cheaply almost always costs it something. The common approach is to first compress the model's architecture — cutting the number of layers, attention heads or neurons — and then quantize what remains, storing each weight in just 4 bits instead of 16, to cut memory and computing needs further. Both steps save resources, but together they typically dull a model's reasoning, maths and coding ability, so developers add a recovery step called "healing" before putting the compressed model into production.

A new paper describes a healing method called Quantization-Aware Healing (QAH). Applied to an open-weight model, GPT-OSS 120B, compressed to 60 billion parameters (the internal values a model learns during training) and quantized to a 4-bit format called MXFP4, QAH produced a model that beat its own full-precision, 16-bit version on 7 of 9 benchmarks — while being smaller and cheaper to run.

The standard healing method, quantization-aware training, keeps fine-tuning the compressed model on its original training objective, which is costly and can grow unstable. A newer method, quantization-aware distillation, instead trains the compressed model to copy the output of a full-precision "teacher" version of the same architecture — but once a model has also had layers or neurons removed, no genuine full-precision version of that smaller architecture exists to serve as teacher. QAH solves this by distilling directly from the original, uncompressed model instead: the teacher and the compressed student do not even share an architecture, but the teacher's output patterns can still guide the student because they do not depend on model size or shape.

The researchers say this also makes training more stable, because tying the student to a fixed teacher output removes the pressure to keep drifting that a standard training loss creates. On the AA-LCR long-context reasoning benchmark, the QAH model scored 42.7 versus 35.3 for the standard recovered version; on the AIME 2025 mathematics competition benchmark, it scored 76.3 versus 70.7.