Liquid AI has introduced new Q4_0 checkpoints for its LFM2.5 model family, trained with a technique designed to address one of quantization’s common challenges: reducing memory usage and increasing speed without sacrificing too much quality. The proposal is aimed especially at running AI models on devices with limited resources.
What quantization-aware distillation brings to the table
The technique is called Quantization-Aware Distillation, or QAD. In simple terms, a high-precision model acts as a teacher and transfers its capabilities to a student model that is already designed to work with quantized weights.
Why does that matter? When a model is converted to Q4_0, its parameters are represented with fewer bits. This reduces memory consumption and can speed up inference, but it can also cause the model’s responses to decline in quality.
With QAD, training takes that reduction in precision into account from the beginning. This allows the model to learn to compensate for part of the loss, rather than trying to recover it later through post-training quantization, known as PTQ.
The central idea is simple: teach the model to live with fewer bits instead of simply compressing it at the end.
Models recover up to 97.4% of performance
Liquid AI compared the QAD Q4_0 checkpoints with their GGUF versions quantized through PTQ. The tests covered reasoning, instruction following, tool use, and agentic capabilities.
The evaluation suite included GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4. It also included a math benchmark adjusted to each model’s size: GSM8K for LFM2.5-230M and LFM2.5-350M, and AIME25 for LFM2.5-1.2B-Instruct and LFM2.5-2.6B.
The results show that the four QAD models retained between 96.5% and 97.4% of the performance of their respective BF16 versions:
- LFM2.5-230M: 97.1%
- LFM2.5-350M: 96.5%
- LFM2.5-1.2B-Instruct: 97.4%
- LFM2.5-2.6B: 96.6%
The figures correspond to the average of five repetitions. In this analysis, the BF16 GGUF serves as the maximum-quality reference within the same format.
More speed without increasing memory usage
The practical advantage of these checkpoints is that they retain the main characteristics of Q4_0: a smaller memory footprint and high decoding performance. This makes them attractive for small computers, phones, and edge devices.
Liquid AI measured generation speed on four platforms:
- MacBook Pro
- NucBox EVO-X2
- Samsung Galaxy S26 Ultra
- Raspberry Pi 5
GPU inference was used on the first two devices. On the Samsung phone and Raspberry Pi 5, the tests were run with Arm CPUs. BF16 and F16 versions were included as full-precision references when available.
The results are also competitive with heavier formats. For the LFM2.5-230M and LFM2.5-350M models, the QAD Q4_0 checkpoints achieved quality similar to Q5_K_M, with 4% to 33% higher decoding performance.
For LFM2.5-1.2B and LFM2.5-2.6B, quality matched Q4_K_M, while speed was between 3% and 14% higher. Where applicable, they also matched the performance of Unsloth’s UD-Q4_K_XL, another post-training quantization checkpoint with strong results.
Models available on Hugging Face
The QAD GGUF files are already available for four models in the LFM2.5 family:
LFM2.5-230MLFM2.5-350MLFM2.5-1.2B-InstructLFM2.5-2.6B
You can use them with llama.cpp or any runtime compatible with Q4_0 GGUF artifacts. For example, to run the 350-million-parameter model with llama-cli:
llama-cli -hf LiquidAI/LFM2.5-350M \\
--hf-file LFM2.5-350M-QAD-Q4_0.gguf \\
-p "What is C. elegans?"
The command downloads the specified model from Hugging Face and generates a response to the query. On a device such as a Raspberry Pi 5 or a modern phone, this type of format can make the difference between running a model locally and relying on a cloud API.
Why this matters for local AI
Quantization is often presented as a choice between quality and efficiency. If you want to save memory, you accept some loss of precision. If you want better responses, you use larger formats and consume more resources. QAD attempts to shift that balance.
This does not mean that a Q4_0 model will always outperform a BF16 one. Liquid AI’s own comparison uses BF16 as the quality ceiling. The proposal is more specific: preserve a large part of that capability while maintaining the size and speed of a quantized model.
This could benefit developers building offline assistants, coding tools for portable devices, applications with local privacy, or embedded systems. Do you need a data center to test a small model? With these formats, the answer is becoming less obvious.
The release also points to an important direction for compact model development: optimizing the architecture and training with the target hardware in mind from the start. In practice, local AI does not depend only on having smaller models, but on training them to work well under real-world memory, precision, and speed constraints.