Liquid AI introduces LFM2.5-DSpark, a new speculative decoding implementation that promises to speed up text generation by up to 3.18x on an H100 GPU and up to 2.87x on a MacBook with an M4 Max chip. The approach aims to make language models respond faster without changing the output produced by the original model.
Is the idea to replace the main model? No. DSpark uses a small auxiliary model to propose several tokens and lets the main model verify them in a single pass. This reduces the number of times the model weights need to be loaded from memory, one of the main speed limits during the decoding phase.
How Speculative Decoding Works
During traditional generation, a model produces one token, runs the process again, and generates the next. This cycle can be costly because inference is often limited by data movement between DRAM and the processor’s faster memory, not just by computing capacity.
Speculative decoding changes the flow. An auxiliary model, known as a drafter, proposes several tokens. Then, the target model analyzes them in a single pass and accepts the ones that match its distribution. If it rejects one, it uses its own token and continues generating.
With greedy decoding, DSpark preserves the target model’s output exactly. The speedup does not depend on sacrificing accuracy to generate faster.
DSpark combines three main components:
- A parallel block inspired by DFlash, conditioned on the target model’s internal representations and capable of producing hidden states for several tokens in a single execution.
- A lightweight sequential head, modeled as a Markov chain between neighboring tokens, which incorporates dependencies between positions and improves the acceptance of subsequent tokens.
- A confidence-scheduled verifier that estimates each token’s survival probability and removes low-confidence suffixes when verifying them would cost more than they save.
Auxiliary Models with Around 300 Million Parameters
Liquid AI trained the DSpark models with a broader mixture of data, including supervised fine-tuning, conversations, code, and function calls. The first versions use simplified architectures based mainly on attention, with five layers and a nine-token block.
Each model was trained for 15 epochs on the full dataset. Instead of choosing the point with the lowest loss, the team selected the epoch with the highest acceptance rate, a decision consistent with the goal of speeding up verification.
The resulting auxiliary models are much smaller than their target models:
- LFM2.5-1.2B-Instruct-DSpark: 295.7 million parameters.
- LFM2.5-8B-A1B-DSpark: 327.7 million parameters.
- LFM2.5-2.6B-DSpark: 327.7 million parameters.
The auxiliary model does not need to be the same size as the main model. Its job is to quickly propose candidates so the target model can verify several tokens at the same time.
Results on H100 and MacBook M4 Max
Liquid AI evaluated the models with SGLang on an 80 GB NVIDIA H100 GPU using BF16 precision. For on-device tests, it used llama.cpp, Metal, and GGUF weights in FP16 on a MacBook Pro with an M4 Max chip.
The tests used a DSpark block size of nine, a batch size of one, a temperature of zero, and up to 256 output tokens. The evaluated datasets were MATH500, HumanEval, MBPP, GSM8K, and MT-Bench.
For the LFM2.5-2.6B model, the average speedup was 2.67x on the H100, increasing from 323 to 864 tokens per second. On the MacBook M4 Max, the average improvement was 2.27x, rising from 61 to 139 tokens per second.
| Dataset | Average Acceptance out of 10 | H100 | M4 Max |
|---|---|---|---|
| MATH500 | 5.42 | 3.06x | 2.25x |
| HumanEval | 4.54 | 2.56x | 2.63x |
| MBPP | 4.71 | 2.64x | 2.11x |
| GSM8K | 4.32 | 2.22x | 2.36x |
| MT-Bench | 5.07 | 2.87x | 1.99x |
| Average | 4.81 | 2.67x | 2.27x |
The result is significant for running assistants locally. A speed close to 139 tokens per second can make a conversational application feel much more immediate, even without relying entirely on a cloud service.
Differences Between the Three Models
The LFM2.5-1.2B-Instruct model achieved an average speedup of 2.10x on the H100 and 2.54x on the M4 Max. However, Liquid AI warns that its results vary more across datasets. The improvement can change by as much as 52%, depending on the text distribution.
The LFM2.5-8B-A1B case highlights an important limitation. On the H100, it achieved an average speedup of 2.54x, with a maximum of 3.18x on MATH500. On the M4 Max, the average improvement was just 1.18x.
The reason is related to its mixture-of-experts architecture, known as MoE. In the current llama.cpp implementation for Metal, verifying multiple tokens can activate more experts and move more weights from memory than conventional generation. That is why higher acceptance does not always automatically translate into greater speed on a device.
Lower Latency for Tool-Using Agents
One of the most interesting results appears in function-calling scenarios. In tests with multiple tools, DSpark reduced latency by an average of 57% for the LFM2.5-2.6B model.
This can benefit agents that need to query a database, run code, check the weather, or call an API before responding. In these systems, every additional token and every reasoning turn can add noticeable delays.
Does that mean every agent will be 57% faster? Not necessarily. The result depends on the model, the tools, the length of the responses, and the acceptance rate. Even so, it points to a practical use: speeding up local agents without changing the target model’s behavior.
Compatibility with llama.cpp and SGLang
DSpark models come with day-one support for llama.cpp and SGLang. Liquid AI published the integrations in the official repositories through PR #27383 for llama.cpp and #31041 for SGLang.
To run the model with SGLang, you need a build with DSpark support and a target model paired with its auxiliary model:
python -m sglang.launch_server \\
--model-path LiquidAI/LFM2.5-2.6B \\
--speculative-algorithm DSPARK \\
--speculative-draft-model-path LiquidAI/LFM2.5-2.6B-DSpark \\
--speculative-draft-attention-backend flashinfer \\
--disable-radix-cache --mem-fraction-static 0.75 --port 30000
The server exposes an endpoint compatible with the OpenAI API at http://localhost:30000/v1. You can obtain the baseline by running the same command without the speculative decoding options.
In llama.cpp, the equivalent configuration uses one GGUF file for the target model and another for the auxiliary model:
llama-server -m LFM2.5-2.6B-F16.gguf \\
-md LFM2.5-2.6B-DSpark-F16.gguf \\
--spec-type draft-dspark --spec-draft-n-max 10 --spec-draft-n-min 0 \\
-fa on -ngl 99
The actual block size is obtained from the model configuration or from the auxiliary file’s metadata. The per-response metrics also report how many tokens the auxiliary model proposed and how many were accepted.
Models Available for Download
The checkpoints are available on Hugging Face in Safetensors and GGUF formats:
GGUF versions prepared for workflows with llama.cpp are also available. As always, performance figures should be treated as reference points: hardware, the backend, quantization, context size, and text type can significantly change the result.
LFM2.5-DSpark does not eliminate the cost of running a large model, but it tackles one of inference’s most important bottlenecks: memory. The possibility of achieving improvements close to 3x on a GPU and more than 2x on a laptop reinforces a clear trend: local AI does not depend only on smaller models, but also on smarter execution techniques.
For developers building agents, coding applications, and offline assistants, that difference can turn a response that feels slow into a genuinely interactive experience.