LFM2.5-Encoders arrive to solve a very practical problem: understanding very long texts without breaking the bank or killing latency. If you work with contracts, transcripts or support threads that run into thousands of tokens, this is directly relevant to you.
Qué son y por qué importan
They are bidirectional encoders pretrained in the LFM2 family, available in two main sizes: 230M and 350M parameters. They’re designed to keep good performance on classic tasks (GLUE, SuperGLUE and multilingual classification) and, above all, to process long context — up to 8,192 tokens — with latency that grows slowly as the input gets larger.
Why does this matter? Many production apps (intent routers, security filters, PII detectors, classifiers) run all day on CPUs and receive long documents. An encoder optimized for long context can save significant costs and respond in acceptable times on common hardware.
Arquitectura y entrenamiento (resumen técnico)
The models are initialized from the LFM2 decoder backbones: LFM2.5-230M and LFM2.5-350M. To convert the causal decoder into a bidirectional encoder, concrete changes were made:
- Bidirectional attention mask: each token sees context on both sides, not just previous ones.
- Non-causal convolutions and symmetric padding: short convolutions mix neighbors to the left and right.
- Masked language modeling objective: 30% of tokens were masked during training.
Training is done in two stages:
- General competence: MLM on short context (1,024 tokens) over a large web corpus to learn language and general patterns.
- Long-context adaptation: extend to 8,192 tokens in the full data mixture to reinforce factual, legal and multilingual competence.
They report careful results: each model is fine-tuned fully for each task and the metrics are the average over five stopped seeds, which gives stable numbers.
Rendimiento: precisión y velocidad
In accuracy, LFM2.5-Encoder-350M ranks fourth among 14 models tested on 17 tasks. The three ahead are larger models (one around 3.5B). LFM2.5-Encoder-230M outperforms ModernBERT-base and several EuroBERTs, while also being smaller than most.
Latency and throughput are where they shine:
- They support 8,192 tokens. On CPU,
LFM2.5-Encoder-230Mis the fastest for all measured sequence lengths. At 8,192 tokens, ModernBERT-base takes about 90 seconds per forward, while LFM2.5-Encoder-230M takes around 28 seconds — roughly 3.7× faster. - On GPU the pattern is similar but with a smaller gap: ModernBERT leads below ~1K tokens on Apple GPU, and the LFM2.5 models take over from ~2K tokens.
Practical translation: on a laptop CPU you can classify or analyze a full contract in under 30 seconds with the 230M encoder. What doors does that open for high-volume, low-cost pipelines?
Casos de uso y demos prácticas
Liquid AI publishes demos that run on Hugging Face spaces using CPU only. Useful examples:
- Zero-shot prompt routing: compare a prompt against several routes defined in free text in a single pass.
- Zero-shot policy linting: evaluate each token against company rules written in natural language in one pass.
- Spell checking token by token.
- PII detection for 40 types of information in 16 languages.
- Masked-diffusion text generation (chatbot mode that gradually unmasks tokens iteratively).
These show the versatility: a well-tuned encoder serves for classification, extraction, scoring and routing, and is usually cheaper than using a generative LLM for understanding tasks.
Cómo empezar (ejemplo práctico)
Install the latest version of transformers:
pip install -U transformers
Minimal example of masked-token prediction:
from transformers import AutoModelForMaskedLM, AutoTokenizer
import torch
model_id = "LiquidAI/LFM2.5-Encoder-230M" # o "LiquidAI/LFM2.5-Encoder-350M"
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
mlm = AutoModelForMaskedLM.from_pretrained(model_id, trust_remote_code=True)
text = f"The capital of France is {tok.mask_token}."
enc = tok(text, return_tensors="pt")
with torch.no_grad():
logits = mlm(**enc).logits
pos = (enc["input_ids"][0] == tok.mask_token_id).nonzero()[0].item()
print([tok.decode([t]).strip() for t in logits[0, pos].topk(5).indices.tolist()])
# -> ['Paris', 'Strasbourg', 'Paris', 'Lyon', 'Versailles']
For downstream tasks, load only the backbone with AutoModel and add your own head (classification, token classification, regression, retrieval).
If your GPU supports it, install flash-attn for additional efficiency:
pip install flash-attn
Liquid AI also offers a fine-tuning tutorial for long legal documents with 8k context.
Cuándo elegir 230M vs 350M y recomendaciones de despliegue
- LFM2.5-Encoder-350M: choose this when maximum accuracy is the priority and your hardware can handle it.
- LFM2.5-Encoder-230M: pick this when you need more throughput or are constrained by CPU/memory.
If your task is high-frequency and runs all day on CPUs, a well-tuned encoder is often the cheapest and most efficient option compared to a generative LLM. For multilingual search there were already LFM2.5-Retrievers; these encoders are more general: classification, routing, extraction and security.
Reflexión final
LFM2.5-Encoders show it’s possible to combine long context, good accuracy and CPU speed without resorting to huge models. If your product needs to understand long documents at scale with low operating cost, they’re worth trying. Are you going to test them in production or do a quick run on your laptop?