LFM2.5 Q4\_0 Checkpoints from Quantization-Aware Distillation
huggingface ·
LiquidAI released updated 4-bit Q4_0 GGUF checkpoints for LFM2.5 models (230M, 350M, 1.2B-Instruct, 2.6B) using Quantization-Aware Distillation (QAD), where a high-precision teacher model trains a quantized student model. These checkpoints retain the same memory and speed as native Q4_0 GGUFs while recovering about 97% of the accuracy lost to quantization. On benchmarks covering reasoning, instruction-following, tool use, and agentic tasks, the QAD checkpoints substantially outperformed post-training-quantized Q4_0 versions, retaining roughly 96–97% of their BF16 baseline performance. On edge hardware (MacBook Pro, NucBox, Galaxy S26 Ultra, Raspberry Pi 5), the QAD Q4_0 checkpoints matched the quality of larger quantized formats like Q5_K_M or Q4_K_M at higher decode throughput. The files are available on Hugging Face and can be used with llama.cpp or any runtime supporting GGUF Q4_0. The article also provides a citation reference for the work.
huggingface