Accelerating vision-language models with LFM2.5-VL-DSpark
huggingface ·
LiquidAI has released an experimental DSpark draft model designed to accelerate the inference of their LFM2.5-VL-3B vision-language model through speculative decoding. This new drafter adds approximately 280 million parameters, representing only an 8.9% increase in memory footprint, while delivering significant speedups of up to 3.13x on device and 2.66x on H100 GPUs. The architecture utilizes a simplified attention-only design with four layers, trained on a mixture of vision-language data to capture hidden states for drafting candidate tokens. Day-one integration support is provided for popular inference engines including llama.cpp, MLX-VLM, and SGLang. Despite these gains, the authors note that speculative decoding primarily accelerates the decode phase, meaning end-to-end improvements are limited by the compute-intensive prefill stage required for image encoding. Benchmarks across diverse tasks such as VQA and image captioning demonstrate consistent performance enhancements on both Apple silicon and NVIDIA hardware. The models are available in Safetensors and GGUF formats, supporting LiquidAI's goal of open-weight AI deployment across various devices.
huggingface