Deepseek open-sources the inference engine that powers its fastest and cheapest AI model
decoder ·
Deepseek has open-sourced DeepInfer, its internal inference framework that powers the V4.1-Flash model, under the Apache License 2.0. This serving software enables the AI to generate responses quickly and at scale by building on vLLM with optimized compute kernels and distributed processing. The release also includes deepseek-recipe, which replicates token processing to ensure self-hosted setups behave identically to the official API. By providing these tools for free, Deepseek aims to help the community lower compute costs and reduce energy consumption. The move significantly reduces the expense of running their already affordable models, filling a gap in cost efficiency. Consequently, this development adds pricing pressure on Western competitors such as OpenAI and Anthropic. Given the ongoing technological rivalry between China and the US, the release may also carry political motivations aimed at impacting Western economic growth targets.
decoder