DeepSeek has released DeepInfer, an inference execution framework that forms the basis of its large model services. This release occurred on October 20th.
DeepInfer supports Nvidia GPUs as well as Huawei Ascend processors. The framework provides several features, including distributed inference, speculative decoding, and structured output.
According to DeepSeek's statement, the goal of this release is to help developers and enterprises increase throughput and reduce latency while decreasing computational costs and energy consumption.
Furthermore, the company made the deepseek-recipe component publicly available. It is responsible for transforming API requests and handling model input and output in accordance with DeepSeek's proprietary services. Currently, DeepInfer supports the DeepSeek-V4.1-Flash-1017 model, and the company intends to continue releasing inference code and optimizations for future models through this project.
