Moore Threads adds support for Kimi K3.1 Pro on MTT S5000 GPU

Alibaba Cloud announced that its Lingjun Zhenwu M890 SuperPoD system has completed the adaptation of the Kimi K3.1 Pro model from Moonshot AI in day-zero mode. A full check of the stack on vLLM and SGLang inference frameworks was also conducted. This was reported on November 5th by IT Home, citing the cloud provider.

The company also opened invite-only testing for a new SuperPoD region in Zhongwei, Ningxia, expanding the list of locations where large open-weight models can be hosted on the company's proprietary accelerators.

Model Characteristics and Joint Adaptation

The Kimi K3.1 Pro model is a sparse mixture-of-experts model with 2.8 trillion parameters and a context window of approximately one million tokens. It utilizes Kimi Delta Attention and Sparse Attention DSA hybrid attention, along with multi-token predictive capability. These features create significant demands on memory bandwidth and high-speed interconnects for scaling.

Alibaba Cloud, T-Head, and the Kimi team jointly adapted this model on T-Head's SAIL software stack. Work on FlashMLA, DeepGEMM MoE, MegaMoE, and MoonEP operators allowed customers to deploy inference either on bare-metal SuperPoD instances or via the REIN intelligent engine service. Furthermore, the model is available through Alibaba's Qwen AI platform API.

Framework Support and New Capabilities

Independent day-zero support reports from SGLang and vLLM projects describe the support for Zhenwu PPU for Kimi K3.1 Pro on M890 and 810E configurations. This release is noted as the first support for a major open model within the hardware backend in the Zhenwu development tree. Alibaba Cloud documentation publishes prefill-decode recipes for Zhenwu M890 SuperPoD instances, including settings for tensor and expert parallelism across multiple nodes.

The invitation to the SuperPoD in Zhongwei includes T-Head's Panmai 920 intelligent network card layered over previously existing high-performance configurations. Alibaba claims that this card uses PCIe 5.0 and 112G PAM4 Ethernet, achieving a peak throughput of 400 Gbps and 400 million packets per second. It supports multi-path RDMA and programmable congestion control, and integrates a PCIe switch on the card itself, allowing GPUs and SSDs to connect with less reliance on an external switch—a design that, according to the company, can reduce system costs by about 30%.

Day-zero SuperPoD instances for large models are available in data centers in Ulanbaatoi and Zhongwei. Alibaba Cloud notes that Lingjun Zhenwu M890 SuperPoD instances already serve over 100 major clients in the fields of large models, autonomous driving, and embodied AI. The company also reiterated that the next-generation Zhenwu V900 SuperPoD instance is expected to provide approximately three times the performance compared to the current generation. This announcement coincides with separate day-zero support for Kimi K3.1 Pro from Huawei Ascend, which was reported the same week, providing two domestic launch paths for the model on SuperPoD stacks.

The Huawei Ascend computing systems team announced the completion of adaptation for inferencing the Kimi K3.1 Pro model, which has 2.8 trillion parameters. The full weights of this model were released by Moonshot AI on November 4, according to Huawei Computing's official account, as reported by IT Home and ifeng Tech.

Using vLLM Ascend and SGLang, developers can now deploy this model on Atlas 800 A3 servers, in the Atlas 900 A3 SuperPoD cluster, and in the Ascend 950 SuperPoD series since its release.

Kimi K3.1 Pro is the first model in Moonshot AI's K3.1 series. It is a mixture-of-experts model with native visual processing and a context window of about one million tokens, activating approximately 104 billion parameters at each inference step. The model retains the hybrid linear attention K3, based on Kimi Delta Attention (KDA) layers, and adds sparse attention layers DSA instead of some full attention layers to reduce the cost of modeling long context.

Most of the adaptation work was related specifically to this architecture. Huawei stated that its inference team relied on the previous adaptation of Kimi K3, pre-analyzing the new model and working in parallel on model integration, operator development, and service validation. Thanks to existing support for KDA and MoE, the team added a new sparse attention path and multimodal inference while preserving existing serving functions in the inference frameworks.

According to Huawei, the optimization covered core operators for historical information retrieval, sparse attention computations, preprocessing, and residual computations, which were combined with distributed caching and framework-level functions to support long-context inference and production serving. The company aimed for users and developers to be able to start testing applications and deployment on the first day of the model's release.

vLLM Ascend documentation indicates that Kimi K3.1 Pro is supported in experimental mode in version vLLM Ascend 0.30.0. Its reference configurations use prefix and decoding splitting across eight nodes with four prefix nodes and four decoding nodes, including a long-context configuration of one million tokens. Atlas 800I A3 servers operate with the quantized checkpoint W4A8, while 950DT products use the W4A8 MXFP variant; the guide notes that Atlas A2 and 950PR products are not yet supported.

The model size makes hardware support a practical issue. IT Home notes that weights in BF16 format take up about 5.6 TB and still require about 1.4 TB when quantized to INT4, ruling out the use of a single consumer GPU. Huawei positioned its A3 and 950 SuperPoD systems for this class of trillion-parameter MoE inference, and the release of Kimi K3.1 Pro was the last open model it supported on launch day.

Moore Threads has made the Megatron-MUSA-Titan project publicly available, presenting an infrastructure for training large models that it developed for use on its own graphics processors. This project is designed to work with trillion-parameter 'mixture-of-experts' (MoE) models on clusters consisting of over 10,000 Moore Threads GPUs, covering both pre-training and reinforcement learning.

The project code is available on GitHub. The company claims that this system was continuously used for pre-training a trillion-parameter MoE on 10,000-GPU clusters, where long training sessions ensured high FLOPs utilization and process stability.

The repository includes an example configuration for a Kimi-K2 style MoE model with 1 trillion parameters. A central design principle was maintaining closeness to existing solutions. Megatron-MUSA-Titan is built upon NVIDIA's Megatron-LM framework. Through integration of PyTorch via torch_musa and compatibility of MUSA C with CUDA C, developers with the MUSA environment can clone official Megatron-LM, slime, and miles repositories and run training on Moore Threads GPUs without modifying the source code.

The project indicates that general algorithms and bug fixes should be directed to upstream projects, while Moore Threads focuses on two types of changes: large-scale optimizations adapted for Moore Threads hardware and clusters, and workarounds for specific MUSA issues.

Pre-training features include block training in FP8 format, which relies on Moore Threads' MT-TransformerEngine. MoE token dispatching and merging are accelerated by DeepEP, running on the company's Async Copy Engine, which also allows expert communication parallel to computations. Other options include activation offloading using virtual pipeline parallelism and scheduling one forward and one backward pass, as well as pipeline parallel communication combined with computation, thanks to Mooncake RDMA.

A distributed tracing analysis tool reads PyTorch traces from each GPU to detect bottlenecks such as pipeline bubbles and low-performance nodes. The project states that it can analyze a 1-trillion-parameter MoE run on 10,000 GPUs in just a few hours.

For reinforcement learning, the stack supports the slime and miles frameworks; Megatron handles the training, and the MUSA build of SGLang generates runs, allowing one base system to serve both pre-training and RL. Optimization for autonomous inference is not within the scope of this project.

For developers considering domestic accelerators, the practical question is what part of the existing training code will remain functional. Moore Threads' answer is that the Megatron ecosystem code requires no changes.