Kimi K3.1 Pro

Kimi K3 plus a DSA lightning indexer in its 24 gated-MLA layers: 69 KDA + 24 gated DSA (top-2,048), still no positional encoding.

Compare withDeepSeek-V4.1-Flash Qwen3.8-2.4T-A95B Inkling Kimi K3 MiniMax-M3 Nemotron 3 Ultra DeepSeek-V4-Pro Gemma 4 31B MiniMax-M2.7 GLM-5 Qwen3.5-397B-A17B gpt-oss-120b

Parameters: 2.8T total, 104B active
Context: 1M tokens (1,048,576)
Depth: 93 layers
Weights: Open, Kimi K3.1 Pro License
Released: October 2026

--- frontier-llm@2026-11+++ kimi-k3-1-pro@2026-10+0~5−2

The November 2026 LLM blueprint, marked with Kimi K3.1 Pro's differences: 19 components the same, 5 done differently, 2 not used. Select any component.

AttentionMixture of expertsResidual & normInput & outputTraining & inferenceData flowResidual / skipAddedReplacedNot usedNot disclosed

What's distinctive

Kimi K3.1 Pro is Kimi K3 with one architectural change: each of K3's 24 gated-MLA layers (every fourth layer, plus layer 93) gains a DeepSeek Sparse Attention (DSA) lightning indexer, so MLA attends to the 2,048 highest-scoring earlier tokens instead of the whole prefix. Moonshot calls these layers Gated DSA. The indexer has 32 heads of width 128, projects its queries from MLA's 1,536-dim query latent, computes one shared 128-dim key per token followed by a LayerNorm, and uses no positional encoding (index_rope_dim = 0), so the model stays NoPE end to end. The 69 KDA layers are untouched.

Everything else matches K3 tensor for tensor: Block Attention Residuals over 12-layer blocks, Stable LatentMoE (16 of 896 SiTU-GLU experts in a 3,584-dim latent, plus two shared experts), Quantile Balancing, MoonViT-V2, the 160K tokenizer and MXFP4 routed experts. The weight index differs from K3's only by five indexer tensors in each DSA layer: 178.5M BF16 parameters, about 0.006% of the 2.78T total.

Moonshot describes K3.1 Pro as post-trained from K3 with large-scale joint RL across hundreds of domains and substantially more post-training compute, and says cyber-related data was removed from both the mid-training corpus and the RL environments for the open-weight release. There is no K3.1 technical report and the checkpoint ships no modeling code, so how the indexer was trained (warm-up, distillation target, token budget) is unpublished; its behaviour here is taken from config.json, the weights and the vLLM and SGLang implementations. The API launched on 24 October 2026; the weights reached Hugging Face on 2 November.

How it differs (7)

Sources

Checked against these sources in November 2026.

Other models