Reading the Qwen3.8-Flash-Next architecture blog felt a bit like déjà vu. Many of the ideas look familiar from Kimi K3, DeepSeek V4 and GLM-5.3-Flash, even if the implementations are different.
1. The first is hybrid attention. Qwen uses a 3:1 mix of Gated DeltaNet and Qwen Sparse Attention, similar in shape to Kimi K3’s 3:1 KDA to Gated MLA design. In both cases, most layers use a fixed-size recurrent state, with periodic attention layers for precise retrieval. The difference is that Kimi compresses KV with MLA, while Qwen uses micro-block indexing to attend only to relevant regions.
2. The same convergence is happening across residual streams. DeepSeek and GLM use widened residual streams through Hyper-Connections, Kimi uses Attention Residuals, and Qwen now uses four Gated Residual branches with dynamic reads and writes.
Qwen also continues the use of sparse MoE and MTP. MTP is particularly useful for speculative decoding. Instead of generating only one token at a time, the model is trained to predict multiple future tokens that can then be verified together. This reduces the number of sequential decode steps and improves inference throughput.
This part was interesting to me in the Qwen blog: “Empirical analysis also reveals that one of these branches naturally emerges as a long-range pathway connecting the first Attention layer to most of the middle and subsequent layers.” It is interesting to see the model learn this kind of specialization rather than having the path explicitly designed for that purpose.