According to CoinMeta, Tencent's Hunyuan Hy4 has been rebuilt from the ground up, incorporating a combination of DeepSeek sparse attention and Zhipu IndexCache. This reconstruction not only increases the scale of the model but also makes significant improvements to the underlying architecture. Hy4 adopts gated sparse attention (gated DSA), which allows the model to prioritize processing the most relevant parts when dealing with extremely long texts without having to recalculate the entire content. In addition, Hy4 introduces IndexCache, enabling some layers to reuse previously selected important information and reducing redundant calculations. Tencent states that this design was inspired by DeepSeek and GLM. The residual structure of Hy4 has also been modified, adopting a IH structure that expands into four main information pathways, allowing for the retention of more information within the deep network. The number of experts in MOE has increased from 192 to 256 routing experts and 1 shared expert; each token selects 8 routing experts, while also passing through the shared expert.