Tencent Open-Sources Hy4 Preview: A 770B Flagship That Helped Train Itself
On August 28, Tencent released and open-sourced its new flagship Hy4 preview (770B total / 49B active MoE, 78 layers): Gated DSA sparse attention + IndexCache cross-layer index reuse + iHC identity Hyper-Connections, with the README openly stating the architecture is "inspired by DeepSeek and GLM". A native MTP layer enables 3-token speculative decoding, context spans 1M tokens, and BF16+FP8 weights ship under Apache 2.0. In Tencent's internal blind eval, 163 experts scored 203 engineering tasks at 2.99/4.00, edging out GLM-5.3 (2.92) and Kimi K3 (2.94, both internal-caliber numbers). The headline is the early loop of recursive self-improvement: the model took part in automating optimization of its own training methods, data strategies, eval frameworks and low-level operators, and autonomously lifted inference end-to-end throughput by 31.8%. OpenRouter snapshot pricing: $0.834 input / $2.501 output per million tokens; free for two weeks on WorkBuddy/CodeBuddy.