Systems Engineering Playbook: Optimizing Qwen 3.5-397B MoE on Ironwood (TPU7x)
To optimize this track, the performance team implemented a series of algorithmic fusions and precision co-designs:1. Causal Conv1D FusionThe GDN recurrent update is preceded by a causal 1D convolution (K=4). Initially, this was compiled as an independent operation, forcing the intermediate convolution outputs to be written to and read from HBM.
- ▪To optimize this track, the performance team implemented a series of algorithmic fusions and precision co-designs:1.
- ▪Causal Conv1D FusionThe GDN recurrent update is preceded by a causal 1D convolution (K=4).
- ▪Initially, this was compiled as an independent operation, forcing the intermediate convolution outputs to be written to and read from HBM.
Google Developers Blog files mainly under programming. We currently carry 20 of its stories.
Opening excerpt (first ~120 words) tap to expand
To optimize this track, the performance team implemented a series of algorithmic fusions and precision co-designs:1. Causal Conv1D FusionThe GDN recurrent update is preceded by a causal 1D convolution (K=4). Initially, this was compiled as an independent operation, forcing the intermediate convolution outputs to be written to and read from HBM. We designed a register-level sliding window algorithm that caches historical token states directly within the TPU's VPU registers. Fusing the 1D convolution and the GDN recurrent state update into a single execution block eliminated 6 redundant HBM round-trips (see PR #2823).2. Algebraic Identity OptimizationsWe restructured the linear attention update equations to exploit algebraic identities.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Google Developers Blog.
Discussion
0 commentsMore from Google Developers Blog
