Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement
Its efficiency depends on the communication and computation latencies of the GPUs, which are linked to the placement of experts in the GPUs. Existing works for optimizing expert placement focus on leveraging past requests' expert activation patterns. However, they demonstrate deficiencies facing diverse and rapidly changing request patterns, calling for an online, proactive approach.
- ▪Its efficiency depends on the communication and computation latencies of the GPUs, which are linked to the placement of experts in the GPUs.
- ▪Existing works for optimizing expert placement focus on leveraging past requests' expert activation patterns.
- ▪However, they demonstrate deficiencies facing diverse and rapidly changing request patterns, calling for an online, proactive approach.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Machine Learning arXiv:2607.08782 (cs) [Submitted on 13 Jun 2026] Title:Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement Authors:Qianli Liu, Kaibin Guo, Zicong Hong, Peng Li, Fahao Chen, Haodong Wang, Jian Lin, Song Guo View a PDF of the paper titled Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement, by Qianli Liu and 6 other authors View PDF HTML (experimental) Abstract:Expert parallelism has become the prevailing paradigm to serve Mixture-of-Experts (MoE) models. Its efficiency depends on the communication and computation latencies of the GPUs, which are linked to the placement of experts in the GPUs.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
