arXiv:2605.27358: MobileMoE brings Mixture-of-Experts architecture to mobile devices with 2-4× fewer FLOPs and up to 3.8× faster inference than dense models
A research team from Meta AI and Qualcomm (Yanbei Chen, Hanxian Huang, and collaborators) introduced MobileMoE — a family of on-device MoE language models with 0.3-0.9B active parameters and 1.3-5.3B total parameters. The architecture achieves 2-4× fewer inference FLOPs and up to 60% fewer parameters than comparable alternatives, with 1.8-3.8× faster prefill and 2.2-3.4× faster decoding on mobile hardware.
This article was generated using artificial intelligence from primary sources.
A research team comprising Yanbei Chen, Hanxian Huang, Ernie Chang, Jacob Szwejbka, Digant Desai, Zechun Liu, Vikas Chandra, and Raghuraman Krishnamoorthi introduced MobileMoE — the first systematic investigation of Mixture-of-Experts (MoE) architectures at the sub-billion parameter scale, specifically optimized for mobile devices.
Why has MoE architecture not been available on mobile devices until now?
Mixture-of-Experts (MoE) is an architectural approach that activates only a subset of specialized “experts” (expert networks) in each layer rather than the entire network. This selective activation delivers major computational savings, but existing MoE models are designed primarily for data centers with abundant memory and powerful GPUs.
Mobile devices have strict memory and computational constraints. The research fills the gap by exploring MoE at the sub-billion parameter scale (0.3-0.9B active parameters, 1.3-5.3B total) with actual inference speed profiling on mobile hardware — rather than purely theoretical FLOP estimates.
What are the concrete results of MobileMoE models?
The MobileMoE family achieves impressive metrics compared to dense baseline models of the same size:
Inference efficiency: MobileMoE models achieve 2-4× fewer inference FLOPs while retaining performance. On mobile hardware, the MobileMoE-S model achieves 1.8-3.8× faster prefill and 2.2-3.4× faster decoding.
Parameter efficiency: MobileMoE achieves comparable results to state-of-the-art MoE alternatives with up to 60% fewer total parameters.
What is the optimal MoE design for mobile devices?
A key contribution of the research is deriving an on-device scaling law that identifies the optimal architectural pattern for mobile deployment: moderate sparsity with fine-grained and shared experts. This design simultaneously balances the memory and computational constraints of mobile devices.
MobileMoE model training progresses through four stages: pre-training, mid-training, instruction fine-tuning, and quantization-aware training using open-source datasets. This comprehensive pipeline ensures models are optimized not only for accuracy, but also for efficient execution on quantized mobile hardware.
The results open the prospect of private and offline AI assistants executing demanding language tasks directly on smartphones, without sending data to the cloud.
Frequently Asked Questions
- What is MobileMoE and what sets it apart from existing MoE models?
- MobileMoE is a family of Mixture-of-Experts language models designed for mobile devices with sub-billion active parameters (0.3-0.9B active, 1.3-5.3B total). Unlike existing MoE models designed for data centers, MobileMoE explores MoE architectures at sub-billion scale with actual profiling on mobile hardware.
- How much faster are MobileMoE models than dense baseline models on mobile devices?
- The MobileMoE-S model achieves 1.8-3.8× faster prefill and 2.2-3.4× faster decoding than dense baseline models. Additionally, MobileMoE models achieve 2-4× fewer inference FLOPs.
- What are the key conclusions about optimal MoE model design for mobile devices?
- Researchers derived an on-device scaling law identifying the optimal pattern: moderate sparsity with fine-grained and shared experts that balance the memory and computational constraints of mobile devices.
📬 AI news in your inbox
A daily digest built your way — pick topics, sources and cadence. One-click unsubscribe.