moe-training
Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers MoE architectures, routing mechanisms, load balancing, expert parallelism, and inference optimization.
- Version
- 1.0.0
- License
- MIT
Pinned to revision df088027ff23, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/moe-training/SKILL.md
- skills/moe-training/references/advanced-patterns.md
- skills/moe-training/references/architectures.md
- skills/moe-training/references/inference.md
- skills/moe-training/references/training.md
Every link opens the file at its source, pinned to the revision this page describes.