hyperpod-nccl
Diagnose NCCL failures and adjacent training-pod failures on HyperPod GPU clusters (EKS or Slurm) — training hangs, AllReduce / collective-op timeouts, EFA or libfabric errors, rendezvous failures, EFA TCP fallback, /dev/shm or memlock issues, NCCL version mismatch across pods, container OOM / exit-137 / OOMKilled, GPU OOM (CUDA out of memory), CrashLoopBackOff / Pending pods, MASTER_ADDR DNS, NetworkPolicy blocking. Not for single-node hardware faults (→ hyperpod-node-debugger § G) or cluster-creation EFA / SSM failures (→ hyperpod-cluster-debugger § A / § F).
- Version
- 0.0.1
Pinned to revision 4e8964e24a5a, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/hyperpod-nccl/SKILL.md
- skills/hyperpod-nccl/LICENSE
- skills/hyperpod-nccl/references/debugging-guide.md
- skills/hyperpod-nccl/references/error-patterns-quick-ref.md
- skills/hyperpod-nccl/references/operations.md
- skills/hyperpod-nccl/references/performance-testing.md
- skills/hyperpod-nccl/scripts/nccl-diagnose.sh
Every link opens the file at its source, pinned to the revision this page describes.