evaluating-llms-harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
- Version
- 1.0.0
- License
- MIT
Pinned to revision df088027ff23, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/evaluating-llms-harness/SKILL.md
- skills/evaluating-llms-harness/references/api-evaluation.md
- skills/evaluating-llms-harness/references/benchmark-guide.md
- skills/evaluating-llms-harness/references/common-issues.md
- skills/evaluating-llms-harness/references/custom-tasks.md
- skills/evaluating-llms-harness/references/distributed-eval.md
- skills/evaluating-llms-harness/references/hardware-requirements.md
Every link opens the file at its source, pinned to the revision this page describes.