llm-evaluation
Evaluate LLM performance comprehensively. Covers automated metrics (BLEU, ROUGE, METEOR, BERTScore, perplexity), LLM-as-judge evaluation, RAGAS for RAG evaluation, human evaluation frameworks, task-specific benchmarks (MMLU, HellaSwag, HumanEval, MT-Bench), safety evaluation (toxicity, bias, hallucination detection), A/B testing for LLMs, evaluation datasets and test suites, regression testing, latency and cost evaluation, multi-turn conversation evaluation, and building custom evaluation pipelines. Use when evaluating LLM quality, comparing models, testing for safety issues, or building evaluation infrastructure.
- Version
- 1.0
- License
- Apache-2.0
Pinned to revision 45cf0fa3c5e7, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/llm-evaluation/SKILL.md
- skills/llm-evaluation/references/REFERENCE.md
- skills/llm-evaluation/scripts/evaluate_llm.py
- skills/llm-evaluation/scripts/safety_eval.py
Every link opens the file at its source, pinned to the revision this page describes.