testing-llm-features
Builds an offline eval suite for LLM-powered features: golden reference cases per prompt template, deterministic scoring where possible, a regression gate on prompt, model and retrieval changes, and CI wiring with promptfoo, Vitest or deepeval. Use when a product ships prompts, agents or RAG, when a model or prompt upgrade needs a regression check, when LLM output is currently verified by someone eyeballing it, or when asked how to test a feature whose output is non-deterministic.
Pinned to revision c4efc46815a7, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/testing-llm-features/SKILL.md
- skills/testing-llm-features/resources/eval-case-design.md
- skills/testing-llm-features/resources/promptfoo-setup.md
- skills/testing-llm-features/resources/scoring-strategies.md
Every link opens the file at its source, pinned to the revision this page describes.