model-evaluation
Generates python code that evaluates SageMaker models. Supports two evaluation types: LLM-as-Judge and Custom Scorer. Use when the user says "evaluate my model", "run a benchmark", "test model performance", "how did my model perform", "compare models", or other similar requests.
- Version
- 3.0.0
Pinned to revision 4e8964e24a5a, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/model-evaluation/SKILL.md
- skills/model-evaluation/LICENSE
- skills/model-evaluation/code_templates/custom_scorer_evaluator.py
- skills/model-evaluation/code_templates/llmaaj_evaluator.py
- skills/model-evaluation/references/code_output_guide.md
- skills/model-evaluation/references/create-reward-function.md
- skills/model-evaluation/references/custom-lambda-scorer.md
- skills/model-evaluation/references/custom-scorer-evaluation.md
- skills/model-evaluation/references/evaluation-type-guide.md
- skills/model-evaluation/references/llmaaj-builtin-evaluation.md
- skills/model-evaluation/references/llmaaj-custom-evaluation.md
- skills/model-evaluation/references/llmaaj-evaluation.md
- skills/model-evaluation/references/supported-judge-models.md
- skills/model-evaluation/scripts/nova_reward_function_source_template.py
- skills/model-evaluation/scripts/reward_function_source_template.py
- skills/model-evaluation/scripts/validate_custom_metrics.py
Every link opens the file at its source, pinned to the revision this page describes.