agent-evaluation
Use this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality - including improving tool selection accuracy, answer quality, reducing costs, or fixing issues where the agent gives wrong/incomplete responses. Evaluates agents systematically using MLflow evaluation with datasets, scorers, and tracing. IMPORTANT - Always also load the instrumenting-with-mlflow-tracing skill before starting any work. Covers end-to-end evaluation workflow or individual components (tracing setup, dataset creation, scorer definition, evaluation execution).
Pinned to revision 4e8964e24a5a, so it is the text this page describes rather than whatever the author pushed since.
Pre-approved tools experimental
Experimental field. Support varies between clients, so this list is what the author declared, not what your client will enforce.
- Read
- Write
- Bash
- Grep
- Glob
- WebFetch
Files
- skills/agent-evaluation/SKILL.md
- skills/agent-evaluation/LICENSE
- skills/agent-evaluation/assets/evaluation_report_template.md
- skills/agent-evaluation/references/advanced-evaluation.md
- skills/agent-evaluation/references/databricks-online-monitoring.md
- skills/agent-evaluation/references/dataset-preparation.md
- skills/agent-evaluation/references/prompt-optimization.md
- skills/agent-evaluation/references/scorers-constraints.md
- skills/agent-evaluation/references/scorers.md
- skills/agent-evaluation/references/setup-guide.md
- skills/agent-evaluation/references/throughput-guide.md
- skills/agent-evaluation/references/troubleshooting.md
- skills/agent-evaluation/scripts/analyze_results.py
- skills/agent-evaluation/scripts/create_dataset_template.py
- skills/agent-evaluation/scripts/list_datasets.py
- skills/agent-evaluation/scripts/run_evaluation_template.py
- skills/agent-evaluation/scripts/setup_mlflow.py
- skills/agent-evaluation/scripts/utils/__init__.py
- skills/agent-evaluation/scripts/utils/env_validation.py
- skills/agent-evaluation/scripts/validate_auth.py
- skills/agent-evaluation/scripts/validate_environment.py
- skills/agent-evaluation/scripts/validate_tracing_runtime.py
Every link opens the file at its source, pinned to the revision this page describes.