writing-pack-test-cases
Writes and maintains test cases for a Felspar Decision Pack - one per outcome, both sides of every threshold, each exception and missing evidence - preferably from real past decisions, plus reading tests that check how Felspar reads different phrasings of each answer, on made-up wording and, with the person's agreement, a small sample of their real records, and agent-shaped eval sets of realistic multi-excerpt cases run against every candidate before release. Use this skill when someone wants to add, fix or review a pack's tests or examples, turn past decisions into tests, test how real wording is read, or check that a policy's edge cases are covered, including when a backtest or a live decision looks wrong or real cases keep being held. Not for drafting the pack's rules (drafting-decision-packs) or approving it.
- Version
- 0.3.3
- Compatibility
- Requires the Felspar connector (list_jobs, read_pack, read_source, add_current_version, check_pack, check_readings, propose_pack, read_evals, save_evals, run_evals, list_eval_runs, read_eval_run) and a person signed in as a workspace member, approver or owner.
Pinned to revision b11f4f7ad6a8, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/writing-pack-test-cases/SKILL.md
- skills/writing-pack-test-cases/references/eval-sets.md
- skills/writing-pack-test-cases/references/reading-tests.md
Every link opens the file at its source, pinned to the revision this page describes.