Skip to content

zotoio/zoto-eval-system

v0.3.1MIT

Generate, run, and consciously update static (pytest) + LLM (@cursor/sdk) evals when AI primitives change — every proposed change requires user confirmation. YAML+JSONSchema results, surgical update on git diff, sovereign user-authored cases, /canvas-rendered cross-run compare.

zoto-judge-evals

Adversarial judge for eval runs. Identifies weak graders, under-covered assertions, verbosity spikes, and accuracy/confidence regressions. Produces an enriched llm.yml with soft-metric annotations. Does not call askQuestion — handoff to /z-eval-update is expressed as needs_user_input for the command to confirm via askQuestion and resume. Uses the configured judgeModel (default claude-opus-4-8[]).

Read SKILL.md at the source

Pinned to revision d79ebb356136, so it is the text this page describes rather than whatever the author pushed since.

Files

Every link opens the file at its source, pinned to the revision this page describes.