Skip to content

zotoio/zoto-eval-system

v0.3.1MIT

Generate, run, and consciously update static (pytest) + LLM (@cursor/sdk) evals when AI primitives change — every proposed change requires user confirmation. YAML+JSONSchema results, surgical update on git diff, sovereign user-authored cases, /canvas-rendered cross-run compare.

zoto-configure-evals

Writes .zoto/eval-system/config.yml from field values pre-collected by /z-eval-configure (command-owned askQuestion). Refuses bundled update.preserveUserAuthoredCases:false or update.writeMetaMarker:false (including nested payload mirrors) before any manifest read or config write. Validates against templates/schema/config.schema.json. Diffs the chosen config against the manifest snapshot (discovery_config.static.framework) to produce a cleanup_plan validated against templates/schema/cleanup-plan.schema.json, and stamps _meta.primitive_analysis.invalidate=true on every cached primitive analysis when the static framework changes. The unified LLM eval harness ships one JSON-first co-located eval per target, with no per-repo selector axis — only the static framework matters here. Never calls askQuestion — returns needs_user_input if validation cannot proceed without user clarification.

Read SKILL.md at the source

Pinned to revision d79ebb356136, so it is the text this page describes rather than whatever the author pushed since.

Files

Every link opens the file at its source, pinned to the revision this page describes.