Changelog

Unreleased

Every command now runs under a Python that has the packages it needs. On a Windows machine where python3 and python were different installs and only python had pytest, /eval-prove stopped with CANNOT PROVE: pytest is not installed and /eval-init set the test-count floor to 1 because collection failed. core/py.sh now prefers a Python 3.10+ with pytest and PyYAML, then one with pytest, then any Python 3.10+ as before.

Output is UTF-8: em dashes printed by the tools reached Claude Code on Windows as U+FFFD.

• core/py.sh: three-pass interpreter choice (importlib.util.find_spec, no imports); PYTHONIOENCODING=utf-8 unless already set.
• tests/test_py_sh.py: a bare-venv python3 next to a python with pytest; /eval-prove proves, /eval-init counts the real suite, output has no U+FFFD.

1.4.3 — 2026-10-20

• /conductor --verify-evidence no longer reads or sends any credential. 1.4.2 read GITHUB_TOKEN from the installer's environment and sent it to the GitHub API; the directory policy check correctly held that as a credential forward. The check is now anonymous: public repos only, and a private repo, rate limit or offline run fails closed with a message that says so. core/gh_auth.py is gone. Nothing else in the plugin touches the network.
• PRIVACY.md now states the one opt-in network call instead of claiming none.

1.4.2 — 2026-10-20

Activation fixes: every command now runs on Windows, nothing strands a new user, and the PyPI package finally runs the same engine as the plugin.

• Windows: commands no longer die on the python3 Store stub. Every command line runs through core/py.sh, which probes python3, python, then py -3 by actually running them (Python 3.10+) instead of trusting command -v, and prints one install message (exit 127) when none works. /eval-init's NEXT STEPS name the interpreter that really ran. *.sh is pinned to LF.
• /eval-seal with no registry exits 2 and writes nothing (it wrote an unsigned seal of registry_sha256: null and exited 0). It says to run /eval-init first.
• /eval-prove gitignores the signing key it creates. evals/.receipt-key, evals/.wedge-baseline.json and .eval-prove-junit.xml are added to .gitignore before the first write, so git add -A can't stage the key. /eval-init now ignores .eval-prove-junit.xml where the wedge actually writes it.
• No more dead-end errors. /eval-gate and /eval-explain with no candidate lead with that error and name /eval-prove and /eval-init (or the example and /eval-import once installed); the no-baseline warning only prints when a decision is made. /eval-verify with no receipts names /eval-prove. /eval-prove --run works in repos without a pytest suite (it was refused with "use --run"), and a missing pytest is named, with the pip command.
• GitHub token read in a URL-free module. core/gh_auth.py reads GITHUB_TOKEN; core/conductor.py builds the request and reads no environment, and sends the token only to api.github.com. A test fails if any shipped .py file both reads the environment and contains a URL.
• Removed the unused userConfig fields (receipt_key, github_token): Claude Code exports userConfig only to hooks and MCP servers, and this plugin has neither. EVAL_RECEIPT_KEY, the auto-generated evals/.receipt-key and GITHUB_TOKEN work as before.
• PyPI package synced with the plugin. src/eval_conductor/_core/ had drifted and missed the 1.4.1 fixes: eval-conductor prove on PyPI 1.4.1 printed a brightgreen "PROVEN 0/0" badge for a run with zero tests. A parity test now fails on any drift.
• Descriptions lead with the user's problem (all commands, both skills, the manifest and the marketplace listing).
• CI runs on windows-latest as well as ubuntu-latest. A live harness (tests/test_commands_live.py) runs every command line the way Claude does, in fresh git repos. Tests: 254 → 365.

1.4.1 — 2026-10-05

Wedge hardening + an honest-limits correction, from deep adversarial testing. The core gate was unaffected (NaN/Inf/string/bool scores, missing metrics, seal p-hacking and receipt tampering all already BLOCK/caught); every fix below is in the zero-config wedge path or its docs.

• Deleting evals/.wedge-baseline.json no longer resets the bar. The bar is reconstructed from the signed, hash-chained receipts (prior PROMOTE manifest.scores) when the local file is missing, so it can't be deleted to launder a regression into a "first run". Pin a lower floor deliberately with --min-pass-rate.
• Zero collected tests → CANNOT PROVE (exit 2), instead of a brightgreen "PROVEN 0/0" badge. The scorer's "0.0 not 1.0" contract is unchanged.
• Corrected a receipt overclaim. "Delete or reorder a receipt and the chain breaks" is true for tamper/reorder/middle-deletion; removing the latest receipt leaves a shorter valid chain, caught by distribution (push) — which the code already enforces via the ANCHOR report. Docs now say exactly that.
• Malformed candidate/baseline JSON → clean FAIL (exit 2), not a raw traceback.
• Tests: 254 passing (added baseline-deletion, zero-tests, and --min-pass-rate cases).

1.4.0 — 2026-10-04

Zero-config wedge: /eval-prove — one command, a signed proof of your tests.

• /eval-prove (core/wedge.py, vendored into your repo) turns an existing test suite into a tamper-evident, signed receipt plus a pasteable PR badge, with no registry and no config. Detects and runs pytest, or consumes a JUnit file (--junit) or any command that writes JUnit (--run).
• Proof + auto-gate (capture-then-gate). The first run has nothing to regress against, so it PROMOTEs and records the measured pass-rate and test-count as the bar (evals/.wedge-baseline.json). Later runs BLOCK if the pass-rate drops or tests disappear (the oldest way to turn a gate green). The bar ratchets up on a clean run and never down — a BLOCK never lowers it. A hardcoded "100% or BLOCK" was rejected: it would fail every flaky suite on run one.
• Same proof engine as the full gate. Receipts come from eval_receipt, so a wedge proof and a gate proof verify identically; a signing key is auto-generated (gitignored) on first run, so proofs are signed by default.
• Fail-closed. No detectable suite and no --junit/--run → exit 2 (no proof of an unmeasured run), with a one-line pointer, never a traceback. Zero executed tests → pass-rate 0.0.
• Honest badge. The badge always shows the test COUNT, and the output states that a green badge proves "not worse than before", not "good", and that one run doesn't calibrate flakiness — graduate to /eval-init for measured noise bands and real thresholds. The wedge is the on-ramp; /eval-init is the next room.
• 12 new tests (tests/test_wedge.py); vendored by /eval-init; wedge artifacts gitignored in the user repo. Dogfooded: proves eval-conductor's own 166 tests.

1.3.0 — 2026-10-04

Pre-registered evals: prove the bar was fixed before the run.

• Pre-registration seals (core/prereg.py, vendored into your repo; /eval-seal). prereg.py seal hashes evals/registry.yaml (and each metric's noise_band floor) into a signed, hash-chained seal under evals/seals/, containing no candidate data. At gate time the live registry is re-hashed against the seal; a bar that moved after sealing is flagged as drift.
• Closes the hole the diff-lint structurally cannot. registry_diff_lint.py proves a registry didn't get weaker than its git base — but a brand-new gate has no base ("nothing to weaken"), so a gate authored to exactly clear a score you already ran passes every check. That is p-hacking your own gate. Pre-registration is the deliberate, anchored base that makes "the bar was fixed before the result" a checkable fact.
• Composes with receipts. The gate records a prereg block on each receipt (sealed, seal_id, registry_match, bands_match, signature_ok), so a signed PROMOTE now proves not just "passed under this registry" but "passed under the registry that was pre-committed in seal X" — and a drifted decision is permanently stamped as such, inside the tamper-evident receipt itself.
• Opt-in enforcement, never breaks an un-sealed gate. Recording status on the receipt is a best-effort side effect that never changes the gate exit code. prereg.py check --require-seal is the fail-closed CI knob (exit 2 on drift or a missing seal); without the flag, drift is reported loudly and exits 0, so upgrading is non-breaking.
• CI: templates/eval-gate.yml gains a commented pre-registration check step. Native eval case prereg-reseal-refuses proves the assistant refuses to re-seal a weakened bar after seeing the scores (the exact laundering move).
• Honest limits carried in the README, /eval-seal and the seal output: a seal proves the bar is byte-identical to what was sealed and — via git anchoring — committed before the result; it does not prove the bar is strict enough (that's the diff-lint's and the reviewer's job), and a shared-secret key can't stop its own holder backdating the sealed_at string — so ordering rests on pushed git history, not the timestamp.

1.2.0 — 2026-10-04

Eval receipts: a provable verdict, not just an exit code.

• Tamper-evident eval receipts (core/eval_receipt.py, vendored into your repo; /eval-verify). Once a signing key exists (/eval-init now generates one, gitignored), every /eval-gate run appends an HMAC-signed, hash-chained receipt to evals/receipts/ binding the PROMOTE/BLOCK decision to the exact registry, candidate and baseline bytes it was computed from. A deleted or reordered receipt breaks the chain.
• Fused from titan-receipts. The canonicalization (core/canonical.py) and fork-detecting chain engine (core/chain_state.py) are ported from the titan-gate plugin — stdlib-only, byte-compatible — rather than reinvented or taken as a runtime dependency.
• Three verify-time tamper checks: chain integrity (one unbroken line from GENESIS, every receipt_hash recomputed from content), signature, and an eval-specific decision-vs-verdicts consistency check — a re-signed receipt whose recorded decision disagrees with its own verdict table is caught. --structure-only verifies without the key; the closing ANCHOR line reports how many receipts are committed and pushed.
• Fail-open emission. Receipting is a side effect that never changes the gate's exit code or crashes CI: a broken chain or unwritable path warns (RECEIPT:) and skips. Enforcement lives in /eval-verify, not the gate.
• CI: templates/eval-gate.yml gains a commented EVAL_RECEIPT_KEY secret and a release-blocking verify step. Native eval case receipt-tamper-blocks proves the assistant treats a verify failure as blocking, never "re-sign it".
• Honest limits carried in the README and verify output: HMAC is a shared secret — a receipt proves the decision is unaltered under a key you control, not who signed it, and not that the scores were honestly produced.

1.1.0 — 2026-10-03

Positioning: the gate for scores you already produce.

• Adapters (core/adapters.py, vendored into your repo; /eval-import): junit (pytest/jest/go JUnit XML), plugin-eval (claude plugin eval aggregate-result.json or results dir — latest run; partial: true refused), promptfoo, scores (flat JSON). All write a verified registry_hash.
• Zero-config first run. /eval-init auto-detects a profile: pytest (gates pass rate, a test-count floor measured at install, failures — CI producer pre-filled, green on first push), plugin-eval (gates overall score, with-vs-without delta, weakest case, case floor), or llm starter. --profile overrides.
• Native eval suite: evals/ now holds claude plugin eval cases for this plugin (threshold-edit refusal, /eval-init triggering, NaN doctrine). Measured mean Δ +0.50 vs no plugin; threshold-edit case Δ +1.00.
• Behaviour fix found by that suite: the eval-gates skill did not trigger on "gate is red, bump the threshold so CI passes" — Claude agreed to the edit with the plugin installed. Its description now names those phrasings; the skill fires and refuses 2/2.
• Template fix: the optional conductor audit step is no longer || true; it is a commented, release-blocking step you opt into.
• Review fixes: test_count floor uses pytest's selected count (deselected tests never reach JUnit) and returns unknown on collection errors; pytest profile requires a pytest config section or tests dir, and a 0-test repo falls back to llm; adapters refuse a missing registry instead of writing an unattested candidate; generated workflow installs requirements/-e .
• Self-gate thresholds raised to the new measured counts (112 tests, 23 fail-closed tests). run_self_eval.py uses the shipped JUnit parser.
• README rewritten around the problem, with a real BLOCK verdict; limits moved to a collapsed section (unchanged in substance).

1.0.1

• Diff-lint and baseline-warning fixes.

1.0.0

• Initial release: fail-closed gate, baseline runner, registry lints, 27-step conductor ledger, eval-gates and evidence-discipline skills.