Skip to content

danielsimonjr/verify-mcp

v0.16.12Apache-2.0

MCP server wrapping danielsimonjr/verify (veriharness).

Changelog

All notable changes to this project are recorded here. The format follows Keep a Changelog, and the project uses Semantic Versioning.

[Unreleased]

[0.16.12] - 2026-10-12

Changed

  • The pin moves to verify 0.12.15. The warning for a window at or below 16384 tokens says that every tool result is cut to 1000 characters, which is what a session measured at that size does; it no longer says that every rollout compacts. The floor for context_size is the same: above 4096.

[0.16.11] - 2026-10-12

Changed

  • The pin moves to verify 0.12.14. A pi worker whose final message is a tool call written as text (prose before the call, or a value that is not JSON) is sent back once, on a turn that offers the tools, with the reason that the call did not run. The guard cuts a message that writes a second such call. The nudge kinds of nudge_timeout are now time, cut, stuck, format, json, schema, ids and call.

[0.16.10] - 2026-10-12

Changed

  • The pin moves to verify 0.12.13. A pi worker is stopped when a tool call holds the answer records (the error stuck, then the nudge with no tools), a call with an argument that the tool does not list is blocked with the arguments it takes, a message that the guard cut is sent back with the reason, and a final message with broken JSON is named at the place where it breaks in every case.

[0.16.9] - 2026-10-12

Changed

  • The pin moves to verify 0.12.12. A pi worker is stopped after three turns in which every tool call failed (the error stuck), each tool result is cut to the room that is left before compaction, a final message with broken JSON is sent back with the place where it breaks, and with ids the output limit follows the size of the batch. retries also runs a stuck or a compacted rollout again.

[0.16.8] - 2026-10-12

Added

  • verify_workers takes frequency_penalty (passed as --frequency-penalty, verify 0.12.11 or later). It is not set by default.

Changed

  • The pin moves to verify 0.12.11. A pi worker loads a repeat guard that blocks a repeated read, grep, find or ls call and cuts a final message that has more records than the batch has items. A pi worker is sampled with temperature 0.6 when temperature is not set. The temperature description says so.

[0.16.7] - 2026-10-11

Changed

  • The pin moves to verify 0.12.10. A pi worker is sent back for its answer once for each kind of failure (time, cut, format, schema, ids) instead of once in all, and with ids its charter lists the ids of the batch. nudge_timeout is the budget of each nudge.

[0.16.6] - 2026-10-11

Added

  • verify_workers takes ids (ARRAY.FIELD, passed as --ids). A report that fits the schema and covers other rows than its batch ends as error ids, is sent back once, and is retried by retries.

Changed

  • The pin moves to verify 0.12.9. A pi worker that made no tool call ends as the new error no-tool-calls instead of complete, and the schema is in the worker charter.

[0.16.5] - 2026-10-11

Changed

  • The pin moves to verify 0.12.5. driver.log lines carry the local time and its UTC offset, and the manifest, each worker.json and result.json name the provider and model of every role, also a Claude Code model.

[0.16.4] - 2026-10-10

Changed

  • The pinned verify is 0.12.4. verify_model_check, verify_batch, verify_workers and verify_driver now report the digest and quantization of a local model, so a re-created tag shows in the records.

[0.16.3] - 2026-10-10

Changed

  • The verify pin is 0.12.3: verify_workers names an --item-tokens value only for a peak that is past the budget of the manifest, since a rebatch with a smaller value gave the same split.

[0.16.2] - 2026-10-10

Changed

  • The verify pin is 0.12.2: the driver line for an investigation that left no record names the compactions of its session.

[0.16.1] - 2026-10-10

Changed

  • The verify pin is 0.12.1: the workers warning about failed tool calls names the class of error (toolErrorKinds) and no longer says the worker reads missing paths when it sent no arguments.

Fixed

  • Two tests that start a process cold (captures stdout and stderr, and the fresh-build check of the committed bundle) now carry an explicit 30 s test timeout. Bun's default of 5 s cut them short on a loaded or freshly scanned disk, so the suite failed once right after bun install and then passed.

[0.16.0] - 2026-10-10

Changed

  • The verify pin is 0.12.0.
  • verify_batch text names each workspace/<file> that the spec or the worker prompt names and no batch holds (manifest.json missing), with items_name as the usual fix.

[0.15.0] - 2026-10-10

Changed

  • The verify pin is 0.11.0.
  • verify_driver text lists the items the reviewer left open (result.json openItems): the delivered file holds a verdict for each of them.
  • verify_workers text shows N s in all for a retried rollout (worker.json totalSeconds), beside the seconds of the last attempt.

[0.14.1] - 2026-10-10

Changed

  • The verify pin is 0.10.1: a record that an investigation wrote in its own folder is no longer judged missing, and the records of an earlier run are set aside to previous/ (verify CHANGELOG 0.10.1).

[0.14.0] - 2026-10-10

Changed

  • The verify pin is 0.10.0.
  • verify_workers names the outcome of each rollout in its result text (b01/r01: max-turns (60 s, 41 turns, peak 28881)) and repeats the harness warnings. A host that shows only the text of an error result no longer loses which rollout failed and why.
  • verify_driver returns the driver's result.json (verify 0.10.0 or later) as structured.result and states in its text which investigations left no record, the base (none means the deliverable was built from the inputs), an invalid delivery, and any write an investigation made outside its own files. Exit 0 says only that the run ended.

[0.13.0] - 2026-10-10

Added

  • verify_workers takes schema, retries, nudge_timeout and allow_compaction. They pass --schema, --retries, --nudge-timeout and --allow-compaction to verify 0.9.0. A deliverable that does not fit the schema ends as schema; a stopped pi worker gets one nudge for its answer; a rollout with a compaction ends as compacted unless allow_compaction is true.

Changed

  • The verify pin is 0.9.0.

[0.12.0] - 2026-10-10

Added

  • VERIFY_MCP_ITEM_TOKENS and VERIFY_MCP_OVERHEAD_TOKENS set the default item_tokens and overhead_tokens of verify_batch, in the server env of settings.json or .mcp.json. What one item costs a worker depends on the task, the tools and the model, so a deployment sets its own measured value and no default is built in. A call that sets its own value keeps it. The variables apply whether or not the call names a model.

Changed

  • The verify pin is 0.8.0. A verify_workers rollout record now also has estTokens, the batch estimate from manifest.json, and verify writes a progress line when the measured peakContext is above it. The line names the --item-tokens value that would have covered the batch. verify-mcp passes the record through unchanged.

[0.11.0] - 2026-10-09

Added

  • verify_batch takes reference, passed as --reference: files every batch gets in workspace/ that the worker only searches, so the budget does not count them.

Changed

  • The verify pin is 0.7.0, the first version with veriharness batch --reference.

Fixed

  • The README's verify_batch example passed the searched CHANGELOG as shared, which puts every batch over the budget. It now uses reference.

[0.10.0] - 2026-10-09

Added

  • verify_workers takes max_turns, passed as --max-turns: a worker that passes that number of assistant turns stops with the error max-turns.

Changed

  • The verify pin is 0.6.0, the first version with veriharness workers --max-turns.

[0.9.0] - 2026-10-09

Added

  • verify_batch: runs veriharness batch. It splits an items file (jsonl, blank-line or heading:REGEX) into task folders that each fit a token budget, batch_tokens or half the worker model's window. The result holds the parsed manifest.json. The default model profile applies when the call sets no batch_tokens.
  • verify_workers: runs veriharness workers, N worker rollouts on each batch. Each finished rollout is one progress notice, sent at once. The result holds rollouts and summary; exit 1 and exit 75 (a usage limit) are errors that still carry the records. Default timeout 12 hours.
  • context_size: "auto" on the main model and on each role. A role, verify_batch and verify_workers take "auto" or a whole number above 4096, as verify does. verify_model_check returns window and windowSource from verify.

Changed

  • Pin verify 0.5.0, which adds --context-size auto, batch and workers.

[0.8.0] - 2026-10-09

Added

  • roles on verify_driver and verify_runner: a strict object keyed by checker, challenger, reviewer and fixer. Each entry is {provider, model, base_url?, context_size?} and becomes --role ROLE=PROVIDER:MODEL, plus --role-base-url and --role-context-size for a local role. README section "A model for each role"; the skill names the field.

Changed

  • Pin verify 0.4.0, which adds the role options. tests/roles.test.ts keeps ROLES equal to the pinned harness/roles.ts.

[0.7.0] - 2026-10-09

Changed

  • The pin moves to verify 0.3.0, which renames the lane flash to fable. LANES and the lane_max keys follow, and max_flash becomes max_fable. A cell on an archived flash pool still runs: verify gives it the fable lane.
  • README: the four phases carry the verify role names Checker, Challenger, Reviewer and Fixer. The Worker makes each rollout. Tool names, fields and files are unchanged.

[0.6.0] - 2026-10-09

Changed

  • The pin moves to verify 0.2.0. All four lanes (flash, opus, haiku, sonnet) run Claude Code, so every lane needs env: "none". The text of the tool schemas, the README and the verify skill says so.
  • verify sets the default lane caps: flash 2, opus 2, haiku 4, sonnet 4. The wrapper default of 4 for haiku and sonnet from 0.5.1 is removed, because the pinned verify now holds the same value.

[0.5.1] - 2026-10-09

Changed

  • verify_runner raises the in-flight cap of the haiku and sonnet lanes from verify's default of 2 to 4. The server adds --lane-max haiku=4 or --lane-max sonnet=4 only when the cells use that lane and lane_max does not name it. A cap that the call sets still wins. A run that uses no Claude Code lane passes no new flag.

[0.5.0] - 2026-10-07

Added

  • Tuning in the default profile: VERIFY_MCP_THINKING and VERIFY_MCP_NUDGE_TIMEOUT (verify_driver), and VERIFY_MCP_MAX_TOKENS and VERIFY_MCP_REQUEST_TIMEOUT (verify_driver and verify_model_check). They follow the same rule as the other profile fields: used only when the call names no provider and no model, and only for a field the call omits. plugin/.mcp.json passes them through empty and example.mcp.json lists them.
  • example.mcp.json: a server entry with every VERIFY_MCP_* variable and placeholder values, for a user to copy and set after installing. The README describes it. Git ignores a root .mcp.json, so a local copy stays local.

[0.4.2] - 2026-10-07

Changed

  • The plugin ships no default model. 0.4.0 and 0.4.1 set one in plugin/.mcp.json, which tied a local setup to every install. plugin/.mcp.json now passes the five VERIFY_MCP_* variables through, empty when unset, and a test pins this. A machine sets its own default in the env block of its Claude Code settings.json.

[0.4.1] - 2026-10-07

Changed

  • The plugin default model changed to a variant that carries a num_ctx, because verify refuses a model that reports none. Removed in 0.4.2.

[0.4.0] - 2026-10-07

Added

  • A default model profile from the environment: VERIFY_MCP_PROVIDER, VERIFY_MCP_MODEL, VERIFY_MCP_BASE_URL, VERIFY_MCP_CONTEXT_SIZE and VERIFY_MCP_ENV. verify_driver and verify_model_check use it when a call names no provider and no model.

Changed

  • verify_model_check no longer requires provider and model in its schema. It returns an error that names the variables when neither the call nor the environment gives them.

[0.3.0] - 2026-10-06

Changed

  • verify-mcp installs verify from npm: the dependency is @danielsimonjr/verify at exactly 0.1.0, and not a GitHub commit. src/pin.ts holds the package name and the version, and a test keeps package.json equal to them. verify_status reports the pin as @danielsimonjr/verify@0.1.0.
  • The plugin still starts the verify checkout through VERIHARNESS_BIN: an installed plugin has no node_modules.

[0.2.2] - 2026-10-06

Changed

  • verify-mcp pins verify 74a39c5. That verify commit installs its pi agent runtime from @danielsimonjr/pi (0.84.4) and not from @earendil-works/pi-coding-agent. The plugin still launches the verify checkout through VERIHARNESS_BIN, so a checkout needs setup_pi.sh run again after it is updated to that commit. No npm publish is involved: verify is a git dependency.
  • The README names the real pinned commit. It still named 0a7bbf3.

[0.2.1] - 2026-10-05

Changed

  • verify-mcp pins verify c10100b (verify PR #19). The dependency is named verify, the package's new name, and the server looks for node_modules/verify. The command and the VERIHARNESS_ variables keep their names.
  • The new pin also includes verify PR #17 and PR #18. In a grader, the stdout cap now kills the whole process tree and runs the cleanup hook, as the timeout does. The live Claude Code test now checks that the verifier picks the right answer.

[0.2.0] - 2026-10-05

Added

  • The Claude Code verifier. verify-mcp pins verify 0a7bbf3 (verify PR #16). Since verify PR #11 (102894a), verify runs the verifier on Claude Haiku or Claude Sonnet through claude -p with the login that Claude Code holds. The provider fields name claude-code. verify_model_check probes it.
  • verify_runner accepts the lanes haiku and sonnet, lane_max (one --lane-max LANE=N per lane; a strict object, so an unknown lane or __proto__ is an input error) and env, which verify passes to every driver. The Claude Code lanes need env: "none".
  • A verify_model_check error for a verify build without the Claude Code provider says so and names the commit that added it.
  • The README, the verify skill and the /verify command describe the Claude Code verifier, its lanes, its Windows setup (VERIHARNESS_CLAUDE_BIN) and the fact that it always runs without the jail.

Fixed

  • The text for a verify build without model-check named the pinned commit as the commit that added it. It names 756bc2b (verify PR #2) now; the pin moves and that commit does not.
  • The detached-grandchild test stops the run when the grandchild has started, not after a fixed 1.5 s timeout. Two Bun cold starts can take longer than 1.5 s on a loaded host; the stop then came before the grandchild existed, and the test failed in a full run but passed alone. A negative control (taskkill without /T) still fails it.

Changed

  • verify_runner refuses a cap set twice for one lane (max_flash with lane_max.flash, or max_opus with lane_max.opus). verify applies lane_max over the other two, so one value was dropped.
  • The README is written for a user of the server. It explains what verify does, gives a quick start in Claude Code (a verify checkout, its pi runtime, the plugin, a first task), describes the task workspace and each output file, and adds workflows, a defaults table, a Windows section, a security section and a source map. It now states that the driver needs pi (harness/scripts/setup_pi.sh), that env: "none" is required off Linux, and that an MCP host can start the server with a reduced environment.
  • Every exported symbol in src/ has a doc comment: repo-tools docs check src passes with no MUST or SHOULD issue. It reported 63 symbols with no comment.
  • The README gives the output tail as 65536 characters. It said 64 KiB, but the tail is cut from the decoded text, not from the bytes.
  • CI runs the tests on windows-latest as well as ubuntu-latest. The Windows process-tree kill and the Windows path handling had no CI coverage.
  • Dependabot proposes weekly bumps for the SHA-pinned GitHub Actions. Dependabot alerts and CodeQL default setup are on for the repository.

Security

  • The verify pin takes verify's symlink hardening. The verifier can write out/ in its task workspace. verify now refuses a symlink there: a grader does not read through one (verify #15), and the driver does not delete, write, create or list through one on the host (verify #16). What the verifier leaves under out/ no longer stops a task before its result is recorded (verify #16). The pin also takes the review fixes of verify #12 and #13, among them an SB2 deliverable that is a symlink is refused, and the jail stops when a read-only remount fails.

[0.1.1] - 2026-10-04

Security

  • verify_runner rejects a run_name or a pool in cells that is not one path segment. verify joins both under VERIHARNESS_RUNS and deletes an existing task workspace there, so a value such as ../x or wb:.. made the runner delete directories outside the runs directory. A segment is letters, digits, ., _ and -, not starting with a dot.
  • A timeout or a client cancel kills the whole process tree. verify's driver starts each agent turn as a detached child, which leads its own process group on POSIX and leaves the parent's job object on Windows, so the old group kill left the turn running. POSIX now takes a ps snapshot of the descendants before it signals them; Windows uses taskkill /T /F.

Fixed

  • verify_runner rejects a lane other than flash or opus and a cell_cap entry whose cap is below 1 or whose key is not a bench or default. verify accepted both, and its scheduler then never started the cell's tasks and waited forever.

  • The skip_inflight description gives the unit as minutes. It said seconds.

  • verify_read_result reads at most max_bytes plus one byte of a text artifact. It read the whole file to return the prefix, so a large driver.log blocked the server and could exhaust its memory.

  • verify_driver, verify_runner, verify_score and verify_materialize carry destructiveHint: true. They overwrite or delete files, and false declares additive changes only.

  • A call that times out and is then cancelled is stopped once. The second stop armed a SIGKILL timer that was never cleared and fired at a stale list of process IDs.

  • The Claude Code plugin installs and starts from a marketplace. It was the repository root, and .mcp.json launched bun on src/index.ts. A plugin cache clone has no node_modules, so the server could not resolve @modelcontextprotocol/server or zod, and the CLI lookup in node_modules/veriharness failed too. The plugin is now plugin/: its .mcp.json launches a committed bundle, plugin/bundle/index.mjs, and sets VERIHARNESS_BIN to a verify checkout at ~/Github/verify. The marketplace entry's source is "./plugin".

  • tests/resolve.test.ts passes on Windows. Two assertions matched / path separators.

Added

  • bun run bundle (scripts/bundle.ts) builds plugin/: the Bun-target bundle plus copies of the skill and the command. It builds from the repository root, so the output is byte-identical wherever it runs.
  • tests/bundle.test.ts: plugin/ must match a fresh build, and the bundle must start and list every tool from a directory with no node_modules in reach.
  • tests/manifests.test.ts: every manifest must carry the package.json version.

Removed

  • The root .mcp.json. Its only role was the Claude Code plugin's MCP registration, which plugin/.mcp.json now holds.

[0.1.0] - 2026-10-04

Added

  • MCP server for the veriharness CLI on Bun, protocol revision 2026-07-28, with plugin manifests for Claude Code, Codex and Cursor.