ghchinoy/mizan-mcp

v0.1.0Apache-2.0

Expose the Mizan eval engine to agents and MCP clients as callable tools via the mizan mcp server. Registers a local stdio MCP server (mizan mcp stdio) advertising four tools — mizan_list_metrics, mizan_get_metric (read-only), mizan_eval_run, and mizan_eval_pairwise — plus an agent-in-the-loop eval skill. Use when an agent should discover a judge and run an evaluation over MCP rather than shelling out to the CLI.

GitHub Open Website

evaluate-with-mcp

Run an agent-in-the-loop Mizan evaluation over the Model Context Protocol (MCP) — discover a judge (mizan_list_metrics -> mizan_get_metric), run it (mizan_eval_run / mizan_eval_pairwise), and read the persisted runId returned INLINE in the tool result. Use when an agent has the mizan MCP server connected (via the mizan-mcp plugin's stdio registration) and should score or A/B-compare a response/asset against a metric using MCP tools rather than shelling out to the mizan CLI.