relai-ai/relai
Continuously evaluate and optimize your AI agent: build Harbor-format evals from your code, requirements, and traces, then get harness improvements as pull requests.
Use when desired agent behavior should be captured as a repeatable RELAI agent test from a prompt, a run log plus feedback, or hard-scenario suggestions from existing RELAI context.
Use when the user has a CSV benchmark, with optional evaluator references or prompt guidance, that should become a RELAI benchmark.
Use when the user asks to operate any public RELAI CLI workflow, including setup, init, ingest, agent tests, benchmarks, evaluators, simulation, regression selection, optimization, memory, tags, plugin marketplaces, shared proactive policy, updates, or shell completions.
Use when the user wants RELAI to ingest a folder of agent traces, transcripts, logs, or other context into EvalsWiki.
Use when RELAI should make or validate behavior-focused changes against existing agent tests or benchmarks.
Use when deciding whether a request should be handled by direct implementation or by a RELAI workflow, when a user's goal needs several RELAI commands in sequence, or when the user asks what RELAI is for.
Use when a planned or completed direct code change needs focused regression checks from existing agent tests, custom-adapter tasks, or benchmark samples without an optimizer run.
Use when measuring current agent behavior against RELAI agent tests, tags, or benchmarks.
Capture durable project decisions, corrections, constraints, preferences, and useful hypotheses from coding-agent conversations into the RELAI EvalsWiki.