round3ai/three-dev
Integrate three.dev into a codebase and work with recorded LLM traffic: route calls through the proxy, define use cases and quality metrics, investigate failure modes, and test prompt or model changes with offline experiments, using the three.dev MCP server.
Runs an offline experiment in three.dev to test a prompt, model, or reasoning change on recorded production traffic, and reads the results. Use when the user has a change in hand and wants to know whether it is better, cheaper, or faster before shipping, wants to compare two models or providers on their own traffic, asks whether a cheaper or smaller model holds the same quality, wants to A/B test a prompt, or asks what an existing offline experiment showed. Also routes live experiments, which run on real users and start in the three.dev app. Needs the three.dev MCP server. Not for finding out why a feature is failing in production; that is the three-dev-failure-modes skill. Not for wiring calls through the proxy; that is the three-dev-setup skill. Not for defining or reporting the quality metric a live experiment measures; that is the three-dev-quality-metrics-setup skill.
Investigates production quality problems in an LLM feature that three.dev already records. Triages the AI Judge's failure modes, reads the flagged conversations, finds the root cause, and proposes a prompt, model, or code fix. Use when the user asks why an LLM feature is failing or misbehaving in production, what its top failure modes, issues, or quality problems are, whether a problem is growing or started after a deploy, or wants to see bad conversations or count or segment requests. Also when the user arrives with a three.dev conversation id, to continue an investigation started in the three.dev chat. Needs the three.dev MCP server. Not for setting up three.dev or routing calls through the proxy; that is the three-dev-setup skill. Not for testing a change on recorded traffic or reading experiment results; that is the three-dev-experiments skill. Not for defining or reporting quality metrics; that is the three-dev-quality-metrics-setup skill.
Sets up quality metrics for an LLM feature that three.dev already records. Decides whether a business outcome is worth tracking and, if so, wires the code that reports it; three.dev creates the metric from the first report. Use when the user wants to start tracking conversions, bookings, resolutions, escalations, accepted suggestions, or another outcome of an LLM feature, or needs a metric wired for a live experiment. Not for reading which metrics exist or how they are performing, which is the list_quality_metrics tool and the three.dev app. Not for adding three.dev to a codebase or routing calls through the proxy; that is the three-dev-setup skill. Not for investigating failure modes or reading recorded conversations; that is the three-dev-failure-modes skill. Not for running or reading offline experiments; that is the three-dev-experiments skill.
Adds three.dev to a codebase so its LLM calls are recorded. Finds every LLM call site, groups the calls into use cases, and routes them through the three.dev proxy (gate.three.dev) with use case, session, and environment headers. Use when the user mentions three.dev, gate.three.dev, or X-Three headers, asks to record or monitor their LLM calls, or wants to set up or extend a three.dev integration. Covers OpenAI, Anthropic, Gemini, Azure OpenAI, OpenRouter, Bedrock, and LiteLLM clients. Read the docs before writing code; never guess header values or base URLs. Not for investigating failure modes or reading recorded conversations; that is the three-dev-failure-modes skill. Not for running or reading experiments; that is the three-dev-experiments skill. Not for defining or reporting quality metrics; that is the three-dev-quality-metrics-setup skill.