orq-compare-agents
Run cross-framework agent comparisons using evaluatorq from orqkit — compares any combination of agents (orq.ai, LangGraph, CrewAI, OpenAI Agents SDK, Vercel AI SDK) head-to-head on the same dataset with LLM-as-a-judge scoring. Use when comparing agents, benchmarking, or wanting side-by-side evaluation. Do NOT use when comparing only orq.ai configurations with no external agents (use orq-run-experiment instead).
Pinned to revision 9634e1d956e4, so it is the text this page describes rather than whatever the author pushed since.
Pre-approved tools experimental
Experimental field. Support varies between clients, so this list is what the author declared, not what your client will enforce.
- Bash(curl:*)
- Read
- Write
- Edit
- Grep
- Glob
- WebFetch
- Task
- AskUserQuestion
- mcp__orq-workspace__search_entities
- mcp__orq-workspace__create_dataset
- mcp__orq-workspace__create_datapoints
- mcp__orq-workspace__create_llm_eval
Files
- skills/orq-compare-agents/SKILL.md
- skills/orq-compare-agents/resources/evaluatorq-api.md
- skills/orq-compare-agents/resources/gotchas.md
- skills/orq-compare-agents/resources/job-patterns.md
Every link opens the file at its source, pinned to the revision this page describes.