agent-feature-review
Guided security review of a new agent feature being switched on for a model or product the organisation already uses, when someone asks "can we enable this?" and no single benchmark answers it. One threat-and-test template per feature type: computer use and browser control, persistent memory, connectors and MCP servers, code execution sandboxes, file search and RAG, multi-agent and subagents, scheduled and background agents. Each template says what the feature changes about the trust boundary and, per threat, the review questions, the vendor evidence to look for, which existing benchmark could test it and through which skill, and what has no standardized test. Threats carry OWASP LLM 2025, OWASP Agentic and MITRE ATLAS ids where verified. Writes a JSON and markdown record, evidence.jsonl rows, and a list of every threat left unassessed. It prescribes no control and makes no recommendation. For an image-generation model use image-model-review.
- License
- MIT
- Compatibility
- Python 3.10+ standard library only. No API key and no network access are needed by the scripts.
Pinned to revision 3fb15e11d58f, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/agent-feature-review/SKILL.md
- skills/agent-feature-review/examples/computer-use.answers.example.json
- skills/agent-feature-review/lib/evalkit.py
- skills/agent-feature-review/references/background-agents.md
- skills/agent-feature-review/references/benchmarks.md
- skills/agent-feature-review/references/code-execution.md
- skills/agent-feature-review/references/computer-use.md
- skills/agent-feature-review/references/connectors-mcp.md
- skills/agent-feature-review/references/deployment-context.md
- skills/agent-feature-review/references/file-search-rag.md
- skills/agent-feature-review/references/memory.md
- skills/agent-feature-review/references/multi-agent.md
- skills/agent-feature-review/scripts/_evidence.py
- skills/agent-feature-review/scripts/templates.py
- skills/agent-feature-review/scripts/write_review.py
- skills/agent-feature-review/templates/_benchmarks.json
- skills/agent-feature-review/templates/_frameworks.json
- skills/agent-feature-review/templates/background-agents.json
- skills/agent-feature-review/templates/code-execution.json
- skills/agent-feature-review/templates/computer-use.json
- skills/agent-feature-review/templates/connectors-mcp.json
- skills/agent-feature-review/templates/file-search-rag.json
- skills/agent-feature-review/templates/memory.json
- skills/agent-feature-review/templates/multi-agent.json
- skills/agent-feature-review/tests/test_agent_feature_review.py
Every link opens the file at its source, pinned to the revision this page describes.