Skip to content

wtcooper/app-evals

v0.1.0MIT

Evaluate an LLM application's endpoint: a fixed adversarial benchmark and adaptive multi-turn red teaming.

app-evals

Evaluate an LLM application's endpoint.

SkillWhat it does
app-benchmarkSend a bundled labelled corpus of attack and benign prompts to your app and score recall, false-block rate and attack success by technique.
app-redteamRun multi-turn attacks that adapt to your app's replies, with repeats, to find weaknesses a fixed test set misses.

Install

The plugin follows Agent Plugins 1.0.0: a root plugin.json and one folder per skill under skills/. Any client that loads Agent Plugins can load this directory.

Claude Code:

/plugin marketplace add wtcooper/ai-security-evals
/plugin install app-evals@ai-security-evals

Each skill is also self-contained: copying its folder into any agent's skills directory works.