Skip to content

wtcooper/app-evals

v0.1.0MIT

Evaluate an LLM application's endpoint: a fixed adversarial benchmark and adaptive multi-turn red teaming.

app-redteam

Adaptive multi-turn red team of an LLM app via promptfoo's crescendo attack engine: genuine multi-turn attacks that read the target's real responses and escalate on refusals, seeded by customized application objectives (with an optional cross-domain cyber pack). Runs fully locally via generate-once → redteam eval. Use to actively find vulnerabilities or stress-test multi-turn robustness. For a fast reproducible benchmark use app-benchmark; for a guardrail in isolation use control-isolate.

License
MIT
Compatibility
Requires node>=18 and python>=3.11 (for the summary).
Read SKILL.md at the source

Pinned to revision 3fb15e11d58f, so it is the text this page describes rather than whatever the author pushed since.

Files

Every link opens the file at its source, pinned to the revision this page describes.