image-model-review
Guided security review of an image-generation model (and, briefly, video) when someone asks "can we enable this?" and no standardized benchmark answers it. Covers the security concerns rather than content safety: phishing and brand-impersonation assets, identity documents, QR and hidden payloads, likenesses of real people, editing of uploaded documents, and whether provenance marks (C2PA metadata, invisible watermarks) are present and survive ordinary handling. Produces a structured record: what the vendor publishes, what a hand-built probe set showed (counts with 95% intervals), what a C2PA marker check found, and every threat that was not assessed. Writes evidence.jsonl rows other review skills consume. It sets no bar and makes no recommendation; the decision is left to people. For a model that reads images use model-benchmark; for agent features use agent-feature-review.
- License
- MIT
- Compatibility
- Python 3.10+ standard library. Pillow is optional: without it the re-encode, resize, crop and screenshot transformations of the provenance check are skipped and recorded as not assessed. No API key and no network access are needed by the scripts.
Pinned to revision 3fb15e11d58f, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/image-model-review/SKILL.md
- skills/image-model-review/examples/probe_outcomes.example.csv
- skills/image-model-review/examples/review_answers.example.json
- skills/image-model-review/lib/evalkit.py
- skills/image-model-review/references/evidence-harvest.md
- skills/image-model-review/references/probe-set-template.md
- skills/image-model-review/references/provenance.md
- skills/image-model-review/references/threats.md
- skills/image-model-review/scripts/_evidence.py
- skills/image-model-review/scripts/make_fixtures.py
- skills/image-model-review/scripts/provenance_check.py
- skills/image-model-review/scripts/record_probes.py
- skills/image-model-review/scripts/record_review.py
- skills/image-model-review/templates/threats.json
- skills/image-model-review/tests/test_image_model_review.py
Every link opens the file at its source, pinned to the revision this page describes.