pil-agent-plugin

An agent plugin that gives coding agents quantitative image measurement — exact colour palettes, per-hue census, layout statistics, perceptual hashes, and changed-region localisation — plus an optional constrained multi-view layer for template-mesh fitting, Blender BVH clearance, and arbitrary matched renders.

It is designed to complement an agent's native multimodal vision, not replace it.

New in 0.10.0: a second plugin, blender-inspect, now owns the Blender tools (depth, mesh-defect audit, inspection renders), and this plugin gains pil_depth for model-inferred depth of concept images. The pil_blender_* and pil_multiview_render copies here are deprecated. 0.9.6: every environment setting is read by its literal name. 0.9.5: clearer embedding-model settings and skill discovery text. 0.9.4: the plugin tree no longer bundles working evidence under runs/, so it is smaller and fully readable. 0.9.3 added credential isolation for MCP tools and public dependency installs. 0.9.2 added submission privacy disclosures, a bundled Codex icon, and explicit data-handling documentation. 0.9.1 added shorter, more discriminating skill descriptions and progressive-disclosure entrypoints tuned for GPT-6 Astra. The 0.9.0 release added explicit EXIF/ICC normalization, image-bound masks, bounded registration, source-disjoint domain calibration, native local changes, and discoverable CLI/MCP tools. See the complete measurement workflows and usage guide.

Two plugins

This repository is a marketplace with two plugins:

Blender work belongs to blender-inspect. The Blender tools that started here (pil_blender_mesh, pil_blender_fit, pil_blender_render, pil_multiview_render) are deprecated in 0.10.0: flagged in docs and in the capability catalog, and removed in the next minor release. They behave as before, except that pil_multiview_render now places its cameras correctly (the same fix as blender_multiview_render), so its renders change. pil_contract_verdict --scene-stats accepts blender_mesh output as well as pil_blender_mesh output. Evals for blender-inspect are not shipped.

Privacy and data sent

Image and mesh processing runs locally. The plugin has no telemetry, hosted processing service, or account/API-key requirement. Reports may contain local paths, image metadata, OCR text, and measurements. The optional MCP adapter sends tool results over local stdio to your agent; that agent may forward results or images to its model provider under its own privacy settings.

The adapter passes only an allowlisted environment to local tool processes, so credentials from the host are not forwarded. The plugin does not use npm.

Installation and bootstrap can contact package registries and download hosts through uv/pip and system package managers. These services receive normal download requests (including your IP address and requested packages), not input images. The optional schema-validation example downloads a public schema. Model weights must be supplied separately. See Privacy policy.

The Codex manifest includes a bundled icon and a privacy-policy URL. This package uses no npx or uvx launchers, so launcher version pinning is not applicable. Python dependency resolutions are recorded in uv.lock.

Why this exists

An agent with multimodal vision already reads images well: it transcribes text, understands layout semantically, identifies objects, and recognises visual style. What it cannot do is produce a number — an exact hex value, a reproducible similarity score, or coordinates of what changed between two renders.

This was validated rather than assumed. Two subagents analysed the same complex image; one was restricted to native vision, the other was permitted Pillow scripting. The result was that neither approach is a superset of the other:

• Native vision, unassisted, produced a full text transcription, semantic layout, object inventory, correct style identification, and a correct reading of the image's colour-coded semantics.
• Naive measurement got one of those things wrong: a global 8-colour quantisation of the image returned zero vivid colours, because ~75% of its pixels were near-black and area-weighted extraction spent its whole budget on dark tones. Vision's qualitative read was right where the measurement was wrong.

So the tools here are shaped around that failure. Full write-up: docs/design-rationale.md.

Standards conformance

This package conforms to Agent Plugins 1.0.0, the vendor-neutral standard for packaging reusable agent components, and ships the portable and Claude Code-native layouts side by side:

All four manifests describe the same package. agents/, .codex-plugin/, and .claude-plugin/ are undefined top-level directories under Agent Plugins, which the specification requires clients to ignore rather than reject, so their presence does not affect portability. The optional scripts/pil_mcp.py stdio adapter exposes the CLI tools through MCP. It requires the mcp extra and explicit host configuration; no root MCP configuration auto-enables it for core-only installs. See MCP setup.

Two honest limits on that claim:

• Claude Code does not itself parse the portable manifest. It installs from .claude-plugin/plugin.json. The root plugin.json exists so that Agent Plugins clients and tooling can consume the same checkout; it is additive, and both claude plugin validate --strict and schema validation of plugin.json pass.
• ${CLAUDE_PLUGIN_ROOT} in the skill is client-specific. Agent Plugins names the equivalent variable ${PLUGIN_ROOT}; SKILL.md states both.

Conformance is enforced by tests/test_packaging_conformance.py, which checks the closed manifest schema, the name patterns, skill discovery at the fixed location, the Agent Skills frontmatter constraints, and that the three manifests have not drifted apart — version now lives in all of them, and claude plugin tag refuses a release when they disagree. To re-check the manifest against the published schema directly:

uv run --with jsonschema python -c "
import json, urllib.request
from jsonschema import Draft202012Validator
schema = json.load(urllib.request.urlopen('https://agent-plugins.org/schemas/1.0.0/plugin.schema.json'))
Draft202012Validator(schema).validate(json.load(open('plugin.json')))
print('valid')"

Requirements

• Claude Code (for plugin installation; the scripts also run standalone)

• Python 3.11+

• Pillow and numpy for the core image tools

• Optional reconstruction extra: OpenCV and SciPy

• Optional embedding extra: onnxruntime plus an ONNX vision model file — uv sync --extra embedding, then point PIL_AGENT_EMBED_MODEL (or pil_embed --model) at one of the two gate-tested models, pairing each with its preprocessing profile (--preprocessing / $PIL_AGENT_EMBED_PREPROCESSING):

  • mobilenetv2-12.onnx (ONNX Model Zoo) with imagenet — sha256 c0c3f76d93fa3fd6580652a45618618a220fced18babf65774ed169de0432ad5
  • clip-vit-b32-visual.onnx (CLIP ViT-B/32 visual tower, ONNX export from jina-ai/clip-as-service) with clip — sha256 06395063c0a5c28b1a8d4bd585261501a878c8f52d1216db6c4cbb651f7c13f1

  Model weights are pinned by hash in every payload, never bundled in this repo.

• Optional depth model for pil_depth (same embedding extra, so onnxruntime): an ONNX export of Depth Anything V2 Small, supplied by you with PIL_AGENT_DEPTH_MODEL (or pil_depth --model). See Concept-image depth for how to obtain it and its licence. Any other model runs, but is flagged model_not_gated and advertises nothing.

• Optional OCR tool: a system tesseract binary (e.g. apt-get install tesseract-ocr)

• Optional Blender tools: a local Blender executable (tested with Blender 5.2)

• uv recommended, for a pinned environment

Installation

Bootstrap

With Python 3.11+ installed, the standalone bootstrap creates or repairs the plugin's .venv and records successful setup. It runs from any working directory; paths below assume you are at the plugin root:

# Windows PowerShell (py -3 must select Python 3.11 or newer)
py -3 .\scripts\pil_bootstrap.py check
py -3 .\scripts\pil_bootstrap.py install
py -3 .\scripts\pil_bootstrap.py check
# Optional capabilities, selected explicitly:
py -3 .\scripts\pil_bootstrap.py install --ocr --reconstruction
py -3 .\scripts\pil_bootstrap.py install --embedding --model 'C:\Models\mobilenetv2-12.onnx' --preprocessing imagenet
py -3 .\scripts\pil_bootstrap.py install --depth --depth-model 'C:\Models\depth_anything_v2_vits.onnx'

# macOS / Linux
python3 ./scripts/pil_bootstrap.py check
python3 ./scripts/pil_bootstrap.py install
python3 ./scripts/pil_bootstrap.py install --ocr

Core installs Pillow and NumPy. --reconstruction adds OpenCV and SciPy; --embedding adds ONNX Runtime and checks a caller-supplied model; --depth adds ONNX Runtime and checks the caller-supplied depth model (--depth-model or $PIL_AGENT_DEPTH_MODEL) by running pil_depth diagnose; --ocr checks Tesseract and installs it if needed using winget (Windows), Homebrew (macOS), or apt-get/dnf (Linux). Install the platform package manager separately if absent. Linux system installation may prompt through sudo, and Windows may request elevation. Blender and model weights are not installed by this script.

The bootstrap uses uv if available, otherwise Python's venv/pip. It installs the dependency ranges declared in pyproject.toml without updating the lockfile or removing unselected extras. For the repository's lockfile-pinned development environment, use the uv sync commands below instead. On Linux, the fallback may require your distribution's python3-venv package.

Use the same capability flags when checking a prior installation. check emits JSON without installing packages or writing state. Exit 0 and already_bootstrapped: true mean both a matching successful receipt and passing live probes. Exit 2 means setup is absent, stale, or incomplete; inspect ready, previously_run, and checks to distinguish those cases. An existing working venv can report ready: true before bootstrap has ever run.

Successful install writes .venv/pil-agent-bootstrap.json; repeated installs with a matching receipt and passing probes skip installers. A receipt is not trusted without live checks. Model diagnostics run each time, so moving or breaking a configured model is detected. --embedding without a valid model can install its Python packages but will refuse completion and write no new success receipt. Configure models as described below; no models are guessed or downloaded. Failed installs return 2 with a reason on stderr and empty stdout.

The bundled bootstrap skill handles this check-first workflow for agents. It operates on the active plugin copy; bootstrapping a source clone does not configure a separate installed cache. The receipt and venv are local state, not portable installation evidence.

Manual Python environment

git clone https://github.com/bsmi021/pil-agent-plugin.git
cd pil-agent-plugin
uv sync                        # installs Pillow + numpy into a local venv
uv sync --extra reconstruction # also installs OpenCV + SciPy

Windows / PowerShell: OCR and embedding setup

From the repository root, install the Python dependencies and run the venv's interpreter directly; activation is unnecessary. The embedding extra installs the CPU runtime, but does not download model weights:

uv sync --extra embedding
& .\.venv\Scripts\python.exe --version
winget install --id UB-Mannheim.TesseractOCR --exact
& .\.venv\Scripts\python.exe .\scripts\pil_ocr.py --diagnose

Tesseract's Windows installation documentation links to the UB Mannheim installer. Discovery checks --tesseract-executable, then PIL_AGENT_TESSERACT, then PATH, then Windows installations under ProgramFiles\Tesseract-OCR, ProgramFiles(x86)\Tesseract-OCR, LOCALAPPDATA\Programs\Tesseract-OCR, and LOCALAPPDATA\Tesseract-OCR. An invalid explicit or environment override refuses instead of falling back. For a custom installation or language-data directory:

$env:PIL_AGENT_TESSERACT = 'C:\Tools\Tesseract-OCR\tesseract.exe'
$env:TESSDATA_PREFIX = 'C:\Tools\Tesseract-OCR\tessdata'
& .\.venv\Scripts\python.exe .\scripts\pil_ocr.py --diagnose --lang eng
& .\.venv\Scripts\python.exe .\scripts\pil_ocr.py 'C:\Images\sign.png' --lang eng
# A one-command override takes precedence over the environment:
& .\.venv\Scripts\python.exe .\scripts\pil_ocr.py --diagnose --tesseract-executable 'C:\Program Files\Tesseract-OCR\tesseract.exe'

--diagnose probes the executable version and installed languages, including each language requested in a combination such as eng+deu. Install the matching Tesseract .traineddata files if a language is missing. Clear an obsolete override with Remove-Item Env:PIL_AGENT_TESSERACT or Remove-Item Env:TESSDATA_PREFIX.

Obtain an ONNX model separately and replace the example path below with its actual location. For the gated models, check the SHA256 against Requirements above; a different export is a different model even if its filename matches. Configure the matching preprocessing profile explicitly:

$env:PIL_AGENT_EMBED_MODEL = 'C:\Models\mobilenetv2-12.onnx'
$env:PIL_AGENT_EMBED_PREPROCESSING = 'imagenet'
Get-FileHash -Algorithm SHA256 -LiteralPath $env:PIL_AGENT_EMBED_MODEL
& .\.venv\Scripts\python.exe .\scripts\pil_embed.py diagnose
& .\.venv\Scripts\python.exe .\scripts\pil_embed.py embed 'C:\Images\photo.jpg'

# For the CLIP visual-tower model described in Requirements:
$env:PIL_AGENT_EMBED_MODEL = 'C:\Models\clip-vit-b32-visual.onnx'
$env:PIL_AGENT_EMBED_PREPROCESSING = 'clip'
& .\.venv\Scripts\python.exe .\scripts\pil_embed.py diagnose
# Flags override the corresponding environment variables:
& .\.venv\Scripts\python.exe .\scripts\pil_embed.py diagnose --model 'C:\Models\mobilenetv2-12.onnx' --preprocessing imagenet

These $env: assignments affect the current PowerShell session and child processes. To persist embedding configuration for future processes:

[Environment]::SetEnvironmentVariable('PIL_AGENT_EMBED_MODEL', $env:PIL_AGENT_EMBED_MODEL, 'User')
[Environment]::SetEnvironmentVariable('PIL_AGENT_EMBED_PREPROCESSING', $env:PIL_AGENT_EMBED_PREPROCESSING, 'User')

Restart an already-running agent or terminal to inherit persisted changes. Embedding diagnostics import ONNX Runtime, load the selected model on CPU, check its declared input size against the profile, and report the interpreter, runtime, model path/hash, gate status, and profile. This is a setup check with no inference or calibration; it cannot detect a same-sized model paired with the wrong normalization profile. Existing per-model capability claims still apply unchanged. compare continues to require neither runtime nor model file.

Both diagnostic commands emit JSON on success (exit 0). Dependency/setup refusals use exit 2, a named reason on stderr, and empty stdout, just like normal OCR/embedding refusal. Check $LASTEXITCODE before consuming output. If ONNX Runtime reports a Windows DLL load failure, install the current Microsoft Visual C++ Redistributable matching your Python architecture, as required by the ONNX Runtime installation guide. For x64 Python, the PowerShell installation command is:

winget install --id Microsoft.VCRedist.2015+.x64 --exact
uv sync --extra embedding
& .\.venv\Scripts\python.exe .\scripts\pil_embed.py diagnose

Plugin registration

Then install it as a plugin. The Claude Code CLI installs from marketplaces, not from a bare directory path, so the repository ships a single-plugin marketplace manifest and registers itself:

claude plugin marketplace add ./                                 # register this repo
claude plugin install pil-agent-plugin@pil-agent-plugin -s user  # all your projects
# or: ... -s project   for this project only

Verify with claude plugin list (or /plugin inside Claude Code) — you should see pil-agent-plugin at scope user, enabled, with four skills and one agent. claude plugin details pil-agent-plugin prints the component inventory.

Two things worth knowing:

• Installing takes a snapshot copy, versioned under ~/.claude/plugins/cache/. Edits to your clone do not reach an installed plugin until you run claude plugin update pil-agent-plugin. If uv sync has already created .venv/, that is copied too — budget ~65 MB.
• claude plugin validate . now validates the marketplace manifest, which takes precedence at the repository root. To validate the plugin manifest, point at it directly: claude plugin validate .claude-plugin/plugin.json --strict.
• Editing marketplace.json needs a refresh: claude plugin marketplace update pil-agent-plugin. Editing the plugin itself needs claude plugin update pil-agent-plugin. They are separate caches.

Installing without uv

The scripts only need Pillow and numpy, so any environment with those works:

pip install "pillow>=10.0" "numpy>=1.26"
claude plugin marketplace add ./
claude plugin install pil-agent-plugin@pil-agent-plugin -s user

The bundled skill prefers uv and falls back to a plain python invocation. Skipping uv sync also keeps .venv/ out of the installed snapshot.

Usage

There are three ways in, from most to least automatic.

1. Just ask

Once installed, the image-measurement skill is invoked automatically when your request calls for it. Its description triggers on questions like:

• "Do these two screenshots match?"
• "Did this render keep the reference's colour scheme?"
• "What exactly changed between these two versions?"
• "What are the exact colours in this logo?"

For work that crosses from images into calibrated multi-view constraints or Blender geometry, the image-analysis umbrella skill composes image-measurement with multiview-reconstruction. It keeps visual interpretation, pixel measurement, reconstruction residuals, and scene geometry as separate evidence layers, then combines them in one report.

2. The comparison agent

For a rigorous two-image comparison, invoke the bundled image-comparison-analyst agent. It is deliberately structured to look first and measure second:

1. Reads both images and commits to a visual description, so the numbers cannot anchor its perception.
2. Runs both measurement tools on the pair.
3. Reports where vision and measurement disagree — rather than silently deferring to whichever spoke last.

It reports findings only; it never edits files.

3. Direct CLI

Both tools emit JSON on stdout, are deterministic (repeated runs are byte-identical, so output can be committed and diffed), and accept one image (analyse) or two (analyse and diff).

uv run python scripts/pil_image_analyze.py  "reference.png"                # maximal one-call profile
uv run python scripts/pil_image_analyze.py  "reference.png" "render.png"   # full comparison, one call
uv run python scripts/pil_palette_diff.py   "reference.png"
uv run python scripts/pil_palette_diff.py   "reference.png" "render.png"
uv run python scripts/pil_structure_diff.py "reference.png" "render.png" --grid 4x3
uv run python scripts/pil_structure_diff.py "view_a.png" "view_b.png" --foreground

With the optional extras, the semantic layers run the same way — note that each embedding model is paired with the preprocessing profile it was trained with, and the payload records which was used:

uv run python scripts/pil_ocr.py "sign.jpg" --region "0.3,0.4,0.7,0.55" --psm 6
uv run --extra embedding python scripts/pil_embed.py embed "a.jpg" "b.jpg" \
    --model mobilenetv2-12.onnx     --preprocessing imagenet   # same-image ID
uv run --extra embedding python scripts/pil_embed.py embed "a.jpg" "b.jpg" \
    --model clip-vit-b32-visual.onnx --preprocessing clip      # + related-scene ranking
uv run python scripts/pil_semantic_record.py seal "a.jpg" --claims claims.json

pil_image_analyze.py is the one-call entry point: it composes the file-fact, palette and structure tools (their blocks are content-identical to standalone runs) and adds what none of them emit for a single image — persistable dhash/ahash fingerprints (hex strings comparable across runs by Hamming distance, so an image profiled today can be identified against a payload stored last week), exact tonal percentiles and clipping fractions, per-channel statistics, an exact distinct-colour count and greyscale test, and edge/sharpness diagnostics. With two images it also contains both tools' pairwise diffs verbatim plus all four fingerprint distances.

Always quote paths — image filenames routinely contain spaces and parentheses.

Use --foreground (both tools) when comparing object renders — a model on a preview backdrop, a product shot, a sprite. Full-frame metrics include the background, and a shared backdrop can be ~98% of both frames: two different swords measured 0.991 full-frame structural similarity because the background was doing the scoring. Foreground mode masks the background out (alpha when the file carries real transparency, border-median OKLab colour otherwise — the same visible-pixel definition as the Synty asset index), crops to the object's bounding box so position in frame stops mattering, and scores only grid cells with real foreground support. When you don't pass it, the tools estimate foreground coverage anyway and flag background_dominant on mostly-background frames — treat that flag as "these scores describe the backdrop".

Concept-image depth (pil_depth)

A concept image has no depth data. pil_depth estimate asks a monocular depth model for one, and pil_depth compare checks that answer against the exact depth of a Blender render. The output of estimate is model-inferred relative inverse depth (larger = nearer, arbitrary scale and shift per image): it is not metric, carries no scene units, and for stylised concept art it is a learned guess. The payload says so (depth_kind: "relative_inverse_depth", metric: false, interpretation_limits).

uv run --extra embedding python scripts/pil_depth.py estimate concept.png \
    --model depth_anything_v2_vits.onnx --output-dir out/ [--mask selection.json]
uv run python scripts/pil_depth.py compare out/concept_inverse_depth.npy \
    render_depth.npy --output-dir out/ [--mask mask.png]

• estimate writes <stem>_inverse_depth.npy (float32, the image's height x width) and a heatmap PNG, and records the model file, its sha256 and the named preprocessing profile (depth-anything-v2: shorter side 518, each side a multiple of 14, bicubic, ImageNet mean/std, or the model's own static input size when it has one). --mask is a selection-mask-v1 manifest from pil_mask.py bound to that image; it scopes the statistics and heatmap normalisation, while the .npy stays the full model output. Alpha is composited onto black before inference.
• compare takes that .npy and the .npy from blender_inspect_render --modes depth (metric depth, NaN background). It converts the render to inverse depth, fits scale and shift by least squares over the pixels valid in both (and inside --mask, a binary PNG at the render's size), and reports the Spearman rank correlation, AbsRel after alignment, the fitted scale/shift, and a disagreement heatmap with the worst grid cells as fractional L,T,R,B boxes. The render's grid is the target: a concept map of a different size is bilinearly resampled to it only when the aspect ratios agree within 1%, otherwise compare refuses. It cannot check that the two maps show the same framing beyond that.
• Nothing is calibrated: there is no pass/fail threshold for Spearman or AbsRel, and because scale and shift are fitted away, agreement says the depth order and shape match, not that distances are right.

Obtaining the model. The tool needs an ONNX export of Depth Anything V2 Small (the ViT-S variant, about 25M parameters). Only the Small model is released under Apache-2.0; the Base, Large and Giant models carry a CC-BY-NC-4.0 (non-commercial) licence, so do not substitute them where that matters. The upstream weights are published by the Depth Anything V2 authors (github.com/DepthAnything/Depth-Anything-V2); an ONNX export is either made from those weights or taken from a community export of the Small model. This repository does not bundle or download it, and does not pin a download URL. Read the licence of the file you actually obtain, then confirm it loads with pil_depth diagnose --model <file> (or pil_bootstrap check --depth --depth-model <file>): the payload gives its sha256, and model_known is false until that hash has been run in this repository.

Worked example

Two images, identical in layout and lightness. The only difference is that one small accent stripe changes from cyan to orange.

uv run python scripts/pil_palette_diff.py before.png after.png

{
  "accent_hue_shift_detected": true,
  "hue_families_lost":      ["cyan"],
  "hue_families_gained":    ["orange"],
  "hue_families_diminished":["cyan"],
  "base_palette_distance":  1.6432,
  "accent_palette_distance": 67.6892
}

Now the same pair through the structure tool:

{
  "structural_similarity": 0.999097,
  "dhash_distance": 0,
  "changed_area_fraction": 0.007241,
  "changed_region_bbox_fractional": [0.0625, 0.1871, 0.3164, 0.2222]
}

This is the whole argument in one example. structural_similarity says 0.999 and dhash_distance says 0 — both are blind to the change, because they are luminance-based and the edit preserved luminance. Only the hue census catches it. Meanwhile changed_region_bbox_fractional pinpoints where it happened.

A caller reading only a similarity score or a hash would have concluded "identical".

Choosing a metric

For input preparation, selections, registration, local change regions, domain profiles, discovery and MCP, use the 0.9.0 workflow guide.

Consult both tools for any "do these match?" question. They are blind to different things. And read flags before any score: background_dominant, accent_support_low, foreground_too_small and friends each name a specific way the numbers beside them are weakened. accent_hue_shift_detected is support-gated — a hue family needs both a minimum pixel count and a minimum frame fraction before its appearance or disappearance can flip the verdict, so a stray anti-aliased pixel cannot — but with accent_support_low flagged, even the gated verdict should not influence a decision.

Colour is reported three ways, deliberately

Read hue_families when a colour matters semantically. Area-weighted palettes systematically miss small vivid accents. In the validation image, a cyan occupying 0.485% of the frame encoded an entire UI state — and it appeared in neither palette, only in the hue census. Semantic importance and pixel area are uncorrelated.

Do not rely on base_palette_distance for colour-scheme questions: measured against a real recolour, it scored the changed image as more similar than an unchanged rescale.

Direction of comparison

hue_families_lost / _gained / _diminished / _amplified and hue_family_fraction_deltas describe what the second image did relative to the first. accent_hue_shift_detected and both palette distances are symmetric under swapping the two.

Scope limit: these tools do not measure geometry

edge_mean and entropy are 2D image-complexity proxies. They are not polygon counts, mesh density, or topology, and must not be used as a proxy for them. Shading, normal maps, lighting and camera angle all move these numbers independently of the underlying model — a smooth-shaded low-poly render can score as more complex than a flat-shaded high-poly one.

For polygon, mesh or topology questions, query the 3D scene's own statistics (for example via a Blender MCP server) instead of analysing a render. Inferring geometry from pixels produces confident, wrong answers.

This limit is emitted in every pil_structure_diff payload under interpretation_limits, and enforced by a test, so it travels with the data rather than living only in this README.

Development

uv sync
uv run pytest -v

972 tests in tests/ (plus 312 in plugins/blender-inspect/tests, run with uv run pytest -q plugins/blender-inspect/tests). Fixtures are generated synthetically in-process, so no binary test assets are committed.

Six tests confirm results against a real reference image and skip when it is absent — so a fresh clone reports those six as skipped, which is expected. Their strongest assertions are duplicated unskipped against a synthetic stand-in, so a clean checkout still guards every known regression. A further nineteen tests across tests/test_blender_mesh.py, tests/test_blender_render.py and tests/test_character_sheet_review.py run only when an external corpus and a Blender install are both present — the corpus path defaults to a known location and can be overridden per test file, mirroring PIL_AGENT_REFERENCE_IMAGE (see runs/2026-08-20-blender-mesh-validation/, runs/2026-08-20-blender-render-validation/, runs/2026-08-20-character-sheet-loop/); they are already counted as passing above on a machine that has both, and skip cleanly otherwise.

To run those six, point the env var at any complex, predominantly dark image with small vivid accents:

PIL_AGENT_REFERENCE_IMAGE="/path/to/image.png" uv run pytest -v

The original validation image is not distributed.

How this was validated

Design decisions here came from measurement, not anticipation. The evidence bundle in runs/2026-08-18-pil-agent-plugin-phase1/ contains a 13-step RED→GREEN ledger, the raw JSON outputs, and a metric discrimination matrix showing that only 4 of 11 metrics separate a genuine colour change from a no-op rescale — and that one metric answers the question backwards.

The tools were then taken to a real production task — reviewing a Blender game character against its concept sheet — in runs/2026-08-18-skeleton-warrior-asset-review/. That trial is the strongest evidence for the vision-first method in this repository: four of the reviewing agent's own confident visual conclusions were wrong, and measurement caught all four before they reached the defect list. Two would have sent an artist to fix work that was already correct. It also records pil_structure_diff correctly refusing the comparison via its own aspect_ratio_mismatch flag, and draws a hard line between what the plugin measured and what harness code around it measured.

The same rule governs the embedding layer: a capability is advertised only where a discrimination gate measured it. The first gate (runs/2026-08-31-embedding-discrimination/) demoted same-venue matching under mobilenetv2-12 because two exhibits of one museum (0.4558) scored below an unrelated pair (0.4701). The follow-up (runs/2026-09-02-clip-embedding-discrimination/) re-ran that identical corpus and pair list against a CLIP ViT-B/32 visual encoder, which separates all three bands — so the capability is advertised for that model only, keyed by its sha256, and every ungated model is flagged as such. That run also records a control worth knowing: feeding CLIP the wrong preprocessing profile does not fail loudly, it quietly costs 36% of the margin.

Documentation

• docs/index.md — documentation index and open items
• docs/design-rationale.md — why each metric exists, and what failed along the way
• docs/blender-inspect-spec.md — the two-plugin design, the migration and deprecation plan, and the requirements for blender-inspect and pil_depth
• docs/phase2-scope.md — planned work: perceptual ΔE2000 colour distance, threshold calibration, contract-driven verdicts

Status

0.10.0 — two-plugin layout and concept-image depth. The repository now ships two plugins from one marketplace: pil-agent-plugin (image measurement) and blender-inspect (Blender scenes; see plugins/blender-inspect/). The Blender tools pil_blender_mesh, pil_blender_fit, pil_blender_render and pil_multiview_render are deprecated: they print a deprecation notice, are marked deprecated in the capability catalog, and are removed in the next minor release. They behave as before except pil_multiview_render: in Blender 5.2 it put every horizontal view's camera at the world origin, and it now places them correctly, so its renders change. Their replacements are blender_mesh, blender_fit, blender_render and blender_multiview_render in blender-inspect. New: pil_depth (Depth Anything V2 Small ONNX, caller-supplied and sha256-pinned) estimates relative depth of a concept image and compares it with a Blender render's exact depth. Skills and the comparison agent route Blender work to blender-inspect.

0.9.6 — literal environment reads. OCR install discovery and the test suite read environment settings by literal name, and process tests start from the allowlisted tool environment. No image-processing behavior changed.

0.9.5 — directory review clean-up. pil_embed.py reads its two settings by literal name, and the image-analysis skill names a concrete discovery command. No image-processing behavior changed.

0.9.4 — lean plugin tree. Working evidence bundles under runs/ are no longer tracked; the published copies remain at the pil-agent-plugin--v0.9.3 tag. The lockfile covers Python 3.11–3.13, the versions CI tests. No image-processing behavior changed.

0.9.3 — credential isolation. MCP tool processes inherit only the OS and local model/OCR settings they need. Bootstrap ignores local package-manager configuration and credential stores when installing public dependencies. The plugin itself needs no user credential.

0.9.2 — plugin submission metadata and privacy. Adds an icon to the Codex manifest and explains local processing, setup downloads, and host-agent data handling. Includes a linked privacy policy and ignores common local credential files. No image-processing behavior changed.

0.9.1 — Astra-ready skill guidance. Rewrites the four shipped skill entrypoints around concise activation, progressive disclosure, task-specific routing, proportional decision boundaries, and explicit completion. The measurement, calibration, embedding, reconstruction, and Blender behavior is unchanged. See the Astra skill audit.

0.9.0 — explicit measurement workflows and agent tooling. Adds EXIF/ICC normalization with preserved alpha and coordinate provenance; model/profile configuration-aware embedding claims; image-bound raster/polygon masks and a pipeline that composes the original image tools; bounded translation/rigid/affine registration with raw/aligned evidence; native ΔE/alpha change regions, heatmaps, crops and optional standard SSIM; domain profiles with source-group holdouts, leakage checks, measured error rates and profile identity checks; and a public tool catalog, failure-preserving batches and optional stdio MCP adapter. Bootstrap accepts --comparison and --mcp. Legacy default measurements remain unchanged apart from the release version and corrected embedding eligibility. New defaults remain diagnostic unless a compatible evaluated profile is used. See usage and execution evidence.

0.8.0 — platform bootstrap and dependency diagnostics. Adds a Python bootstrap for a plugin-local environment on Windows, macOS, and Linux, with opt-in OCR, embedding, and reconstruction dependencies. Its check command distinguishes a recorded successful bootstrap from current readiness, rechecks live dependencies, and skips repeat installation only when both pass. A bundled bootstrap skill guides agents through setup and status checks.

OCR now discovers standard Windows Tesseract installations and supports an environment override and image-free executable/language diagnostics. Embedding diagnostics validate runtime/model setup; DLL and model-load failures preserve exit 2 with empty stdout. PowerShell installation and model/profile configuration commands are documented. Focused process tests exercise missing/discovered dependencies, override precedence, paths with spaces, and bootstrap lifecycle. Model weights remain caller-supplied; embedding claims and calibration results are unchanged.

0.7.0 — one-call profiling and the semantic layers. pil_image_analyze composes the file-fact, palette and structure tools into a single maximal profile and adds persistable hex fingerprints, tonal percentiles and clipping fractions, channel statistics and (uncalibrated) detail diagnostics. pil_semantic_record seals vision claims to an image's exact sha256 — attributable and checkable, never a truth claim. pil_ocr reads text with frame-mapped boxes and feeds those records. pil_embed adds ONNX embedding fingerprints under the new optional embedding extra, with per-model preprocessing profiles and advertised capabilities keyed by model sha256: mobilenetv2-12 is gated for same-image identification only, while CLIP ViT-B/32 also passes same-venue ranking on full band separation (runs/2026-09-02-clip-embedding-discrimination/). Ungated models advertise nothing. Existing tool payload shapes are unchanged.

0.6.0 — constrained multi-view reconstruction. The optional multiview-reconstruction skill adds OpenCV contour preparation, a SciPy least-squares template solver with full-rank and conflict refusal states, Blender BVH clearance probing/bounded copy fitting, arbitrary locked-framing Workbench renders, and worst-case review over every named view. The core Pillow/NumPy install and existing image-measurement behavior are unchanged. See docs/phase4-scope.md.

0.5.0 — Phase 3 complete. All of Track A5 and Track B1/B2/B3 now ships alongside Track A1-A4 from 0.4.0. The release also closes the first PR #7 critic round: hard-failed views make every contract predicate UNMEASURABLE, character-sheet JSON contains stable logical render identifiers rather than deleted temporary paths, support-insufficient comparisons refuse, and Blender renders are staged atomically so failure preserves any caller-owned output.

Track B2 + B3 — the character-sheet loop ships. pil_blender_render.py renders front/side/back from a .blend scene (headless Blender Workbench, auto-framed from the mesh bounding box) and registers each against a caller-supplied reference via pil_structure_diff --foreground; pil_character_sheet_review.py composes those matched views into one pil_contract_verdict --pairs call, so a whole character-sheet review is a single contract evaluated over N registered pairs — worst-case aggregated, proven end to end on real renders (a deliberately swapped view is not averaged away by two good ones; a hard-failed view is represented, not dropped). Verified against a real production character (C:\Projects\tms-heim\art\skeleton-crusaders\brute\): the camera-axis convention was verified two independent ways rather than assumed, a real render-determinism defect (Blender's per-render PNG metadata) was found and fixed, and matched real renders VIOLATE Phase 2's calibrated similarity thresholds — an honest content-difference finding, not a defect; both new tools accept --thresholds for callers who need a different bar. Cross-machine render determinism is not claimed, only same-machine/same-install; comparison metrics on a fixed image pair remain fully deterministic everywhere. Full numbers: docs/index.md.

Track A5's three new-metric candidates each ran a real discrimination gate, and it did its job: connected-component counting, silhouette shape descriptors and projection-profile alignment were all demoted — each tool still exists, is tested, and degrades honestly (a demoted candidate reports real diagnostic numbers with an explicit low-confidence flag, never a silent overclaim), but none is advertised as a validated discrimination capability. The one A5 candidate that ships is pil_alignment's WCAG contrast half, which needed no gate — verified against the standard's own published worked examples.

Track B1 ships: pil_blender_mesh.py reads real polygon/vertex/material counts and bounding dimensions from a Blender scene headlessly, and pil_contract_verdict.py's geometry.* predicates now resolve real verdicts when scene stats are supplied (refusing exactly as before when they are not). Verified against a real production asset's revision history — including resolving a genuine discrepancy between two sources of "ground truth" and finding a claimed "topology-preserving" pair was actually scene-level violated. Full numbers: docs/index.md, docs/phase3-scope.md.

0.4.0 — coverage-weighted foreground, three new tools, region scoping.

The headline fix: on RGBA input in --foreground mode, statistics are now coverage-weighted. Previously a partially-covered edge pixel was composited onto black and then counted like a fully-opaque one, which read a 5px-wide blade 27.9 code values too dark and — worse — made the reading depend on where the object landed on the pixel grid, so a quarter-pixel re-render of an unchanged asset could cross a calibrated threshold and report a change that never happened. Measured after the fix: error against alpha-weighted truth is ±0.0004 across all 19 corpus scenes, and the placement excursion is ±0.0006 against a bound of 0.5. Full-frame behaviour is byte-identical.

Three new tools close the loop between measurement and vision: pil_crop hands vision a native-resolution view of a region the tools located, pil_annotate draws numbered boxes so a model can point precisely, and pil_image_info reports the file facts an image never carries into a vision encoder. --region scopes every metric on both diff tools, byte-equal to pre-cropping with pil_crop.

Foreground thresholds are now split by mask source and the alpha path is calibrated for the first time (luminance 0.997 against the estimate path's 34.166, n=380, α=0.01). The annotation tool's legibility is verified by read-back — fresh agents shown only the image transcribed 27 of 27 numerals correctly, and the bundle records an earlier round that failed.

Phase 1 complete: tools built and validated, plugin packaged, claude plugin validate --strict passing, and the package audited against and conforming to Agent Plugins 1.0.0.

0.2.0 adds foreground separation, driven by a production failure: run over two asset renders sharing a preview background, full-frame similarity scored two different objects 0.991. Both tools now estimate foreground coverage and flag background_dominant, accept --foreground for masked, bbox-registered measurement (alpha-derived or border-median OKLab, matching the Synty asset index's visible-pixel definition), and support-gate the hue-shift verdict so a handful of anti-aliased pixels cannot flip it.

0.3.0 implements phase 2 in full: CIEDE2000 colour distance hand-rolled in numpy and verified against all 34 published Sharma reference values (base/accent_palette_distance_de2000 are now the primary colour signal); Neyman–Pearson threshold calibration over synthetic ground truth with published per-metric detection limits (the calibration bundle records every derived constant with its n, α and CI — including one derivation that was rejected, with the reason in scripts/pil_common.py); an opt-in LCh accent gate (--accent-space lch); and pil_contract_verdict.py — declared-intent verdicts where every null result carries its detection limit, geometry.* and style.* refuse rather than approximate, and multi-pair aggregation is worst-case so one broken view cannot be averaged away.

Those thresholds were then validated against a real production corpus — a game asset's actual revision history — rather than only the synthetic data they were derived from: zero false alarms across 160 real no-change controls in full-frame mode, 21 of 24 real revision pairs detected in both modes, and the three undetected ones below the published 2%-of-frame detection limit exactly as that limit predicts. The published detection limits transfer to production input, which is what makes a SATISFIED invariant's limit worth reading.

Known limitations, tracked in docs/index.md:

• Thresholds are calibrated against a small sample and need broader validation.
• CIEDE2000 is the primary palette-distance signal; legacy Euclidean RGB distance remains supporting detail.
• Declared-intent contracts are supported, but geometry/style claims still require their own evidence, and thresholds do not transfer between domains.
• Domain profiles require representative source-disjoint data. The four-domain execution examples demonstrate operation, not production-domain accuracy.

License

MIT — see LICENSE.