imagegen-sidekick

A portable Agent Plugins 1.0 plugin bundling media-generation skills:

• gpt-imagegen generates and edits images using the OpenAI Images API against Azure OpenAI, with guidance for Blender-ready 2D assets.
• foundry-tts creates SSML and synthesizes speech with the Foundry Speech SDK (MAI-Voice-2, Harper), writing the SSML, audio and a JSON word timeline.
• walkthrough-video records narrated web app walkthroughs (MP4) with Playwright. Actions follow the foundry-tts word timeline, with highlights, labels, an animated cursor, gpt-imagegen image cards and karaoke captions.

Install and configure

copilot plugin install .

Or, after publishing this repository:

copilot plugin install vieiraae/imagegen-sidekick

Restart the session and check /skills list. Each skill installs its dependencies on first use. For manual setup from the plugin root:

python3 -m venv skills/gpt-imagegen/resources/.venv
skills/gpt-imagegen/resources/.venv/bin/python -m pip install -r skills/gpt-imagegen/resources/requirements.txt

Install Azure CLI and sign in with your identity:

az login
az account set --subscription "<subscription-name-or-id>"

For a different directory, use az login --tenant "<tenant-id>". Your identity needs Azure OpenAI data-plane access, such as Cognitive Services OpenAI User on the foundry-imagegen resource. Subscription Reader/Contributor alone does not grant model-inference access.

The script uses AzureCliCredential with a refreshable bearer-token provider, not API keys or a default credential chain. Copilot must run as the same OS user with access to az and its login session. No credentials belong in .env or .env.local; the latter is only for local setting overrides. Installed plugins are cached: reinstall to pick up changes.

Defaults in skills/gpt-imagegen/.env:

Precedence: .env < .env.local < process environment < CLI flags. Azure must already have the named deployment and support the requested parameters; the plugin does not provision it or silently fall back to another model.

Use

Ask Copilot to "generate a transparent leaf decal for Blender" or "edit this photo to replace the sky, preserving the buildings." For direct CLI use:

skills/gpt-imagegen/resources/.venv/bin/python skills/gpt-imagegen/resources/imagegen.py generate \
  --prompt "Seamless stone base-color texture, orthographic, evenly lit, no shadows" \
  --background opaque --quality high --output outputs/stone.png

skills/gpt-imagegen/resources/.venv/bin/python skills/gpt-imagegen/resources/imagegen.py edit \
  --image source.png --mask mask.png --prompt "Replace only the masked area with moss" \
  --background opaque --output outputs/moss.png

Use a new output path for each run; existing files are never overwritten. Repeat --image for references. See the concise skill for mask rules and asset guidance. Generated assets are images, not meshes, calibrated PBR maps, or HDR environments; inspect them before use in Blender.

foundry-tts

Written in TypeScript and run directly by Node 24+ (no Python, no build step):

npm install --prefix skills/foundry-tts/resources

It also uses AzureCliCredential; your identity needs Cognitive Services Speech User on the Foundry resource. The Speech endpoint is derived from the project endpoint (<resource>.services.ai.azure.com → <resource>.cognitiveservices.azure.com); set SPEECH_ENDPOINT to override.

Defaults in skills/foundry-tts/.env:

node skills/foundry-tts/resources/tts.ts \
  --text "Welcome to the scene." --style narrator --output outputs/intro.wav

This writes outputs/intro.ssml, outputs/intro.wav and outputs/intro.json (words[] with start_ms/end_ms). Use --ssml-file for hand-written SSML and --ssml-only to preview SSML without calling the service.

walkthrough-video

Requires Node 24+ and ffmpeg on PATH:

npm install --prefix skills/walkthrough-video/resources
npx --prefix skills/walkthrough-video/resources playwright install chromium   # if missing

Describe the walkthrough as JSON steps: say is the narration, and actions are highlight, click, fill, hover, press, scroll, goto or image, optionally anchored to a phrase with at. Then:

W=skills/walkthrough-video/resources/walkthrough.ts
node $W check walkthrough.json
node $W narration walkthrough.json --output out/narration.txt
node skills/foundry-tts/resources/tts.ts --text-file out/narration.txt --output out/narration.wav
node $W record walkthrough.json --timeline out/narration.json --output out/walkthrough.mp4

Highlights, cursor, image cards and karaoke captions are drawn in the page during recording. ffmpeg only trims the video, adds the audio and encodes H.264/AAC. Defaults (1280x720, 30 fps, accent colours) are in skills/walkthrough-video/.env.

Development

All executable code and dependencies live under the skill's resources/. Run offline tests without credentials or paid API calls:

skills/gpt-imagegen/resources/.venv/bin/python -m unittest discover -s skills/gpt-imagegen/resources -p 'test_*.py'
npm test --prefix skills/foundry-tts/resources
npm test --prefix skills/walkthrough-video/resources   # also records a short local MP4 with Playwright + ffmpeg

API implementation follows the OpenAI image-generation guide and Azure v1 endpoint guidance. Speech follows the MAI voices guide.