saurav-shakya/neoyngpt-video-to-code

What this is

NeoynGPT:video-to-code is an agent skill plus a small CLI. It does not play YouTube inside the editor. It sends a public YouTube URL (or uploads a local .mp4 / .webm / .mov) to Gemini, samples the pixels, and returns visual notes any Agent Skills harness can act on.

It is not locked to one tool. Cursor, Claude Code, Codex, Antigravity CLI (agy), and any other host that loads SKILL.md all work. Gemini CLI moved to Antigravity CLI — use that.

It is not a transcript tool. If you only care about what someone said, use captions instead.

Harness support

A harness is the agent runner (IDE chat or terminal) that loads skills and lets the model call tools. This repo ships a standard SKILL.md, so it supports all Agent Skills harnesses — not a Cursor-only or Claude-only plugin.

The vision step always uses the Gemini API (GEMINI_API_KEY). The harness is whoever runs the skill afterward.

How it works

%%{init: {"theme": "dark", "themeVariables": {"primaryColor": "#16122E", "primaryTextColor": "#F5F7FF", "primaryBorderColor": "#8B7CFF", "lineColor": "#2EE6D6", "secondaryColor": "#0A1022", "tertiaryColor": "#070B16", "fontFamily": "ui-sans-serif, system-ui, sans-serif"}}}%%
flowchart LR
  A["YouTube URL<br/>or local recording"] --> B["analyze.mjs"]
  B --> C["Gemini samples pixels<br/>~1 fps, plus audio"]
  C --> D["Visual facts<br/>+ timestamps"]
  D --> E["Agent maps them<br/>onto this repo"]

1. You paste a public YouTube URL or attach a recording, plus a visual question.
2. The skill runs scripts/analyze.mjs (Agent mode only — it shells out to Node).
3. Gemini watches a time window (default 3 minutes if a start time exists).
4. Stdout is treated as ground truth for pixels, not as code. The agent then applies it here.

Honesty: this is sampled video, not a frame-perfect 60 fps watch. You can say a tab slides with a bounce. You cannot invent damping: 0.82 unless that number is on screen.

Download

Need Node 18+ and a Gemini API key from Google AI Studio. Pick one path.

Latest code always lives on main.

Install the skill (any harness)

Installs from GitHub (latest main) into every common harness folder:

npx skills add saurav-shakya/NeoynGPT-video-to-code -g

Then once, in the folder your harness uses:

cd ~/.cursor/skills/neoyngpt-video-to-code          # Cursor
# cd ~/.claude/skills/neoyngpt-video-to-code         # Claude Code
# cd ~/.agents/skills/neoyngpt-video-to-code         # Codex / project-style hosts
# cd ~/.gemini/antigravity-cli/skills/neoyngpt-video-to-code   # Antigravity CLI
npm install

Antigravity CLI (replaces Gemini CLI) — after the copy, you can also drop the skill at the project root:

mkdir -p .agents/skills
cp -R skills/neoyngpt-video-to-code .agents/skills/
# then: agy   →  /neoyngpt-video-to-code

Claude Code plugin (same repo):

/plugin marketplace add saurav-shakya/NeoynGPT-video-to-code

Then install neoyngpt-video-to-code. No plugin? Copy skills/neoyngpt-video-to-code into that harness’s skills folder and run npm install there.

In Cursor: Agent mode → new chat → /neoyngpt-video-to-code or paste a YouTube URL. Ask / Chat mode cannot run this. Same idea in Claude Code, Codex, and Antigravity CLI (agy): the harness must be allowed to run shell.

CLI from npm

Installs from npm:

npm install -g neoyngpt-video-to-code

Try it:

neoyngpt-video-to-code --help

neoyngpt-video-to-code --url "https://www.youtube.com/watch?v=jSWuepkuFrU&t=75s" \
  --question "How does the calorie tracker UI feel? What’s moving on the right?"

neoyngpt-video-to-code --file ./recording.mp4 \
  --question "The tab transition stutters. What is on screen?"

No global install — one-off:

npx neoyngpt-video-to-code --help

Need the GitHub commit before it hits npm?

npm install -g github:saurav-shakya/NeoynGPT-video-to-code

Clone the repo

git clone https://github.com/saurav-shakya/NeoynGPT-video-to-code.git
cd NeoynGPT-video-to-code
npm install
npx neoyngpt-video-to-code --help

Set your Gemini key

Required for every path. Either export it, or copy .env.example to .env in the project (gitignored). GOOGLE_API_KEY works as a fallback.

export GEMINI_API_KEY=your_key
# optional
# export GEMINI_MODEL=gemini-3.6-flash

# .env
GEMINI_API_KEY=your_key

Update later

# Skill — latest GitHub
npx skills add saurav-shakya/NeoynGPT-video-to-code -g

# CLI — latest npm
npm install -g neoyngpt-video-to-code@latest

# Clone
git pull origin main && npm install

Use it

In your harness

This skill cannot run in Ask / Chat (read-only) mode. The harness must be able to run Node (Cursor Agent, Claude Code, Codex, Antigravity CLI agy, or any other Agent Skills host). Start a new session, then type /neoyngpt-video-to-code or paste a YouTube URL.

Cursor loads skills from .cursor/skills/ and ~/.cursor/skills/. Antigravity CLI loads ~/.gemini/antigravity-cli/skills/ and project .agents/skills/. This repo already links the skill under .cursor/skills and .agents/skills. Confirm in your harness’s skill list.

The agent must run scripts/analyze.mjs before answering visual questions. If it did not run, it did not watch the video.

YouTube demo

https://www.youtube.com/watch?v=jSWuepkuFrU&t=75s How does the calorie tracker UI feel? What’s moving on the right?

Your app’s recording

[attach bug.mp4] At 0:08 the tab janks. What’s on screen, and how do we fix it in this repo?

Prefer a timestamp on long videos (t=75s in the link, or say “start at 1:15”).

Good questions vs the wrong tool

What you get back

Answers are shaped like this:

1. Visual facts — timestamps + what is actually visible (motion, spacing, icons, jank)
2. Design principles — why that craft would feel native instead of “vibe-coded”
3. How to apply or fix it here — mapped onto the open repo

Example shape (shortened):

## Visual facts
- 00:08–00:12  Tab bar: outline icons idle, filled icon on the selected tab.
- 00:12        Page push from the right with a short overshoot (spring), not a fade.

## Design principles
- One icon family. Line = idle, filled = active. Mixing stroke and fill in one bar looks broken.

## Apply here
- Use the existing TabBar component. Idle = outline, selected = filled. Don’t add a second icon set.

If a repo is open:

• YouTube demo → 3 concrete UI/motion takeaways for existing components
• Your recording → name the broken motion/layout, then fix this codebase

Limits

Live streams and premieres are unreliable until a public VOD exists.

CLI

npx neoyngpt-video-to-code --url <youtube-url> --question "<visual question>"
npx neoyngpt-video-to-code --file <recording.mp4> --question "<visual question>"
# after: npm install -g neoyngpt-video-to-code
neoyngpt-video-to-code --url <youtube-url> --question "<visual question>"

From a clone you can also run:

npm run analyze -- --url "https://youtu.be/jSWuepkuFrU?t=75" --question "What is on screen?"
node skills/neoyngpt-video-to-code/scripts/analyze.mjs --help

Pass exactly one of --url or --file.

Stdout = visual answer. Stderr = diagnostics (upload, model fallback, quota).

Troubleshooting

More detail: skills/neoyngpt-video-to-code/references/prompts.md and providers.md.

Develop

npm test
node skills/neoyngpt-video-to-code/scripts/analyze.mjs --help

Layout:

skills/neoyngpt-video-to-code/
  SKILL.md              ← what the agent reads
  scripts/analyze.mjs   ← Gemini API video CLI
  lib/parse.mjs         ← URL, timestamps, flags
  lib/env.mjs           ← .env loader
  references/           ← prompts + provider limits

License

MIT © 2026 NeoynGPT:video-to-code contributors