Skip to content

xhoantran/scrapeio

v1.0.0MIT

Public social media data for agents: profiles, posts, comments, transcripts, search, trends and ad libraries from TikTok, Instagram, YouTube, X, Facebook and 28 more platforms, through ScrapeIO's remote MCP server.


A library of agent skills for social media research, plus the MCP server and plugin manifests that power them.

Skills

SkillWhat it does
scrapeioCore: pick the right endpoint, page, report numbers with links
influencer-discoveryFind creators in a niche, filter by followers and real engagement, export a shortlist
creator-vettingSpot fake followers, bought engagement and brand-safety issues before a deal
competitor-social-auditBenchmark brands across TikTok, Instagram, YouTube, X and LinkedIn
video-transcript-researchTranscripts from YouTube, TikTok, reels, X and LinkedIn; summaries, quotes, mentions
comment-insightsTurn comments into questions, objections, sentiment and buying signals
social-trend-researchTrending videos, hashtags and sounds by country, sized and dated
ad-library-researchCompetitors' ads in the Meta, Google and LinkedIn ad libraries

Install all of them (you can pick which ones when prompted):

npx skills add xhoantran/scrapeio-agent

Every skill works two ways: through the MCP server's tools when it is connected, or over plain HTTP with curl when it is not.

Also in this repo

  • MCP server at https://scrapeio.com/mcp (Streamable HTTP): 85 tools such as tiktok_profile, instagram_comments, youtube_transcript, twitter_search, plus list_endpoints and call_endpoint for the long tail.
  • Plugin manifests for Claude Code, Cursor and Hermes Agent (Agent Plugins v1), so one install adds the server and all skills.

Ask things like:

Compare @nike and @adidas on TikTok: followers, average views on their last 20 videos, and the top 3 posts.

Pull the transcript of this YouTube video and list every product it mentions.

Find 20 fitness creators on Instagram with 10k to 100k followers.

1. Get a key

Sign in at scrapeio.com/login (100 free credits, no card). Then set it once:

export SCRAPEIO_API_KEY=sk_...

2. Install

Claude Code

claude mcp add --transport http scrapeio https://scrapeio.com/mcp --header "x-api-key: $SCRAPEIO_API_KEY"

Or install the plugin (MCP server plus all skills):

/plugin marketplace add xhoantran/scrapeio-agent
/plugin install scrapeio@scrapeio

Cursor

Add to Cursor (replace YOUR_SCRAPEIO_KEY after install), or add this to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "scrapeio": {
      "url": "https://scrapeio.com/mcp",
      "headers": { "x-api-key": "${env:SCRAPEIO_API_KEY}" }
    }
  }
}

Codex

In ~/.codex/config.toml:

[mcp_servers.scrapeio]
url = "https://scrapeio.com/mcp"
bearer_token_env_var = "SCRAPEIO_API_KEY"

Hermes Agent

hermes plugins install xhoantran/scrapeio-agent
hermes plugins enable scrapeio

Hermes asks for the key on install and stores it as SCRAPEIO_API_KEY.

Gemini CLI

In ~/.gemini/settings.json:

{
  "mcpServers": {
    "scrapeio": {
      "httpUrl": "https://scrapeio.com/mcp",
      "headers": { "x-api-key": "$SCRAPEIO_API_KEY" }
    }
  }
}

VS Code (Copilot agent mode)

In .vscode/mcp.json:

{
  "inputs": [{ "type": "promptString", "id": "scrapeio-key", "description": "ScrapeIO API key", "password": true }],
  "servers": {
    "scrapeio": {
      "type": "http",
      "url": "https://scrapeio.com/mcp",
      "headers": { "x-api-key": "${input:scrapeio-key}" }
    }
  }
}

Windsurf

In ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "scrapeio": {
      "serverUrl": "https://scrapeio.com/mcp",
      "headers": { "x-api-key": "${env:SCRAPEIO_API_KEY}" }
    }
  }
}

Anything else

Any MCP client that speaks Streamable HTTP works: URL https://scrapeio.com/mcp, header x-api-key: <key> (or Authorization: Bearer <key>). The server is also in the official MCP registry as com.scrapeio/scrapeio.

No MCP? Drop AGENTS.md into your project root and the agent will call the HTTP API directly.

What it costs

Each tool call spends credits; the tool description lists how many. Cached answers and failed calls are free, and credits never expire. See pricing.

Privacy and network use

The MCP server and skill send requests only to scrapeio.com. Tools read public data and never post, follow or message on any platform. No telemetry is collected by this package. ScrapeIO logs which endpoint each key calls, for billing.

License

MIT


This repository used to host Open Interview Coder (archived in 2025). Its code is in this repo's git history and at xhoantran/open-interview-coder.