🔍 ToolLens

Give your agent 2,468 tools. It sees 3.

An MCP gateway in Rust. Put any number of MCP servers behind it: your agent discovers, inspects and calls exactly the tools it needs, and finding a tool never grants permission to run it.

imageCI imageLicense: MIT imageRust 1.89+ imageMCP imagePRs welcome

Real run, not a mock-up: Claude Opus 5.5 against 110 vendor-published MCP servers. Full 4K/60 fps video in the v0.1.0 release.

MCP clients send every tool definition with every request. Connect enough servers and that stops working: the 110 real vendor servers in this repo's benchmark expose 2,453 tool definitions, which come to 1.43M tokens per request, and the API rejects every call before the model runs. Below that limit you still pay for, and distract the model with, hundreds of tools it will never use.

ToolLens sits between the agent and your servers and shows the agent three tools instead:

Results

Same 28 tasks, same model (Claude Opus 5.5), 110 real MCP servers, 2,468 tools:

One run per task: 26 vs 25 is a tie. The result is that ToolLens matched Anthropic's server-side tool search at half the cost per task, while exposing every tool was not an option at all. Methodology, the controlled benchmark and every caveat are below; every number comes from transcripts committed in bench/results/.

What do your MCP servers cost you?

toollens tax starts the servers in your existing client config and measures what their tool definitions add to every request. It runs locally; --exact instead counts with Anthropic's free count_tokens endpoint, which means sending it the tool definitions (never env or args).

toollens tax --config .mcp.json        # Claude Code project servers; ~/.claude.json for user-scope ones
toollens tax --config "<Claude Desktop config>"   # %APPDATA%\Claude\ or ~/Library/Application Support/Claude/claude_desktop_config.json

It ranks servers and tools by tokens and lists tools whose schemas the Claude API refuses and servers that failed to start. Across the benchmark's 110 servers (full table), DigitalOcean alone is about 130K tokens per request, and its apps-create-app-from-spec tool about 19.7K by itself.

Quick start

Download the archive for your platform (Linux x86_64, macOS Apple silicon, Windows x86_64) from the latest release and unpack it. Or build from source with Rust 1.89+, then use ./target/release/toollens:

git clone https://github.com/krishnashakula/toollens && cd toollens && cargo build --release

From the unpacked folder:

# Three simulated servers (deploy, observability, vcs) plus 982 generated distractor tools:
./toollens discover --config examples/gateway.json --noise 982 --goal "investigate a failed deployment"

On macOS the binary is unsigned; if it is blocked, run xattr -d com.apple.quarantine toollens.

{
  "tools": [
    { "id": "deploy/get_deployment",   "role": "match",      "why": "matches fail, deploy" },
    { "id": "deploy/list_deployments", "role": "dependency", "why": "provides `deployment_id` that deploy/get_deployment requires" },
    { "id": "deploy/search",           "role": "dependency", "why": "provides `service_id` that deploy/list_deployments requires" },
    { "id": "vcs/get_commit",          "role": "dependency", "why": "can use `commit_sha` returned by deploy/get_deployment" },
    { "id": "observability/get_logs",  "role": "dependency", "why": "can use `deployment_id` returned by deploy/get_deployment" }
    // … 3 more direct matches
  ],
  "catalog_size": 1000,
  "latency_ms": 0.27
}

Keyword search alone finds the deployment tool. The dependency graph adds the chain the agent will actually need: the lookup that resolves a service name to its id, the logs, and the commit.

Use it from Claude Code, Claude Desktop or any MCP client

1. Put your servers behind the gateway. toollens init reads your client config, writes a gateway config with the same server entries (toollens.json beside it), and prints the client change; --write applies it and keeps a .bak. ~/.claude.json is never edited, only printed, because Claude Code rewrites it while running.

toollens init --from .mcp.json         # or claude_desktop_config.json

The gateway config uses the same mcpServers shape as Claude Desktop and Claude Code, including ${VAR} and ${VAR:-default} references, plus a policy:

{
  "mcpServers": {
    "github": { "command": "npx", "args": ["-y", "@modelcontextprotocol/server-github"], "env": { "GITHUB_PERSONAL_ACCESS_TOKEN": "..." } },
    "filesystem": { "command": "npx", "args": ["-y", "@modelcontextprotocol/server-filesystem", "/work"] }
  },
  "policy": { "default": "allow", "deny": ["github/merge_pull_request"], "allow": [], "denyDestructive": true }
}

2. Point your client at ToolLens instead of at the servers. This is what init prints. For Claude Code it goes in .mcp.json at your project root; Claude Desktop uses claude_desktop_config.json:

{
  "mcpServers": {
    "toollens": {
      "command": "/path/to/toollens",
      "args": ["serve", "--config", "/path/to/gateway.json"]
    }
  }
}

The agent now sees three tools however many servers sit behind them. To check what a goal would find before involving an agent: toollens discover --config gateway.json --goal "...".

For many servers, set "lazy": true (and optionally "maxLive": 12): tool lists are indexed once and cached in SQLite, and each server starts only when one of its tools is called, with at most maxLive running and the least recently used stopped first. Bare launcher names such as npx and uvx resolve on Windows too.

The end-to-end test drives serve with the official rmcp client over stdio. Tested in Claude Code 2.1.312 with examples/gateway.json: asked to investigate a failed deployment and roll it back, Claude found the cause through the gateway (deployment, commit, logs) and did not attempt the rollback because inspect_tool showed the policy denies it. Claude Desktop is untested (#1); reports welcome.

In Docker

The image serves the gateway.json in a mounted folder. Servers launched with npx or uvx run inside the container; pass their keys with -e (for ${VAR} references) or write them in the file's env. Servers started any other way (docker, paths on your machine) do not work from inside it. With no arguments the image serves the built-in demo servers.

docker build -t toollens .
docker run -i --rm -v /path/to/folder:/config:ro toollens serve --config /config/gateway.json --db /app/toollens.db

The console

./target/release/toollens console --config examples/gateway.json --noise 982
#   ToolLens console: http://127.0.0.1:7300/#token=…

A local web UI over the running gateway: browse the catalog, try discovery for any goal, replay recorded benchmark runs, and run a live agent (needs ANTHROPIC_API_KEY) to watch every discover, inspect and invoke step as it happens, next to the gateway's own audit log. The demo above is this console. It binds to 127.0.0.1, and every API call needs the token it prints.

How it works

flowchart LR
    A["Agent<br/>Claude Code · Claude Desktop · any MCP client"] -- "3 tools" --> G
    subgraph G["ToolLens"]
        D["discover_tools<br/>BM25 · embeddings · dependency graph"]
        I["inspect_tool<br/>schema · provenance · policy decision"]
        V["invoke_tool<br/>policy → schema check → audit log"]
    end
    G -- "started on demand, at most maxLive" --> S["Your MCP servers<br/>110 in the benchmark, 2,468 tools"]

1. Index. Servers start six at a time with a timeout each; one that fails is reported with its last stderr lines and the rest keep working. Names, descriptions, input and output schemas and server identity are indexed. Ids are server/tool, so overlapping names (get_status exists on 13 servers in the demo) are never ambiguous.

2. Retrieve. BM25 keyword search (default), semantic search (Ollama embeddings, cached in SQLite), or a hybrid of the two (reciprocal rank fusion). At most two direct matches per tool family, so five CRUD variants of one near miss cannot fill the result.

3. Expand with dependencies.

   • Schema edges: tool A's output has a field that tool B takes as input. Producers that resolve names to ids rank first, and each shared field is weighted by how rare it is, so commit_sha is strong evidence and service_id is weak. Tools of different servers are linked only through fields used by at most 3 servers: across real vendors, region matching is coincidence, not a data flow.
   • Observed edges: B was called successfully with a value A returned earlier in the same session. Learned at runtime and persisted.

   Destructive tools are never suggested as speculative next steps, and dependencies fill at most half the result slots, so they never push out the best direct matches.

4. Execute. Policy is checked; arguments are validated against the JSON schema (failing closed if the schema itself is broken); the call is forwarded with a 30 s timeout, and a timeout or a client cancellation cancels it upstream through MCP. At most 8 calls per server are in flight.

Security model

• Discovery never grants permission. Policy is checked on every invoke and nowhere else. Deny globs win; then denyDestructive; then allow globs; then the default. With no policy, everything is denied, and a misspelled policy key fails the config instead of being ignored.
• Server annotations may only tighten policy. denyDestructive denies tools a server marks destructiveHint: true unless an allow glob names them.
• Upstream servers never see the gateway's secrets. They get system basics plus their own configured env, never the gateway's environment (API keys).
• Nothing sensitive reaches the model or the audit log. Provenance shows a server's program name, never its arguments (which often carry credentials). The SQLite audit log records argument names, never values.
• The console is not an open door. Binding to 127.0.0.1 is not enough on its own: Docker containers and DNS-rebinding pages can reach it. Every API call needs the per-start token.

Found a hole? See SECURITY.md.

Known limitations

• denyDestructive acts only on tools explicitly annotated destructiveHint: true. Tools with no annotations are not treated as destructive (the MCP spec's default would be); deny them by glob.
• Lazy mode trusts the cached tool list until --reindex. Pin server versions (pkg@1.2.3, image digests) so an upstream update cannot change a tool behind the gateway's back.
• maxLive is not a hard bound: a server evicted while one of its calls runs keeps running until that call ends, and a new call can start a second copy.
• Concurrent first calls to a server that fails to start wait in turn; there is no backoff.
• Benchmarks: one model, one run per task, tasks written alongside the system.

Benchmarks

Two benchmarks, both on Claude Opus 5.5 at effort medium, one run per task. Every tool call goes through the same gateway policy and validation, and metrics are read back from the gateway's audit log.

Real world: 110 vendor MCP servers, 2,468 tools

The catalog.

• Every server is a published npm or PyPI package whose registry metadata links to the vendor's own GitHub organization, or that sits under the vendor's own npm scope. Examples: AWS Labs (51 servers), GitHub, GitLab, Slack, Notion, Stripe, PayPal, Twilio, DigitalOcean, Heroku, CircleCI, MongoDB, ClickHouse, dbt, Playwright, Context7.
• 137 packages were verified; 112 started inside the sandbox, and 2 of those expose no tools without an account, which leaves 110. The 25 that did not start are listed in bench/real/index-report.json: dependency rot in the published package, a required CLI or OAuth login, a startup credential check that rejects the dummy key, or a crash.
• Each server runs in its own Docker container (bench/real/sandbox.Dockerfile) with dummy credentials. Tools are real; calls to keyed services fail authentication, so no account is touched.

The tasks. 28 (bench/real/tasks.json): 6 answer tasks run end to end on servers that need no key (time, fetch, git, filesystem, AWS documentation); 22 selection tasks pass when the agent calls a correct tool with schema-valid arguments (the call then fails on the dummy key). Two leave the vendor open: "geocode the Eiffel Tower" accepts any of 4 geocoders, and "search the web" any of 7 search tools.

The three arms. Full catalog: every tool registered with the API. ToolLens: the three tools. Native tool search: Anthropic's server-side BM25 tool search, every tool sent with defer_loading.

What this says:

• Exposing every tool is not an option. Every request is rejected with prompt is too long: 1429159 tokens > 1000000 maximum. Real tools are heavy: about 580 tokens each, against about 90 for synthetic ones. Separately, 15 real tools cannot be registered with the API at all: their schemas use property names such as $vectorSearch or EndTime< (Twilio, MongoDB, Contentful, Postman), and one such tool makes the API reject the whole request. ToolLens serves them like any other tool.
• ToolLens and native tool search tie on completion (26 and 25). ToolLens read 28% fewer input tokens and cost half as much; native search used half the model requests (it searches and calls in one turn). Native search also sends all 2,454 deferred definitions, about 6 MB of JSON, with every request; on this connection that dominated its wall time. On a datacenter link the time gap would be much smaller; the payload would not.
• The failures were mostly task design and hit both arms. Notion's own server marks its search tool destructive, so the conservative policy denied "find a Notion page"; MongoDB's server needs a connect before count. The grading was written before the runs and not changed.

Offline retrieval on the same catalog (top 8, the raw task text as the goal):

Dependencies neither help nor hurt here (most real tasks are single-step), and embeddings (nomic-embed-text) trail plain keyword search on real descriptions. Agents still do better than these numbers because they reformulate and call discovery again.

Controlled: simulated servers plus generated distractors

Three simulated servers plus generated tools spread across 10 servers with realistic names, including deliberate near misses (CI build logs, pod logs, deploy windows). The same 8 tasks run with the full catalog exposed and with ToolLens.

• Completion tied. Opus 5.5 picked no wrong tools even with 1,000 synthetic tools exposed: at this scale the full catalog costs context and money, not accuracy.
• At 1,000 tools ToolLens is cheaper on every count: 96× fewer tool tokens per request, 20× fewer input tokens per task, 13× lower cost.
• ToolLens is not free at small sizes. At 30 tools it cost 19% more. The crossover is between 30 and 100 tools.
• Discovery is separate from authorization. With ToolLens the agent saw the deny through inspect_tool and never tried the rollback; with the full catalog it tried once per size and the gateway blocked it.
• Dependency expansion pays off when servers share ids by design, as one organization's internal suite would: recall at 1,000 tools rises from 0.65 (keyword) to 0.84 (keyword + dependencies).

Caveats for both: one run per task, one model, tasks written alongside the system, selection tasks prove the right tool was chosen rather than that the service did the work, and observed-edge learning was off. This benchmark found real bugs before the final numbers; all are in BUGS.md.

Reproduce

./target/release/toollens bench retrieval --config examples/gateway.json --tasks bench/tasks.json
./target/release/toollens bench agent --config examples/gateway.json --tasks bench/tasks.json   # ~$7.60, needs ANTHROPIC_API_KEY

--runs N repeats every task N times. Runs already on disk are kept, so an interrupted run resumes, and agent.json then reports each arm's min, mean and max across runs. An episode whose server could not start (Docker stopped, say) is recorded as an error, not a failed task, and runs again next time; a run stops after its first episode if that one could not reach its server.

The real-world benchmark needs Docker; its exact steps are in CLAUDE.md ("Real-world catalog"). The full three-arm run cost about $4 at list price. The demo video is made by demo/record.mjs: it runs the tasks live, then renders frame by frame on a virtual clock at 3840×2160 and 60 fps (demo/STORYBOARD.md).

Project layout

src/        lens.rs (engine), gateway.rs (MCP face), meta.rs (the three-tool contract),
            keyword.rs + semantic.rs (retrieval), graph.rs + discover.rs (dependency graph),
            policy.rs, validate.rs, upstream.rs (server pool), store.rs (SQLite), console/, bench/
tests/      end to end over real MCP stdio
examples/   gateway.json
bench/      tasks, the real-world catalog and sandbox, committed results
demo/       STORYBOARD.md, record.mjs

Contributing

Issues and pull requests are welcome; CONTRIBUTING.md has the commands CI runs. Every bug found so far, with its reproduction and fix, is in BUGS.md.

If ToolLens saves your agent some context, a ⭐ helps other people find it.

License

MIT