vertex-doctor

Prototype. Personal hobby project. Not a Google product, not supported, not affiliated with anything official. It makes real API calls against your project (tiny ones, with cost gates) and can mutate project settings only when you pass --yes. Read the output before you trust it; file issues with the JSON records when it is wrong.

vertex-doctor answers one question about Google Cloud Agent Platform (the product formerly branded Vertex AI; the API host is still aiplatform.googleapis.com):

Which publisher models can this credential + this project call, in which location, through which invocation surface, and for every cell that fails, what is the one action that unblocks it?

Model access is the intersection of seven independent gates (credential binding, API enablement and quota project, IAM, org policy, per-publisher agreements, per-location per-model quota, model lifecycle), each failing with its own status code and message shape. The console, gcloud ai model-garden and gateway health checks do not compose them. This tool probes a matrix of projects x credentials x models x locations, classifies every result into a fixed taxonomy, and prints the fix once per cause.

Start here: one model, one location

vertex-doctor journey -m claude-fable-5-1@default -l global --rpm 400        # show the path
vertex-doctor journey -m claude-fable-5-1@default -l global --rpm 400 --fix  # walk it (asks before each change)

The journey is the same ten steps every time, in the order the gates are actually hit. Each step is [ok], [next] (the one thing to do now, and who does it), [later], or [you] (a human step the tool will not take: linking billing, a quota request the automated adjuster refuses).

The happy path (all ten green, then the curl and SDK call that work)

--rpm N is the only number you need: token quotas are sized from what sibling models already have in the project. Without --rpm the tool recommends a value the same way.

What can surprise you, and what the tool does about it

• Changes take a minute or two to propagate (an enabled API, the data-sharing flag, a quota grant). After every change the tool re-probes until the answer changes instead of reporting the stale one.
• A brand-new project fails on five things before the first real gate (Resource Manager, Billing, Agent Platform, Cloud Quotas and procurement APIs, then billing itself). The tool enables the APIs; billing is yours.
• The catalog is not access: models listed there can be uncallable, and callable models can be missing from it. Only the real request counts, so every step 5 to 9 is proven by one.
• claude-x@default is only valid when the catalog versionId is default; older models want the dated version. The tool re-probes the right spelling.
• Newer Claude lineages are served at global (and us/eu), not in most regions, and their quota bucket is the lineage (anthropic-claude-fable), not the model id. The tool re-targets the request and names the bucket.
• New projects have zero quota for partner models. The automated adjuster grants some requests in a minute and denies others just as fast without saying why, and it phrases a refusal at least two ways ("Quota request denied" / "We cannot grant ... '0' was granted"), so the tool decides on the granted numbers, never the sentence. When it denies, re-filing does not help; the tool says so and hands you the form instead of looping.

Probing a whole matrix instead (models x locations)

$ vertex-doctor probe -m claude-sonnet-4-6@default,claude-sonnet-4-5@default,gemini-flash-latest -l global,us-east5

my-gcp-project  anthropic  [adc]
model                       global      us-east5
claude-sonnet-4-6@default   ok 2292ms   ok 1345ms
claude-sonnet-4-5@default   x A17 404   x A17 404

my-gcp-project  google  [adc]
model                 global      us-east5
gemini-flash-latest   ok 523ms    x A16 404

What to do
1. [A17] WRONG_SURFACE_OR_SPELLING (high)  2 cell(s)
   version suffix '@default' is not valid for this model; the catalog versionId is '20250929'
   run: vertex-doctor probe --project my-gcp-project --model anthropic/claude-sonnet-4-5@20250929 --location global
2. [A16] MODEL_NOT_SERVED_IN_LOCATION (high)  1 cell(s)
   google/gemini-flash-latest is not served in us-east5 but works at global
   run: vertex-doctor probe --project my-gcp-project --model google/gemini-flash-latest --location global

Install

Python 3.11+. No published package yet; run from the repo or from git with uv:

uvx --from git+https://github.com/zeroasterisk/vertex-doctor vertex-doctor --help
# or
uv tool install git+https://github.com/zeroasterisk/vertex-doctor
# or, from a checkout
uv sync && uv run vertex-doctor

Quick start

vertex-doctor                       # interactive wizard on a TTY (help when piped); the journey is the first option
vertex-doctor journey -m MODEL -l global --rpm 60 [--fix]   # one model: the ordered path to callable
vertex-doctor doctor                # many models x locations: env -> probe -> diagnose -> plan fixes (dry-run) -> report
vertex-doctor doctor --fix          # also re-probe corrected cells; mutations ask on a TTY (or need --yes)
vertex-doctor env                   # zero-cost environment check: gcloud/CBA, ADC, APIs, IAM
vertex-doctor discover -P anthropic # catalog with the exact spelling for each surface
vertex-doctor probe -m anthropic,google -l default --save probe.jsonl
vertex-doctor quota-plan -m claude-fable-5-1@default -l us-east5 --rpm 100 --input-tpm 40000000 --output-tpm 1000000
vertex-doctor client-config --from probe.jsonl                    # curl + how to call it; --format list for SDKs, harnesses, gateways, frameworks

The doctor loop

doctor is the "walk through everything" command. It runs the environment check, fixes blockers it is allowed to fix (API enablement, ADC quota project), probes the matrix, turns every failing cell into a Fix of one of four kinds, applies the allowed ones, re-probes the affected cells and prints before -> after:

Without --fix every mutation is dry-run and printed with its HTTP request and gcloud equivalent, so an agent can show the plan and a human can approve it.

Project, credential and locations are all configurable; the wizard sniffs what the machine already has (env vars, gcloud config, the ADC file, client configs and previous runs in the current directory) and leads with a concrete recommendation such as "use my-gcp-project (gcloud config)" or "global + us-east5 ($CLOUDSDK_COMPUTE_REGION)". The flags use the same detection:

Credential specs: auto, adc, gcloud, sa:/path/key.json, impersonate:SA_EMAIL, token:RAW. impersonate: is the one to use when the question is "what can the gateway's service account call?" from a developer laptop.

Cost gates

Probing sends one tiny fixed prompt ("Reply with the single word: pong", max_tokens 512, thinking disabled where the surface allows it) per cell. Three gates:

• --no-cost: only free calls (countTokens, catalog, config reads). Proves auth, API enablement and IAM. Gemini countTokens also validates model + location; the Anthropic count-tokens endpoint accepts any model name, so it proves less. No free surface exists for the OpenAI-compatible MaaS endpoint; those cells are skipped.
• --low-cost (default): real predictions on non-premium models. Premium patterns (opus, image/video/audio/TTS/OCR) are skipped.
• --any-cost: everything.

Plus --max-cells (200), --concurrency (4), --max-total-output-tokens (20k) and --dry-run, which prints every request with the token redacted and sends nothing.

Output for agents

--json / --jsonl emit one schema-versioned record per cell (see src/vertex_doctor/schema.py). Diagnostics go to stderr. Color is off when stdout is not a TTY or NO_COLOR is set. Exit codes:

Commands

Every mutation has an idempotent pre-check, --dry-run that prints the exact HTTP request, a gcloud/curl equivalent, and the IAM it needs.

Diagnosis taxonomy

Matching is on HTTP status + gRPC status + google.rpc.ErrorInfo.reason + message fragment + corroborating reads (catalog get, fetchPublisherModelConfig, modelGardenEula:check, QuotaInfo, sibling cells in the same run). Precedence: auth > API/quota project > IAM/org policy > publisher agreement > location > quota > existence > response shape.

Annotations that ride along independently: CATALOG_FLAG_MISMATCH (callable but not in the catalog, or vice versa: listing is not access), PUBLIC_PREVIEW, TRUNCATED_AT_DEFAULT_BUDGET, RATE_LIMITED_TRANSIENT.

Model ID spellings

All of these normalize to the same (publisher, model_id); discover prints the right one for each surface:

Surfaces: Anthropic rawPredict (anthropic_version: vertex-2023-10-16), Gemini generateContent (thinkingBudget: 0, dropped on 400), embeddings predict, Mistral rawPredict with a Mistral-native body, and endpoints/openapi/chat/completions for the rest. Host is aiplatform.googleapis.com for global, {location}-aiplatform.googleapis.com otherwise.

Things the probe taught us (and now checks for)

• @default is not universal. The Anthropic REST path takes name@versionId from the catalog. @default works only for models whose catalog versionId is default; older ones want the dated version (claude-sonnet-4-5@20250929). The wrong suffix is a 404 that looks exactly like a retired model; A17 tells them apart from the catalog.
• CBA tokens have a server-side tell. A certificate-bound gcloud auth print-access-token token fails tokeninfo with invalid_token and fails aiplatform with 401 reason=ACCESS_TOKEN_TYPE_UNSUPPORTED, while ADC works. env shows both.
• Listing is not access, in both directions. openai, meta, deepseek-ai and qwen MaaS models are absent from the un-projected catalog yet callable; some catalog entries are not callable by a given project; a retired dated Claude version was still callable while absent from the catalog. Only a real request is ground truth.
• 429 messages name the quota metric; QuotaInfo ids are CamelCase. The message says online_prediction_input_tokens_per_minute_per_base_model; the id you file against is OnlinePredictionInputTokensPerMinutePerRegionPerBaseModel. quota-plan resolves one from the other (and prefers the per-minute quota when a per-day one shares the metric).
• Newest Claude lineages have no regional quota rows at all. QuotaInfo for this project lists anthropic-claude-fable, -sonnet, -opus buckets only under the Global... and Us/Eu... multi-region quotas; regional rows exist only for older per-version buckets. A regional 429 for those models is "quota is zero here", not "you are burning it". The lineage fallback in request-quota maps anthropic-claude-fable-5-1 to anthropic-claude-fable.
• Data sharing is per model generation. In the same project, older Claude models answered while the newest returned PUBLISHER_DATA_SHARING_REQUIRED. fetchPublisherModelConfig returns 404 until something is set, so absence of a config does not predict the gate.
• The OpenAI-compatible endpoint wraps errors in a list ([{"error": {...}}]). Handled.
• A brand-new project fails five times before the first real gate. On a project created minutes earlier the doctor found, in order: cloudresourcemanager, cloudbilling, aiplatform, cloudquotas and the procurement API disabled (it enabled them), then BILLING_DISABLED on every cell. Billing is a money decision, so that one is never taken automatically: the doctor prints the console links and stops. Once billing was linked, Gemini, embeddings and Grok answered immediately while Claude returned 429 with an unset global quota row: partner models start at zero and the project has to ask for the bundle (QUOTA_ZERO_FOR_PROJECT).
• Some catalog entries want another API entirely (... only supported in the Interactions API and cannot be called directly via generateContent). Classified A17, not treated as a tool bug.

Where will you use the model? (clients)

The core never names a client. Once a cell is callable, client-config (or the last question of the wizard) renders the client block through an emitter. --format list prints the registry:

Each emitter only drives the surfaces its target documents (an SDK for Claude does not pretend to call Grok); cells it cannot drive are listed on stderr. Every emitter was written against the target's primary docs; the module docstring cites the URL and marks anything unverified. Adding one is a ~40-line module in src/vertex_doctor/emitters/ that registers itself; the selection above is the top of each category by adoption and documented Vertex partner-model support, not an endorsement.

Quotas: what to ask for

A real quota request is a bundle for exactly one project, one base_model bucket and one endpoint type, not one number. quota-plan reads QuotaInfo and answers the four questions people get wrong:

1. Which family? Anthropic / Mistral run on online_prediction_* quotas; Gemini and the OpenAI-compatible MaaS partners (grok, gpt-oss, ...) on generate_content_*; embeddings on embed_content_* (regional request quota under online_prediction). A model always needs the requests/min quota plus either input + output tokens/min (current lineages) or one combined tokens/min (older ones).
2. Which endpoint type? Prefix "" (regional, dimensioned on region + base_model), global_, us_multi_region_, eu_multi_region_. Each is a different quota with a different CamelCase id (OnlinePredictionInputTokensPerMinutePerRegionPerBaseModel vs GlobalOnlinePredictionInputTokensPerMinutePerBaseModel).
3. Which bucket? The base_model dimension is not the model id: claude-fable-5-1@default is metered as anthropic-claude-fable, mistral-small-2503 as mistralai-mistral-small-2503, gemini-embedding-001 as gemini-embedding, grok-4.1-fast-reasoning as itself. The planner walks the lineage (-5-1 -> -5 -> family) until QuotaInfo has a row.
4. Does the row exist where you are asking? For the newest Claude lineages there are no regional rows at all; a regional request cannot be granted. The plan re-targets to the endpoint that has rows (global / us / eu) and says so on the form.

That form is the same shape support / account-team request templates use; request-quota --yes files the same three values as QuotaPreferences, waits up to 90 s for the automated adjuster, and reports granted / denied / pending per quota. With no targets, quota-plan recommends values: the median of what sibling buckets of the same publisher already have at that endpoint in this project (a number that was granted before is the most defensible ask), or a starter tier (60 QPM, 2M input, 200k output tokens/min) when there is nothing to compare with; request-quota --recommended files those. Existing requests for the bucket are listed with their state, and a denial comes with the three usual reasons (not yet entitled to the model, bucket not self-serve, ask too large). Other per-base-model quota varieties exist (responses_* for the Responses API, bidi/live concurrency, image/audio/video generation and per-resolution input quotas, long-running predictions for video); they are listed by QuotaInfo and out of scope for the planner today.

Seed data

src/vertex_doctor/ids.py carries a small table per publisher: surface, default locations, a fallback model list for publishers the catalog does not list, id style (bare vs publisher-prefixed), quota base_model prefix. It is a starting point the catalog and the probe correct, not a serving guarantee. gemini-flash-latest is a good evergreen alias for Gemini and is global-only.

For agents

Use --json/--jsonl, read diagnosis.code, diagnosis.remediation.command and the exit code. Never run enable-* / request-quota without --dry-run first. See skills/vertex-doctor/SKILL.md and AGENTS.md.

The repo is packaged the way google/skills packages its google-cloud-developer plugin (plugin.json, .claude-plugin/, .codex-plugin/, gemini-extension.json, skills/<name>/SKILL.md with a category and a "Use when / Don't use for" description), and the skill defers to that catalog's google-cloud-recipe-auth, gcloud, agent-platform-inference and agent-platform-troubleshooting skills for what they own:

npx skills add google/skills          # the official companions
claude plugin marketplace add zeroasterisk/vertex-doctor && claude plugin install vertex-doctor@vertex-doctor

Demos and recordings

The GIFs above come from demo/*.tape (rendered with vhs) and the walkthrough at zeroasterisk.github.io/vertex-doctor from demo/record.py, which drives the real CLI in a pty, answers its prompts, redacts identifiers and refuses to export if one survives. Re-record with:

for t in journey journey-ok quota-plan probe doctor; do VD_DEMO_PROJECT=... vhs demo/$t.tape; done   # -> docs/gifs/
uv run python demo/record.py all                                          # -> docs/casts/

Both make the same tiny real requests the tool always makes, against VD_DEMO_PROJECT, and replace every real=replacement pair listed in demo/redact.local (gitignored; see demo/redact.local.example) before anything is written. The recorder refuses to export a cast in which a real identifier survived.

Development

uv sync
uv run pytest            # offline: fixtures under tests/fixtures are real, scrubbed responses
uv run ruff check src tests && uv run ruff format src tests

Adding a rule: put the scrubbed error body in tests/fixtures/errors/, add a branch to classify() in diagnose.py in precedence order, add a test in tests/test_diagnose.py, add the row to the table above.

Not goals

Deploying self-hosted Model Garden models, benchmarking, Provisioned Throughput, replacing the gateway's own runtime health checks (the emitters configure them instead), being a general gcloud wrapper.

License

Apache-2.0. Contributions welcome but expectations should be low: this is a hobby prototype.