vstack is a formal verification toolkit for coding agents. Naive agents routinely game formal checkers by weakening invariants, dropping timeout branches, or adding unverified assumptions; vstack enforces mechanical gates across TLA+, Apalache, TLAPS, and Z3 so agents find real bugs and prove critical state machines without cheating.
vstack workflow
News
- [2026-11-17] Shared 5 SGLang bug case studies and fix PRs on X, with 40+ bugs now tracked across SGLang and vLLM in our bug tracker.
- [2026-11-13] Shared 5 vLLM bug case studies on X, with 15+ bugs tracked and growing in our bug tracker.
- [2026-11-11] Released
vstackalong with our blog post, VeriServe, and launch X thread.
Quick Start
git clone https://github.com/hao-ai-lab/vstack.git && cd vstack
uv tool install -e .
vstack setup # checks Java/TLC/Z3 and installs skills, agents and the docs they link to into Claude Code, Codex, or Cursor
Or install directly as a plugin in Claude Code via /plugin marketplace add hao-ai-lab/vstack and /plugin install vstack, or in Cursor via /add-plugin vstack.
Run /vstack-mode on a target module and safety property:
/vstack-mode verify that our RDMA KV cache lease manager never recycles an in-flight page on timeout.
Anti-Reward-Hacking Gates
| Gate | What It Enforces |
|---|---|
| Role isolation | Separate coordinator, worker, and red-team agents keep specs, modeling, and review independent. |
| Hash-pinned specs | SHA-256 locks safety properties and theorem signatures so workers cannot weaken the goal. |
| Line-anchored models | Every state variable and action maps to an exact file:line@commit across TLC, Apalache, and Z3. |
| Mutation & trace gates | Models must fail on historical bugs, pass with the fix, fail when guards are removed, and replay real traces. |
| Exposure gate | A finding is reported as caught by formal checking only if a faithful model with every bug switch off violates a named property, a red team fails to refute it, and it reproduces on a real engine next to a passing control. Shell models, experiment-only failures and calibration runs do not count. |
| Proof & runtime checks | Critical state machines scale to unbounded TLAPS proofs with preconditions mirrored as runtime checks. |
The model adequacy readout is not a gate: it reports how closely a model follows its code (trace acceptance, historical fix coverage and blind detection recall, each with its denominator and interval) and sets no thresholds. Bug hunts report a light version; full verification and regression skills report all of it.
Recommended Models
| Workflow Stage | Recommended Models | Why |
|---|---|---|
| Modeling, Proofs & Red-Team | Fable 5.2, GPT-6.1-Sol, Gemini 4 Carbon at high reasoning effort | Strongest at state-machine abstraction, inductive invariants, and catching missing guards. |
| Code Analysis & Repro Harness | Claude Opus 5.5, Gemini 4 Carbon | Fast and reliable at navigating large Python/CUDA codebases and building multi-GPU reproducers. |
Case Studies
- AI Infra Bug Hunting: Concurrency, lifecycle, and numerical bugs uncovered across SGLang (X case studies) and vLLM (X case studies), tracked in Verified AI Infra Bugs and synced from
bugs/verified-bugs.csvwithPrior Statusseparating net-new (Unknown) discoveries from partly tracked (Partly Known) and independently caught unfixed upstream bugs (Known). - Community Verification: Reproduce an
In Progresscandidate on hardware, validate an open fix PR, or contribute a new formally caught infra bug via PR tobugs/to be credited underHuman Verifier. - VeriServe: A Prefill-Decode serving engine built with
vstack, proved across 226 TLAPS modules with +0.18% average p50 TPOT overhead on NVIDIA GB200.
Contributors
Acknowledgments
Skill orchestration draws inspiration from Lauren Tan's pstack. Trace validation and proof gates build on lessons from Specula, SysMoBench, and TLAPS-Bench. We thank the TLA+, Z3, SGLang, and vLLM communities.
Citation
@misc{vstack2026,
title = {Towards Verifiable {LLM} Infra},
author = {Zhang, Yuxuan and Zhang, Minshen and Su, Junda and Sun, Yi and Chen, Junda and Zhang, Hao},
year = {2026},
month = nov,
howpublished = {Hao AI Lab Blog},
url = {https://haoailab.com/blogs/verifiable-llm-infra/}
}