Skip to content

hao-ai-lab/vstack

v0.108.0MIT

Formal verification workflows for concurrent, distributed and systems software.

vstack is a formal verification toolkit for coding agents. Naive agents routinely game formal checkers by weakening invariants, dropping timeout branches, or adding unverified assumptions; vstack enforces mechanical gates across TLA+, Apalache, TLAPS, and Z3 so agents find real bugs and prove critical state machines without cheating.

vstack workflow

News

Quick Start

git clone https://github.com/hao-ai-lab/vstack.git && cd vstack
uv tool install -e .
vstack setup   # checks Java/TLC/Z3 and installs skills, agents and the docs they link to into Claude Code, Codex, or Cursor

Or install directly as a plugin in Claude Code via /plugin marketplace add hao-ai-lab/vstack and /plugin install vstack, or in Cursor via /add-plugin vstack.

Run /vstack-mode on a target module and safety property:

/vstack-mode verify that our RDMA KV cache lease manager never recycles an in-flight page on timeout.

Anti-Reward-Hacking Gates

GateWhat It Enforces
Role isolationSeparate coordinator, worker, and red-team agents keep specs, modeling, and review independent.
Hash-pinned specsSHA-256 locks safety properties and theorem signatures so workers cannot weaken the goal.
Line-anchored modelsEvery state variable and action maps to an exact file:line@commit across TLC, Apalache, and Z3.
Mutation & trace gatesModels must fail on historical bugs, pass with the fix, fail when guards are removed, and replay real traces.
Exposure gateA finding is reported as caught by formal checking only if a faithful model with every bug switch off violates a named property, a red team fails to refute it, and it reproduces on a real engine next to a passing control. Shell models, experiment-only failures and calibration runs do not count.
Proof & runtime checksCritical state machines scale to unbounded TLAPS proofs with preconditions mirrored as runtime checks.

The model adequacy readout is not a gate: it reports how closely a model follows its code (trace acceptance, historical fix coverage and blind detection recall, each with its denominator and interval) and sets no thresholds. Bug hunts report a light version; full verification and regression skills report all of it.

Recommended Models

Workflow StageRecommended ModelsWhy
Modeling, Proofs & Red-TeamFable 5.2, GPT-6.1-Sol, Gemini 4 Carbon at high reasoning effortStrongest at state-machine abstraction, inductive invariants, and catching missing guards.
Code Analysis & Repro HarnessClaude Opus 5.5, Gemini 4 CarbonFast and reliable at navigating large Python/CUDA codebases and building multi-GPU reproducers.

Case Studies

  • AI Infra Bug Hunting: Concurrency, lifecycle, and numerical bugs uncovered across SGLang (X case studies) and vLLM (X case studies), tracked in Verified AI Infra Bugs and synced from bugs/verified-bugs.csv with Prior Status separating net-new (Unknown) discoveries from partly tracked (Partly Known) and independently caught unfixed upstream bugs (Known).
  • Community Verification: Reproduce an In Progress candidate on hardware, validate an open fix PR, or contribute a new formally caught infra bug via PR to bugs/ to be credited under Human Verifier.
  • VeriServe: A Prefill-Decode serving engine built with vstack, proved across 226 TLAPS modules with +0.18% average p50 TPOT overhead on NVIDIA GB200.

Contributors

Acknowledgments

Skill orchestration draws inspiration from Lauren Tan's pstack. Trace validation and proof gates build on lessons from Specula, SysMoBench, and TLAPS-Bench. We thank the TLA+, Z3, SGLang, and vLLM communities.

Citation

@misc{vstack2026,
  title        = {Towards Verifiable {LLM} Infra},
  author       = {Zhang, Yuxuan and Zhang, Minshen and Su, Junda and Sun, Yi and Chen, Junda and Zhang, Hao},
  year         = {2026},
  month        = nov,
  howpublished = {Hao AI Lab Blog},
  url          = {https://haoailab.com/blogs/verifiable-llm-infra/}
}

License

MIT License — see LICENSE and NOTICE.md.