split-inference
Shard ONE model across MULTIPLE machines' GPUs with llama.cpp RPC — build with -DGGML_RPC=ON, start rpc-server on the backend hosts, launch the main server with --rpc, and PROVE the split is real instead of a silent local-only fallback. The reference topology is Bonsai-27B across the local 5090 and the DGX Spark over the ~1ms LAN.
Pinned to revision 685376e99ae5, so it is the text this page describes rather than whatever the author pushed since.
Pre-approved tools experimental
Experimental field. Support varies between clients, so this list is what the author declared, not what your client will enforce.
- split_detect_topology
- split_resolve_recipe
- split_plan_deployment
- split_apply
- split_verify
- Read
- Bash
Files
Every link opens the file at its source, pinned to the revision this page describes.