pooled-inference
Run a model no single box can hold, across every tier you own. A 284B sparse MoE at Q4 is ~155 GB. No consumer box holds that. But a desktop with 128 GB DDR5 + a 32 GB GPU, plus a 128 GB unified-memory box, holds it three times over — if you can spread it. llama.cpp's RPC backend does exactly that, and it works.
Pinned to revision 685376e99ae5, so it is the text this page describes rather than whatever the author pushed since.
Files
Every link opens the file at its source, pinned to the revision this page describes.