bonsai-27b
Run a 27B model in ~3.8 GB on a plain CPU box. [Bonsai-27B](https://huggingface.co/prism-ml/Bonsai-27B-gguf) (by PrismML) is a 27-billion- parameter model quantized to 1 bit per weight (Q1_0, ~1.1 bit/weight) — the whole thing is a 3.8 GB GGUF that runs on an ordinary CPU with 8 GB of RAM. No GPU, no cloud. This skill takes a spare machine from nothing installed to a live OpenAI-compatible endpoint serving Bonsai.
Pinned to revision 685376e99ae5, so it is the text this page describes rather than whatever the author pushed since.
Files
Every link opens the file at its source, pinned to the revision this page describes.