bonsai2-27b
Serve PrismML's Bonsai 2 27B (Ternary-Bonsai-2-27B, PTQ1_0 5.9 GB or PQ2_0 7.2 GB) as an OpenAI-compatible endpoint with the PrismML llama.cpp fork. Use on 'run Bonsai 2 locally', 'self-host a 27B on a GPU or CPU box', 'why does Bonsai 2 output gibberish'. Stock llama.cpp cannot serve it.
Pinned to revision 685376e99ae5, so it is the text this page describes rather than whatever the author pushed since.
Files
Every link opens the file at its source, pinned to the revision this page describes.