#Bonsai 8B: A 1-Bit LLM That Fits an 8-Billion-Parameter Model Into 1.15 GB

If an AI model with eight billion parameters could weigh just over one gigabyte, it could run on your phone. No cloud round-trip required. That is the promise of Bonsai 8B from PrismML.

#What it is

Bonsai 8B is an 8.2-billion-parameter language model that PrismML describes as a "true 1-bit end-to-end" LLM, with a footprint of approximately 1.15 GB. The core claim is that this model is roughly 12 to 14 times smaller than a full-precision 16-bit model of the same parameter count, while remaining competitive within the 8B-class benchmark range. It is released under the Apache 2.0 license.

The concept here is not about being the most powerful model. It is about intelligence density -- packing as much capability as possible into the smallest possible footprint. Think of it as fitting an entire AI assistant into a file smaller than most mobile apps.

#Why it matters

• Local AI becomes practical. If an 8B model only needs about 1.15 GB, running AI on phones, laptops, edge devices, and enterprise offline systems becomes realistic rather than theoretical.
• Lower latency. Without a round-trip to a cloud server, responses arrive faster. For interactive applications, this is a meaningful improvement.
• Better privacy. Sensitive data can stay on the device. No prompt leaves your hardware, which matters for healthcare, legal, financial, and enterprise use cases.
• Lower inference cost. For agent workflows that run many steps, cheaper and faster inference compounds across every iteration.
• A new optimization direction. Instead of only increasing parameter counts, Bonsai pushes toward maximizing intelligence per gigabyte -- a different axis of progress.

#How it works

Bonsai 8B uses 1-bit quantization across the entire model -- embeddings, attention, MLP, and the language modeling head all operate in a 1-bit representation. This is what PrismML means by "true 1-bit end-to-end": the compression is not applied after training as a post-processing step, but is baked into the model architecture itself.

PrismML has published speed claims that are attention-grabbing if accurate: approximately 131 tokens per second on a Mac M4 Pro, 368 tokens per second on an RTX 4090, and roughly 44 tokens per second on an iPhone 17 Pro Max. They also claim roughly 4 to 5 times better energy efficiency compared to 16-bit full-precision counterparts. The benchmark average across their test suite is reported as 70.5, with an intelligence density of 1.06 per gigabyte.

#Caveats

These numbers are claims from PrismML, not independently verified benchmarks. The whitepaper and HuggingFace collection are available for review, but until third parties reproduce the results, the performance figures should be treated as promising signals rather than confirmed facts. Bonsai 8B is not a replacement for frontier cloud models like GPT-5 or the largest Claude variants. It is a step forward in deployability and local AI, not a claim to the highest benchmark scores. The real value proposition is that a model this small can remain competitive at all within its class -- that is the breakthrough direction worth watching.

#Who it's for

Bonsai 8B is for developers, researchers, and teams working on local AI, edge deployment, on-device inference, privacy-sensitive applications, and cost-sensitive agent workflows. If you need capable AI that runs without a network connection or without sending data to a third-party cloud, this is directly relevant to your work.

Bonsai 8B is not GPT in your phone yet. But it is a signal worth following -- and one that still needs independent benchmarking to confirm.

Source: https://github.com/PrismML-Eng/Bonsai-demo/