PrismML, a startup founded by Caltech researchers and led by professor Babak Hassibi, is pushing the boundaries of what’s possible with large language models (LLMs). It’s betting that high-performing reasoning systems don’t need to be massive—they just need to be smartly compressed to run on devices you own like PCs or high-end smartphones.
What’s New: Bonsai 2 27B
PrismML’s latest release, Bonsai 2 27B, shrinks Alibaba’s 27-billion parameter Qwen 3.8 model down to just 5.9 GB—roughly a ninth or tenth of its original memory footprint. This level of compression means you might soon get powerful reasoning AI without needing a data center or cloud access. Along with size, PrismML claims performance stays strong: Bonsai 2 hits about 98% of Qwen’s aggregate benchmark scores, a meaningful improvement from the first Bonsai (released in March), which landed at 95%. The earlier Bonsai model has been downloaded more than 11 million times; the smaller ones have added another 2.6 million downloads, showing strong interest in LLM compression.
How the Compression Works—and What’s Next
PrismML’s approach cuts down the data each model weight stores. Conventionally, weights need 16 bits; PrismML uses “ternary” weights—each weight takes one of three values (+1, –1, or 0)—dramatically reducing storage needs. That makes it feasible to pack serious models into smaller spaces, like that 5.9 GB file size for Bonsai 2.
Looking ahead, the company plans to apply this technique to models with hundreds of billions of parameters. Hassibi believes that as models grow larger, there’s more room to compress without sacrificing what really matters: intelligence. He suggests that bigger models might even approach what he calls “100%,” though admits perfect equivalence is mostly academic given benchmarks don’t always capture real-world performance, plus other software layers matter too.
Backers & Potential Impact
PrismML counts Khosla Ventures, Cerberus Capital, and Caltech among its backers, and has raised a $22.25 million seed round. Alongside Hassibi, Ion Stoica—co-founder of Databricks and director of Berkeley’s Sky Computing Lab—is advising the company. It’s also rumored PrismML is in talks with Apple, though neither side has confirmed that.
The implications are big: compressed LLMs could democratize AI access, enabling powerful agents to run offline on devices people already have. That could dramatically improve privacy (less reliance on cloud processing) and cost (less infrastructure needed). It also places emphasis on the software that wraps around LLMs—how it influences real performance.
PrismML is part of a growing field focused on making AI smaller and more efficient. Others are also working on compression tech, but PrismML claims its models lose very little in benchmarks despite drastic size cuts, marking a possible turning point in how LLMs are deployed.
Why this matters: We’re headed toward a future where AI isn’t just accessible through expensive servers and fast links, but embedded in our everyday devices. If PrismML succeeds, the trade-offs we accept now in speed, cost, and privacy could almost disappear. Key things to watch: how these compressed models perform in real tasks (not just benchmarks), how quickly larger models can be compressed with minimal degradation, and whether hardware makers like Apple or smartphone makers embrace this shift away from cloud-centric AI.
