Cerebras Systems, now a leading public company specializing in AI chips for inference, counts industry giants like OpenAI and AWS among its clientele. The company recently celebrated a highly successful initial public offering (IPO), catapulting its valuation to approximately $60 billion and elevating its co-founders to billionaire status.
However, the journey to this pinnacle was fraught with challenges. In 2019, just three years after its inception, Cerebras faced a critical juncture that nearly led to its demise. The company was expending around $8 million monthly, accumulating nearly $200 million in pursuit of a formidable technical challenge. This relentless expenditure was driven by the ambition to solve a problem that many in the semiconductor industry deemed insurmountable.
Founder and CEO Andrew Feldman recounted the arduous period, stating, We were spending about $8 million a month. At this point, we had incinerated nearly $200 million trying to solve one technical problem. Each board meeting became a daunting task, as Feldman had to report ongoing failures and escalating costs. Despite the mounting pressure, the team remained resolute, understanding that without a breakthrough, the company’s future was bleak.
Cerebras was established with a visionary concept: to revolutionize microprocessor design by creating a single, massive chip from an entire silicon wafer. Traditional microprocessor development focused on enhancing CPU performance by increasing transistor density and reducing chip size. However, the computational demands of artificial intelligence necessitated the integration of multiple chips, leading to complex communication challenges. The founders believed that a monolithic, large-scale chip could deliver superior performance by eliminating these inter-chip communication bottlenecks.
This innovative approach presented unprecedented engineering challenges. No one had previously succeeded in creating such a large and dense chip. The endeavor required orchestrating countless microscopic electronic components on a substantial yet delicate surface, introducing a myriad of technical complexities.
After successfully designing the mega chip and partnering with Taiwan Semiconductor Manufacturing Company (TSMC) for its production, Cerebras encountered a significant obstacle: packaging. This phase encompassed all post-manufacturing processes, including mounting the chip onto a motherboard, supplying power, managing thermal dissipation, and establishing data pathways. Feldman explained, We were using 40 times as much power as anybody had ever used. The chip’s unprecedented size and power consumption meant that existing solutions for heat sinks, vendors, and manufacturing partners were inadequate. The team faced a daunting task, as previous attempts by leading microprocessor engineers to build such large and dense chips had ended in failure.
The Cerebras team embarked on a rigorous process of trial and error, resulting in the destruction of numerous chips and the expenditure of substantial financial resources. Without effective packaging, the chip’s functionality was compromised. Through meticulous analysis of each setback, the team gradually addressed critical issues, such as cooling mechanisms and data transmission. In one notable instance, they developed a specialized machine capable of simultaneously securing 40 screws to attach the wafer to a board without causing damage.
Feldman vividly recalled the breakthrough moment in July 2019 when their efforts culminated in success. Upon installing the packaged chip into a computer and powering it on, the entire founding team observed in awe as the system operated flawlessly. Watching a computer run is about as exciting as watching paint dry. But there we were watching lights flashing on the computer, stunned that we’d solved this, Feldman recounted. He described it as one of the most significant moments of his life, underscoring the magnitude of their achievement. This milestone was particularly remarkable considering that the same founding team had previously built and sold the pioneering cloud server startup, SeaMicro, to AMD for $334 million in 2012.
The day their chip functioned as intended was also approximately two years after OpenAI had engaged in discussions to acquire Cerebras. Feldman confirmed that these talks occurred, as indicated by publicly disclosed emails. However, the negotiations fell through due to internal disagreements among OpenAI’s founders, some of whom were angel investors in Cerebras.
Today, OpenAI stands as both a customer and a partner of Cerebras, having extended a $1 billion loan secured by warrants. These warrants conditionally grant OpenAI approximately 33 million shares of Cerebras’ stock, as disclosed in the S-1 filing. At Friday’s closing price of $279 per share, these shares are valued at over $9 billion.
Interestingly, as part of the loan agreement, Cerebras agreed to refrain from selling its products to specific competitors of OpenAI. While Feldman did not confirm whether this restriction pertained to companies like Anthropic, he noted that the limitation is temporary. It’s limited in time, and it was designed to make sure that we could get OpenAI the capacity, he explained.
Feldman likened the process of selling AI compute capacity to an all-you-can-eat buffet. Instead of attempting to serve all potential customers simultaneously, Cerebras chose to focus on a select group, ensuring they could meet demand effectively before expanding further. We’re going to work with part of the buffet only, and we’re going to get comfortable with that, before we attack the rest, he stated.
Cerebras’ journey from the brink of collapse to becoming a $60 billion AI chip leader is a testament to the resilience and innovation of its team. Their unwavering commitment to solving complex technical challenges has not only secured their place in the industry but also paved the way for future advancements in AI hardware.
