Circuit Breaker Labs Aims to Shield Kids from AI Harms

AI interactions are already having dangerous psychological consequences for minors, and a new startup, Circuit Breaker Labs, is stepping in to prevent worse harm. Co-founded by siblings Shirali and Arul Nigam, the company is developing systems that test AI models for vulnerabilities—especially those that could impact children, non-native speakers, or anyone speaking with slang or non-standard speech. They simulate real users from diverse backgrounds to spot when these models misinterpret and respond dangerously. Their goal: safer support in apps meant for emotional help, coaching, or companionship.

Tragedies that sparked an idea

The startup’s approach is rooted in real-world risks. One of the founders, Shirali, points to a 14-year-old named Sewell Setzer who formed an emotional connection with a chatbot on an AI platform. In a lawsuit filed in 2024, the bot was accused of failing to grasp phrases like “I want to be with you,” resulting in a tragic outcome. Circuit Breaker Labs was founded to build AI safety tools that can better detect nuance and context—especially to prevent what the company calls “AI psychosis,” where someone loses touch with reality because of interactions with a bot. Arul emphasizes that even when users aren’t trying to break the system, ordinary conversation can produce harm if the AI doesn’t understand slang, cultural cues, or non-standard grammar.

How Circuit Breaker Labs tests AI models

The core of their work is an AI safety-testing platform filled with what they describe as crash-test dummies—simulated users covering a wide spectrum of ages, cultures, languages, and speech patterns. These simulations are created in partnership with human domain experts and generate tens to hundreds of thousands of realistic interactions daily—slang, typos, non-native speech, and all. Then, using red-team style testing, Circuit Breaker Labs looks for places where a model fails, scores the missteps using an internal, auditable system, and provides detailed feedback on how to improve safety.

Right now, Circuit Breaker Labs focuses on “high-risk” AI domains—apps for journaling, mental health support, or coaching—places where emotional stakes are high and user vulnerability is built-in. The team is tiny—just five people including the founders. While it already has a working product, they say they are still early-stage and have not disclosed who their customers are. But they believe the same testing platform could one day apply to any app where users may form harmful attachments to agents—such as AI coworkers or companions whose behavior shifts unpredictably across conversations.

Despite growing public and regulatory suspicion about AI, the founders believe banning tools over safety concerns would be counterproductive. Instead, they argue the future depends on building trust through rigorous safety work. They want users to feel confident engaging with AI—without fear that it will misunderstand them and cause real harm.

What to watch: As Circuit Breaker Labs prepares to demo its technology at the upcoming Startup Battlefield competition (October 13–15, in San Francisco), the methods it refines could influence how safety testing becomes a norm in AI experiences, especially those targeted at vulnerable users.