On July 14, 2025, Cloudflare’s widely utilized 1.1.1.1 DNS resolver service experienced a significant global outage lasting 62 minutes, from 21:52 UTC to 22:54 UTC. This disruption impacted millions of users worldwide, rendering numerous websites and online services inaccessible.
Understanding the 1.1.1.1 DNS Service
Launched in April 2018, Cloudflare’s 1.1.1.1 DNS resolver is renowned for its speed and commitment to user privacy. As a recursive DNS resolver, it translates human-readable domain names into IP addresses, facilitating seamless internet navigation. Over time, it has become a preferred choice for users seeking enhanced performance and security.
The Root Cause: Internal Configuration Error
Contrary to initial speculations of a cyberattack or BGP hijack, Cloudflare identified the outage’s root cause as an internal configuration error. The sequence of events leading to the disruption is as follows:
1. June 6, 2025 – Initial Configuration Change: Cloudflare made preparatory changes for its upcoming Data Localization Suite (DLS) service. During this update, prefixes associated with the 1.1.1.1 Resolver service were inadvertently included alongside those intended for the new DLS service. This misconfiguration remained dormant, as the new DLS service was not yet active.
2. July 14, 2025 – Activation of Misconfiguration: At 21:48 UTC, a subsequent update introduced a test location to the non-production DLS service. This action triggered a global refresh of network configurations, causing the 1.1.1.1 Resolver prefixes to be withdrawn from Cloudflare’s production data centers. Consequently, the service became unreachable worldwide.
Impact on Services and Users
The misconfiguration affected several critical IP ranges, including:
– 1.1.1.0/24
– 1.0.0.0/24
– 2606:4700:4700::/48
As a result, DNS traffic over UDP, TCP, and DNS over TLS (DoT) experienced immediate drops. Interestingly, DNS-over-HTTPS (DoH) traffic remained relatively stable, as it utilizes the domain cloudflare-dns.com instead of direct IP addresses.
Coincidental BGP Hijack
During the outage investigation, Cloudflare detected that Tata Communications India (AS4755) began advertising the 1.1.1.0/24 prefix at 21:54 UTC, two minutes after the DNS traffic drop. While this appeared to be a BGP hijack, Cloudflare clarified that it was unrelated to the outage’s root cause. The hijack became visible due to Cloudflare’s withdrawal of its routes during the incident.
Cloudflare’s Response and Remediation
Upon identifying the issue, Cloudflare initiated a revert to the previous configuration at 22:20 UTC, restoring approximately 77% of normal traffic levels. Full service restoration was achieved by 22:54 UTC.
To prevent similar incidents in the future, Cloudflare announced plans to:
– Deprecate Legacy Systems: Transition away from outdated systems lacking progressive deployment methodologies.
– Implement Staged Deployments: Adopt deployment processes with health monitoring capabilities to detect and address issues promptly.
Lessons Learned and the Path Forward
This incident underscores the complexities of managing global DNS services and the critical importance of meticulous configuration management. While the outage was brief, its widespread impact highlights the need for robust deployment practices and continuous monitoring.
Cloudflare’s proactive approach to identifying the root cause and implementing preventive measures demonstrates its commitment to maintaining a reliable and secure internet infrastructure.
