On December 2, I received a text from a friend, “ChatGPT is down. I don’t think I’ll get anything done today.” After responding with a laughing emoji, I opened ChatGPT myself, and indeed, it was down. You may also recall recent AWS outages, which were short-lived but disrupted large portions of the internet. It’s a puzzling phenomenon: Networks are designed with multiple backup systems and built-in redundancies to prevent failure, yet outages happen. So, what keeps causing them? Between 2015 and 2025, the largest technology firms in the world have suffered failures that have affected billions of users, businesses and emergency services. Although there could be many reasons, one common thread is that, in many instances, good engineers were carrying out what they considered well-known, safe activities when the latent nature of complex systems allowed a minor error to turn into an international affair.
As the internet continues to expand exponentially, the potential surface area for failures grows accordingly. The concepts of cloud consolidation, centralized identity services, automated deployments and tightly coupled control planes have brought about new points of failure, which are usually hidden behind abstractions to provide ease of operations. The blast radius of such systems, when they fail, is very wide even beyond one service or organization.
After spending the last 18 years troubleshooting and designing networks that theoretically shouldn’t fail, I decided to document the root causes behind major outages from the past decade. These incidents span multiple companies and technology sectors, from Border Gateway Protocol (BGP) route leaks and DNS breakdowns to collapses of cloud control planes and mishaps in AI-era infrastructure. The aim is to identify the trends that consistently reappear and the engineering lessons we can learn from them. For anyone running production infrastructure, these incidents are not distant warning stories, but they are part of daily operations and a preview of what can happen when relevant safeguards are deficient.
BGP and Internet Routing Incidents
To level set, BGP is the fundamental routing protocol that powers the internet. Every time you access a service such as Instagram or check your bank account, your request travels through multiple interconnected internet routers. These routers communicate with each other via BGP to learn where Instagram’s servers or your bank’s infrastructure are located and how to efficiently route traffic to those locations. In short ‘no BGP — no internet’.
Microsoft 365 Services Outage (January 2023): On January 25, 2023, Microsoft users worldwide lost access to various cloud services such as Azure cloud, Teams, email and SharePoint. The event started when Microsoft introduced an erroneous internal WAN routing change during routine maintenance, which disrupted traffic flow to Microsoft data centers. The downtime across services lasted about 90 minutes and Microsoft reverted the network change to get the service back up and running. This incident showed that even internal network changes at hyperscale providers can have global consequences.
Verizon Route Leak (June 2019): A small ISP (Allegheny Technologies) leaked thousands of BGP routes and because of how Verizon’s route filtering was set up, the traffic around the internet was sent astray in a chaotic manner. This led to the services of Cloudflare, Amazon and Facebook go down and caused extreme slowdowns. The incident demonstrated that the routing changes in one ISP can spread across the world in minutes.
CenturyLink/Level 3 Outage (August 2020): CenturyLink (Level 3), a global ISP, experienced a major outage on August 30, 2020, with a BGP-related hiccup that lasted close to five hours. One internal route update triggered a control plane failure and caused widespread outages across North America, impacting hundreds of downstream networks globally. With a significant backbone ISP impaired, hundreds of peer networks experienced packet loss, demonstrating how interconnected the internet is and how a fault in one network’s configuration can spread worldwide.
Rogers Canada Blackout (July 2022): A routine system update to a maintenance system at Rogers Communications (the largest ISP in Canada) on July 8, 2022, removed a vital routing filter. This resulted in Rogers’ core IP routers becoming overloaded, disrupting wireless, wireline and emergency 911 services for more than 12 million customers in Canada. It was later reported by the government that the lack of redundancy in the network design of Rogers made the outage even worse. This event showed that a single configuration mistake within the network of a major provider in the country was enough to cause nationwide service disruption.
Operational Pattern: BGP is the language the internet speaks. It works on trust at global scale, and trust failures spread just as quickly.
Cloud Service and Data Center Outages
Google Global Outage (December 2020): On December 14, 2020, a Google outage occurred that caused all Google services to go offline, including Gmail, YouTube, Google Drive and Google Calendar, for about 45 minutes. The outage was triggered by an internal bug: Google’s identity management system exhausted its storage quota, which caused the authentication system responsible for logging users in to stop working. In effect, an error occurred within Google’s login and account APIs, which did not allow access to not only Google applications but also to third-party apps using Google OAuth. The effect of this brief downturn was disproportionate, accentuated by the fact that a single point of failure (a design choice of internal storage) was hidden and might affect billions of users.
Microsoft Azure AD Outage (September 2020): In late September 2020, Microsoft experienced a worldwide authentication failure, which left users unable to access numerous Microsoft services (Office 365, Teams, Azure), along with third-party services that use the Azure Active Directory. The error that users were getting was of internal faults with the Azure AD identity service, which Microsoft later acknowledged as a root cause. The several-hour incident was a demonstration of the complexity of cloud applications that proved that the failure of a centralized authentication service can cause the crash of a whole ecosystem of applications depending on it.
AWS Multi-Region Outage (December 2021): In December 2021, AWS was hit with a major outage centered on the us-east-1 region that lasted several hours, disrupting services such as Spotify, DoorDash and Venmo. Amazon attributed the event to diverse subsystem errors caused by an automation tangle that resulted in elevated load on parts of its infrastructure. The downturn also hampered the access of certain services by AWS to users within its cloud solutions. This incident demonstrated that even the best cloud infrastructures may be brought down by errors in automated processes, and it led to demands of multi-cloud backup and enhanced fail-safes.
Trend: When centralized identity and automation systems fail, large blast radii occur.
DNS and CDN Infrastructure Disruptions
Dyn DNS DDoS (October 2016): In October 2016, a DDoS attack targeted Dyn — a large DNS provider at the time — which later became one of the largest DDoS attacks in the history. The attack itself was in three overpowering waves, whereby Dyn’s DNS servers were flooded and rendered in disarray. This made several websites owned by numerous users that used Dyn as their DNS provider unusable, including Twitter, Spotify, Netflix and Reddit. The Dyn attack, which was caused by the Mirai botnet, highlighted the importance of DNS to the availability of the internet and prompted efforts to secure DNS-based infrastructures against DDoS attacks.
Cloudflare Outages (2019 and 2024): Cloudflare, a company that provides DNS resolving and a CDN over a significant fraction of the internet, has suffered a number of high-profile outages. Cloudflare experienced a major system-wide outage on July 2, 2019, after a misconfigured BGP routing policy caused widespread traffic misdirection. This glitch brought down websites that relied on Cloudflare’s CDN and also impacted DNS lookups. More recently, in September 2024, Cloudflare was again affected by another significant service disruption that lasted approximately two hours and affected several services (e.g., users could not access applications such as Zoom and HubSpot). Cloudflare was mentioned to be remarkably fast in responding as well as open in reporting on the incident in both instances. However, the incidents underscored the extent to which much of the internet infrastructure is concentrated in the hands of a few infrastructure operators.
Fastly CDN Outage (June 2021): A large-scale outage in a content delivery network may affect vast areas of the web, and that is what happened when Fastly experienced a major outage on June 8, 2021. A routine, valid configuration change made by a customer triggered a latent bug in Fastly’s software platform. As a result, 85% of Fastly-hosted network resources began returning errors, causing various high-profile websites to go down. The faulty update was discovered promptly, and the engineers quickly reverted the update and the service was restored within a few hours. The Fastly incident encouraged multiple organizations to reconsider their dependency on single CDNs and consider multi-CDN redundancy to clasp similar risks.
Operational Pattern: Internet-scale services increasingly rely on a small number of shared infrastructure providers.
Major Platform and Application Outages
Facebook/Meta Global Outage (October 2021): On October 4, 2021, Facebook (since renamed Meta) and its family of apps experienced an unprecedented global outage, disabling Facebook, Instagram and WhatsApp within approximately 5–6 hours. The fault impacted about 3.5 billion users, which is virtually all the worldwide users of such services. It followed a backbone network configuration change performed during routine network maintenance at Facebook, and it inadvertently removed all the BGP routing to the DNS servers of Facebook. Practically, the very network of Facebook became isolated with the internet. To make matters worse, the internal Facebook tools and authentication system were also unavailable. Being one of the biggest outages in the history of any social media company, it showed the vulnerability of centralized network architectures and the need to test configuration changes comprehensively.
Slack Outage (January 2021): On January 4, 2021, the initial working day of the new year Slack suffered a massive outage, putting users worldwide without the ability to send messages or connect for approximately 2.5 hours. The disturbance affected companies in the U.S., Europe and Asia. Slack later cited scaling and networking capacity issues within its AWS-hosted infrastructure that links the Slack services that could not support a traffic burst when people come back to work after holidays. This event showed that the cloud-based collaboration systems could be overwhelmed by capacity, or even by networking capacity and the necessity to plan capacity and use graceful degradation methods to respond to spikes in activity.
OpenAI ChatGPT Outage (December 2024): As people became committed to AI services, they become prone to failures as well. On December 11, 2024, ChatGPT (along with a new generative video service named Sora) of OpenAI crashed, rendering incomplete page loads and HTTP 403 errors. According to OpenAI’s postmortem, the outage was caused by the deployment of a new telemetry service that unintentionally overwhelmed the Kubernetes control plane, which led to cascading back-end failures across critical systems. The incident had a significant impact on many users due to the rapid expansion of OpenAI. The downturn highlights that the modification of the infrastructure of the applications of even the latest AI services must be well-balanced to prevent the disconnection of key systems.
CrowdStrike Falcon Outage (July 2024): On July 19, 2024, a routine update to CrowdStrike’s Falcon endpoint security platform caused widespread system failures across millions of Windows devices. Affected users were unable to use their machines due to repeated system failures and boot loops. The impact was immediate and far-reaching, disrupting airlines, hospitals, banks, retailers, government agencies and emergency services internationally. CrowdStrike paused the update and provided remediation guidance, but recovery was complicated because remediation required manual intervention on many systems, which extended downtime to multiple hours.
How Can We Prevent Future Outages?
Although we can never eliminate all risks, there are several strategies that can be followed to reduce the frequency and impact of outages. These include not only engineering best practices but also disciplined operational processes. Some of the most important measures are outlined below.
Powerful DDoS Protection
To reduce the effects of DDoS attacks, organizations should utilize globally distributed networks and specialized DDoS mitigation services to absorb attack traffic. Internet-exposed services, particularly DNS and APIs, must be built with sufficient bandwidth headroom and automated filtering. Always-on traffic scrubbing and redundant DNS architectures help ensure that no single point of failure can be overwhelmed. Regular drills that simulate DDoS scenarios can also help incident-response teams react quickly and effectively when real attacks occur.
Strict Change Management
Since many outages stem from configuration or code changes gone wrong, it is critical to have rigorous processes around change management. These include peer review, automated testing and staged rollouts. Network policy updates such as BGP announcements or firewall rules, should be tested in sandbox environments or deployed with a limited scope before global rollout. Several large outages might have been avoided by systems capable of detecting large-scale route withdrawals during dry-run validation. Practices such as configuration as code and continuous integration allow tooling to lint and simulate changes before they are applied to production networks.
Redundancy and Multi-Provider Strategies
Organizations that depend on a single cloud provider or CDN are particularly vulnerable to provider-specific failures. Resilient architectures often involve multi-cloud or hybrid deployments so that the failure of one provider does not halt production entirely. Critical DNS and CDN services should have automated failover paths rather than manual recovery procedures. The Fastly outage in 2021 demonstrated that redundancy without tested automation is often ineffective. At the physical layer, telecom operators continue to invest in alternate fiber routes and are increasingly evaluating low-earth-orbit satellite connectivity as backup paths when terrestrial infrastructure fails.
BGP Security Measures
To address route leaks and BGP hijacks, the networking community has increasingly adopted RPKI and improved filtering policies. Network operators should deploy RPKI validation so routers can cryptographically verify whether a BGP route announcement is authorized by the IP prefix owner. This can prevent the acceptance of hijacked or leaked routes that have caused major outages in the past. In April 2020, Rostelecom, a Russian telecom operator, inadvertently propagated thousands of BGP prefixes; networks that had implemented RPKI validation were largely unaffected, while others experienced disruption. Route monitoring and alerting services that observe global BGP tables can also provide early warning when prefixes disappear or when unexpected announcements occur. With sufficient automation, operators can quickly withdraw or correct faulty routes before they propagate widely.
Better Software QA and Isolation
Given the complex interactions among software components in cloud systems, providers must continue to improve quality assurance and fault isolation. Techniques such as chaos engineering, where components are deliberately disrupted during testing, can expose weaknesses before they surface in production. Building stronger isolation into architectures is equally important. Service providers have been rethinking how to prevent a single subsystem, such as DNS or a control plane, from triggering cross-service meltdowns. This can include regional control-plane partitioning, stronger circuit breakers and automated rollback mechanisms. Monitoring systems should be capable of rapidly detecting anomalies, such as sudden traffic drops or error spikes, and triggering corrective action before failures cascade.
Postmortem Learning and Culture
Outages should be treated as learning opportunities. The most effective technology organizations conduct thorough root cause analyses and openly share what failed, why it failed, and how similar incidents will be prevented in the future. This culture of postmortems and knowledge sharing, often published publicly, helps improve reliability across the industry. Following the Fastly outage, CDN providers reviewed similar edge cases in their own platforms; after Facebook’s BGP incident, many organizations re-examined their backbone change-management procedures. Creating an environment where engineers can report near-misses and weaknesses without fear of blame is critical, as it allows issues to be addressed before they result in outages.
Conclusion: Failure is the Default at Internet Scale
High-profile internet outages over the past 10 years show one thing, and that is that massive systems can hardly malfunction unexpectedly. They fail along well-worn paths: Configuration changes, centralized control planes, complex automation and lack of isolation. This was not negligence or incompetence that caused such incidents. They are the logical result of running complex, tightly coupled systems at a global scale.
With identity services, network infrastructure and cloud platforms becoming increasingly consolidated, the cost of the smallest errors increases. With automation, success and failure are quickened, so that routine changes are converted into global incidents in minutes or seconds. Even tools designed to simplify operations can conceal risk until it is too late.
Complete prevention is not a reality, but sustainability is possible. The most effective defense against this is engineering discipline such as rigorous change management, redundancy, fault isolation and high routing protections. Improved observability, simulation and AI-assisted systems are able to strengthen this field, where instability is detected early enough before failures become uncontrollable.
Finally, reliability is not simply a technical issue; it is a cultural perspective. By openly investigating failures, being open and transparent about postmortems and focusing on learning over blame, organizations do not just make their systems better, but the internet itself. Each lesson that has gone viral makes it less likely that a fallacy could occur again in different regions.
The next major outage is unavoidable. The question is whether it will be a short-term, limited episode or the new world news headline. When operating on an internet-scale, we should not consider it a success to ensure that systems fail gracefully, but rather to build in a way that fails fast, recovers quickly, and lets us learn lessons so that we can create better systems.
