A configuration error in Cloudflare’s backbone network caused an outage for Internet properties and Cloudflare services that lasted 27 minutes. The outage caused a 50% drop in traffic across Cloud its network. According to the company’s blog post, because of the architecture of Cloudflare’s backbone, this outage did not affect the entire Cloudflare network and was localized to certain geographies.
A backbone to control Internet requests and traffic
San Jose, Dallas, Seattle, Los Angeles, Chicago, Washington, DC, Richmond, Newark, Atlanta, London, Amsterdam, Frankfurt, Paris, Stockholm, Moscow, St. Petersburg, São Paulo, Curitiba, and Porto Alegre are the affected locations.
Cloudflare operates a backbone between many of our data centers around the world. With the backbone, they have far greater control over where and how to route Internet requests and traffic than the public Internet provides.
In addition to this, the company made two main changes for this outage:
- Introduce a maximum-prefix limit on our backbone BGP sessions – this would have shut down the backbone in Atlanta, but our network is built to function properly without a backbone. This change will be deployed on Monday, July 20.
- Change the BGP local-preference for local server routes. This change will prevent a single location from attracting other locations’ traffic in a similar manner. This change has been deployed following the incident.
