A huge outage hit across the Internet, taking many popular sites offline including Amazon, Twitch, and Reddit. The New York Times, The Guardian, The Financial Times and The Verge were also affected. The popular CDN provider Fastly has confirmed it’s facing an outage on its status website published a blog post on this massive outage. The company wrote on Twitter that its POPs (points of presence) was disrupted by a file that has now been disabled on early June 8.
Disruption by a file
Nick Rockwell, Senior Vice President of Engineering and Infrastructure wrote the blog post titled “summary of June 8 outage”. Rockwell said,
“We detected the disruption within one minute, then identified and isolated the cause, and disabled the configuration. Within 49 minutes, 95% of our network was operating as normal.”
Fastly began a software deployment that introduced a bug that could be triggered by a specific customer configuration under specific circumstances, on May 12. “Early June 8, a customer pushed a valid configuration change that included the specific circumstances that triggered the bug, which caused 85% of our network to return errors,” according to Rockwell.
Timeline of June 8 outage
A timeline of the day’s activity (UTC):
09:47 Initial onset of global disruption
09:48 Global disruption identified by Fastly monitoring
09:58 Status post is published
10:27 Fastly Engineering identified the customer configuration
10:36 Impacted services began to recover
11:00 Majority of services recovered
12:35 Incident mitigated
12:44 Status post resolved
17:25 Bug fix deployment began
What is the Content Delivery Network (CDN)?