OVH is one of the largest hosting providers in Europe. It is also defined as the third-largest cloud computing company in the world. The company made its name with the burndown of the data centers from Strasbourg, France in March. It seems the problems do not leave the company. Today OVH services went down due to a routing configuration issue, during planned maintenance.
The reason is bad router configuration
OVH has 32 data centers on four continents with over 300.000 servers and a total of 20 Tbit/s global network capacity for providing web hosting, cloud computing services, and dedicated servers for its 1.300.000 enterprise customers. It seems the services went down, because of a bad router configuration. The company announced the maintenance on its status page before the services went down:
“We will do maintenance on our routers on VIN DC to improve our routing. Maintenance is planned for 13/Oct/21 9:00 AM to 10:30 AM (UTC+2). No impact expected, device will be isolated before the change.”
During the planned maintenance, the human error factor moved in, and due to a reconfiguration of the network on the DC to VH (US-EST) OVH had a problem on the whole backbone.
OVH founder Octave Klaba explained the reason for the problem on his Twitter account:
“In recent days, the intensity of DDoS attacks has increased significantly. We have decided to increase our DDoS processing capacity by adding new infrastructures in our DC VH (US-EST). A bad configuration of the router caused the failure of the network.”
Today, in the early hours when visitors tried to enter the OVH website, they encountered an “Error 503: Backend unavailable” error. OVH’s status page also went down during the maintenance. On the status page, it was displaying: “The connection has timed out. An error occurred during a connection to status.ovh.com.”
Services are now restored back
While the services were down, OVH clients such as videogame maker Rust, free chess server Lichess.org, encryption utility VeraCyrpt, and many more were impacted. Klaba also explained that the data center called SBG2 in Strasbourg was an older generation data center built in 2011. OVH is now working on the replacement of the old infrastructure with new technology.
The OVH services are now restored back. The company stated the services have been now restored with this announcement:
“On the morning of October 13 at 9:12 am (CET / Paris time), we carried out interventions on a router at our Vint Hill Datacentre in the United States, which caused disruptions on our entire network. These interventions were aimed at strengthening our anti-DDoS protection, attacks which have been particularly intense in recent weeks.
OVHcloud teams quickly intervened to isolate the equipment at 10:15 am.Services have been restored since this intervention. We are currently running a verification campaign with our clients to confirm the restoration of all their services. We offer our sincere apologies to all of our impacted customers and will be as transparent as possible about the causes and consequences of this incident.”