Facebook Scrambles to Fix a Massive Outage

0 0
Read Time:2 Minute, 43 Second

“The disruption to network traffic had caused a cascading effect on the way to our data centers to communicate, bringing our services to halt”, told the company’s engineering team in a blog post on Monday night.

This outage began around 11:40 AM ET on Monday, bringing down all Facebook’s apps- including Instagram, and WhatsApp. It affected billions of users and many advertisers. According to the reports, the internal systems of Facebook were also affected by this outage. There was no internal connection between the employees to communication and work. Around 6 PM ET, it appears that most of the services were back online.

The outage had a seismic impact on immobilizing a suite of services that around 3 billion people rely on to communicate, do business, and consume news. Chief Executive Officer Mark Zuckerberg wrote on his Facebook page after the network had been restored,

“Sorry for the disruption today- I know how much you rely on our services to stay connected with the people you care about”.

Mark Zuckerberg

Various employees required Facebook to log in with their work accounts to access various tools like Google Docs, and Zoom, those services weren’t working as well which lead to some employees using alternative services like Apple’s Facetime, and Discord. Only the employees who were already authenticated with the non-Facebook tools such as Google Docs before the outage still had access.

The engineers from Facebook were sent to one of the main US data centers in California to restore various services since the fix couldn’t be done remotely. The matter was further complicated as few employees even lost access to the company’s buildings and conference rooms, according to the New York Times.

After the service was restored, an employee Mike Schroepfer, CTO said that the issue was, “affecting our networking backbone that connects all our data centers”. He even cautioned in the memo that was seen by “The Verge”, If you are not actively working on the recovery, please be patient and don’t rush to reload everything to prevent slowing down the bring up.

Facebook has not revealed a detailed explanation of the outage, however, according to the outside experts, there was an issue with the networking technology BGP or Border Gateway Protocol. Late Monday evening, the VP of Facebook’s infrastructure, Santosh Janardhan published a corporate blog post saying the outage was a result of a faulty configuration change and there is no evidence that user data was compromised as a result of this downtime”.

The engineering team looked into the matter and have learned that configuration changes on the backbone routers coordinate network traffic between our data centers caused issues that interrupted this communication.

It is not uncommon for Facebook’s app to face occasional glitches, however, they last for few minutes. The Downdetector that monitors all the internet problems said that the Facebook outage was the largest it had seen, with more than 1.5 million reports worldwide. It will not be wrong to say that Facebook’s loss turned into a profit for other social media platforms. 

More Resources from Escale

Happy
Happy
0 %
Sad
Sad
0 %
Excited
Excited
0 %
Sleepy
Sleepy
0 %
Angry
Angry
0 %
Surprise
Surprise
0 %
Default image
Escale Solutions
Articles: 61

Average Rating

5 Star
0%
4 Star
0%
3 Star
0%
2 Star
0%
1 Star
0%

Leave a Reply

Your email address will not be published. Required fields are marked *