After nearly a full day of fielding complaints from disgruntled users and developers, Facebook revealed a global outage was caused by a server configuration change.
Facebook tweeted before 12:30 p.m. ET to inform users and developers that the issue had been resolved, capping off a nearly 24-hour period of intense scrutiny of the world's largest social networking platform.
"Yesterday, as a result of a server configuration change, many people had trouble accessing our apps and services. We've now resolved the issues and our systems are recovering. We’re very sorry for the inconvenience and appreciate everyone’s patience," Facebook said.
Yesterday, as a result of a server configuration change, many people had trouble accessing our apps and services. We've now resolved the issues and our systems are recovering. We’re very sorry for the inconvenience and appreciate everyone’s patience.— Facebook (@facebook) March 14, 2019
Facebook, as well as sister social sites Instagram and WhatsApp, started showing problems just before noon ET on Wednesday. The partial outage spanned much of the United States and extended into other parts of the world.
Almost two hours after reports emerged, the status page for the Facebook Platform, used by developers to build apps on Facebook, changed from reporting health to notifying users of a partial outage that began about an hour earlier.
Downdetector.com, which tracks in real time website outages and problems, showed a spike in reported problems starting before noon ET. Those tapered off by late evening—more than 12,000 reports of Facebook outages before midnight ET had dropped to 370 as of noon ET on Thursday.
One-third of those more reports were of a "total blackout." Another third complained of problems with the Facebook newsfeed, and the final third had trouble logging in.
Some Facebook users tweeted the site or app notified them of "required maintenance,” with a message that said it would be working again in minutes. Of those affected, many said they could not log into their accounts, or had trouble updating their personal and professional pages.
Network monitoring company ThousandEyes said it had observed "500 internal server errors" from Facebook by the end of Wednesday.
"Given the sheer scale and continuous changes that these web scale providers are constantly making to their applications and infrastructure, sometimes things break as a result of these changes, even in the most capable hands," ThousandEyes said in a blog post.