Salesforce's North American and European customers endured a 15-hour outage over the weekend.
The trouble started early on May 18th Australian time, when the firm told customers it was experiencing a “major issue with its service” and wasattempting to resolve it as quickly as possible.
At the time the firm's status page said the incident came about after "The Salesforce Technology team blocked access to certain instances that contain customers affected by a database script deployment that inadvertently gave users broader data access than intended."
"To protect our customers, we blocked access to all instances that contain affected customers until we could block access to orgs with the inadvertent permissions," the status page said.
"As a result, customers who were not affected may also experienced service disruption."
The outage was especially harsh on past or present Pardot users, who were all-but-frozen-outof their orgs. "
Salesforce execs took to social media to apologise. Here's one from co-founder and CTO Parker Harris.
To all of our @salesforce customers, please be aware that we are experiencing a major issue with our service and apologize for the impact it is having on you. Please know that we have all hands on this issue and are resolving as quickly as possible.
— Parker Harris (@parkerharris) May 17, 2019
"It appears to be massive," one channel partner told CRN anonymously. "Many of my Salesforce connections have reached out about and have been impacted in one way or another.”
Salesforce said it is working in parallel to restore the original permissions as quickly as possible. The company did not immediately respond to a request for comment from CRN.
Salesforce users took to Twitter to complain about the outage.
Everyone without #salesforce right now. Go explore the outside. Take a deep breath and look at real clouds. It’s ok. It’ll come back up. #SalesforceDown pic.twitter.com/RjRPbrdWFp
— Collin A. Zimmerman (@collinz) May 17, 2019
Been using @salesforce for over a decade and today is the first time I've been impacted by #salesforcedown. I'm sure impressed how I went over a decade without having down time. All I can say is in @parkerharris we trust.
— Nick Lindberg (@NickersUniverse) May 17, 2019
Sadness and stress! #salesforcedown
— Julianne (@Julianne0223) May 17, 2019
Salesforce's incident report says the service disruption lasted 15 hours and eight minutes. At the time of writing, 6PM May 19th Sydney time, the company advised "We have restored administrators' access to all affected orgs and have prepared a set of instructions for admins that may need guidance on how to manually restore user permissions."
Additional reporting from Joseph Tsidulko