SolarWinds MSP told its 15,000 solution provider customers that it would yank the digital certificates for its MSP tools, revoking them in four days time, and forcing customers to “digitally re-sign” into its products as fallout from the massive hack into its software roils its way across the channel.
The MSPs who rely on SolarWinds products to run their business were told via email Wednesday night that the PSA and RMM tools used by solution providers appear to be safe, according to the email obtained by CRN. However, “as a best practice” SolarWinds is yanking existing digital certificates, and asking partner to reinstall new ones.
“Based upon our current investigation, we have found no evidence that our SolarWinds MSP products are vulnerable to the supply chain attack,” the email, signed by SolarWinds general manager John Pagliuca, said. “As a best practice, to further enhance the security of our products, we have retained third-party cybersecurity experts to assist us in these matters, guiding us in improving our processes and controls. To that end and to provide additional assurance to all of our customers, we have made the decision to digitally re-sign our products and have requested (and received) a new digital certificate, which reflects a recertification of the authenticity of SolarWinds products, both current and future.”
SolarWinds said it will begin issuing the new certificates on Thursday and will revoke all of its old certificates by Dec. 21.
“While we understand that this requires effort on your part, we believe that this is the right step to help ensure the security of our products and retain the trust you have in us,” Paglica said in the email. “Please know that we are doing our very best to minimize the impact to your business and to help ensure the protection of you and your customers.”
Pagliuca is expected to be named CEO of SolarWinds MSP, if the unit is successfully spun off from its parent company, SolarWinds. The decision to spin off its MSP tool unit -- possibly into a publicly traded company -- was brought up in August. SolarWinds MSP director of communications Kim Cecchini has not responded to several requests for comment via email, text, phone, and LinkedIn about the company’s MSP software.
Longtime SolarWinds MSP customer Richard Delany, chief technology officer at Delany Computer Services in New York and New Jersey, told CRN that vendors have a responsibility to deal honestly with the MSP customers whose businesses and customers rely on the security of SolarWinds products to run their businesses.
“What it comes down to is the need for these guys to be transparent, to treat their partners with enough respect to understand the liability that we have, and to be more vigilant with their software,” he said. “I think they‘re afraid. They’ve got liability, and they don’t know what to say, so everybody’s told to keep their mouth shut. Instead of being focused on the issue at hand, they’re worried about lawsuits.”
SolarWinds Orion platform, which is used by the part of the company that manages IT solutions for enterprise customers in private enterprise as well as government, fell prey to a targeted, nation-state attack that let loose a cascade of security breaches into sensitive government networks including the Department of Homeland Security, and U.S. Treasury. SolarWinds Orion is a network monitoring platform used by technologically-sophisticated government agencies, including the NSA.
Fortunately, SolarWinds’ Orion platform is not one that is commonly deployed by MSPs.
However, Michael Crean, president and CEO of Solutions Granted -- an MSSP in Woodbridge, Virginia -- said that doesn’t necessarily mean MSPs are safe. His company immediately ran a security screening of his customers when news of the breach happened. They found “hashes” -- or indicators of compromise -- were present in an MSP that had installed a trial version of Orion. Crean’s director of threat intelligence, Corey Clark, said they are digging into the data to see what they can learn.
“With this particular compromise, it‘s known that they put persistence in the environment, so that’s something we’re looking into to see if these files are getting regenerated, even though the antivirus is quarantining it, which would be obviously an indicator of persistence,” said Clark.
Ed Tatsch, president of ETS Networks, an Arden, N.C.-based MSP and SolarWinds channel partner, works primarily with smaller clients.
“All our customers are small businesses,” Tatsch told CRN. “They don’t know the software. We sell our services, not SolarWinds. So if there’s an issue, it’s not a SolarWinds issue. It’s our problem. Customers don’t care what tools we use.”
SolarWinds has been doing a good job of monitoring and managing clients’ environments, but clients know they can’t rely only on SolarWinds to take care of those environments, Tatsch said.
“This speaks to the need for a multi-layer approach at all times,” he said. “Not just in terms of monitoring environments, but also reacting to issues. As far as I know, SolarWinds is a good platform, if properly secured.”
Michael Strong, co-owner and chief operating officer at Blue Layer IT, a Lubbock, Texas-based MSP, said that SolarWinds has a whole set of MSP tools, but as far as he knows does not bring Orion through its MSP channel.
Strong told CRN that any time one sees news of the scope and magnitude of the state-sponsored hack through SolarWinds, there is always the chance that there may be more news to follow.
“I don’t know if there will be more SolarWinds news,” he said. “It can be problematic when a widely-used tool like Orion is used in an attack.”
Strong said his company has had clients call about the Orion issue, and that those that call are usually large enough to have their own IT team that is augmented with Blue Layer IT’s support and so they know what tools they are using.
“Smaller shops may not know the names of the tools we offer,” he said. “If a customer asks, we tell them we are not using the Orion product, and that as far as we know that is the only area with a problem.”