The chief executive of cloud data management company Rubrik is taking issue with the latest version of the Gartner Magic Quadrant for Data Center Backup and Recovery Solutions, citing inaccuracies and potential bias that seems to have stemmed in part from changes in the Gartner analyst team.
Sinha wasn’t alone in his criticism. Oussama El-Hilali, chief technology officer for Arcserve, told CRN USA that, despite a heavy investment in following Gartner guidelines for growing its business, there was no positive movement in Arcserve's positioning on the Magic Quadrant.
Gartner publishes its Magic Quadrant for a variety of tech industries. In the Magic Quadrant, or MQ, Gartner reviews how competitors are faring against each other, how they are executing on their stated visions, and how well they are performing against Gartner's market views.
Gartner says the vendors are placed in one of four quadrants: "Leaders" who execute well against their visions and are well positioned for the future, "Visionaries" who have a good vision of where the market is going but have not yet executed on that vision well, "Niche Players" who focus on a small market segment or who do not have outstanding innovation or performance, and "Challengers" who dominate a market challenge but may not demonstrate it knows the market's directions.
For its part, Gartner referred CRN USA to a new blog post posted Friday from its ombudsman and written by Nancy Erskine, Gartner's group vice president of ethics and integrity.
Erskine wrote that the Gartner Magic for Data Center Backup and Recovery Solutions was significantly delayed following a complaint from Rubrik that claimed its evaluation was inaccurate and that one of the report's authors unsuccessfully applied for a job at Rubrik a year before that analyst was assigned to the Magic Quadrant.
Gartner investigated the preparation of the report and its underlying analysis, and brought in a senior analyst from another team to do a deep review.
"He found nothing improper in the data, research process, analysis, the analyst’s interactions with Rubrik, the advice provided by this analyst to more than 100 end-user clients regarding this vendor, or the conclusions presented in the draft Magic Quadrant. … We believe Gartner’s opinion on vendor capabilities in this market is accurately expressed in the Magic Quadrant, a rigorous, independent analysis that helps buyers navigate technology purchase decisions," Erskine wrote.
The 2019 Gartner Magic Quadrant for Data Center Backup and Recovery Solutions has yet to be made public. The last version, published 31 July 2017 and updated 1 January 2019, was written by Gartner analysts Dave Russell, Pushan Rinnen, and Robert Rhame.
Russell is currently vice president of enterprise strategy at Veeam Software, but at one point had accepted an offer to work at Rubrik. Rinnen is currently director of product strategy for Rubrik. Rhame is director of EMEA product marketing at Rubrik. At least one other former Gartner analyst, Ray Schafer, is director of competitive intelligence at Rubrik.
In his blog post, Sinha noted that the 2018 Magic Quadrant for Data Center Backup and Recovery Solutions was not published, and that Rubrik worked for several months to remedy a number of what he termed significant issues and concerns.
"So we felt it was important that the market, including our customers, potential customers, partners and employees, have the full set of facts that are pertinent in objectively evaluating the information contained in this MQ," he wrote.
Between February 2017 and February 2019, Rubrik saw dramatic progress with a 500-percent growth in bookings and employee count, making its business comparable to that of other leading vendors like Commvault and Veeam, and during that time added "tech titans" including former Microsoft Chairman John Thompson, former Symantec CEO Enrique Salem, and Cisco Chairman Emeritus John Chambers as board members and advisors, Sinha said.
Despite Rubrik's efforts to work with Gartner analysts, Gartner made several errors in its analysis, and even confused Rubrik with a smaller competitor in one case, Sinha wrote. Gartner at one point felt Rubrik might be too young to be in the Leader quadrant, but Sinha wrote the vendor gave examples of multiple companies across several Magic Quadrants that were younger and smaller than Rubrik and yet made the Leader quadrant.
Sinha also wrote that the Gartner analyst who did the backup and recovery Magic Quadrant report may have had a bias against the company, noting that five Gartner analysts in the backup and recovery part of its business separately left the analyst firm and approached Rubrik for employment. Rubrik made offers to four of those analysts.
"What happened to Analyst #5? After his sustained pursuit of a position with Rubrik, we declined to offer Analyst #5 a role, and he expressed his clear disappointment in light of his colleagues’ hirings at Rubrik. Approximately four months after communicating this displeasure, Analyst #5 would change teams and be named as a lead analyst on the MQ covering Rubrik. We expected that this conflict of interest would have been disclosed, as Gartner’s policy requires, and it was not," he wrote.
El-Hilali told CRN USA that his company, Arcserve, had a similar experience to that of Rubrik.
"We thought it would be good for partners, customers, and the tech community at large to be aware this didn't happen only to Rubrik," he said.
Arcserve worked closely with Gartner on developing a plan for growth, at one point directing about 30 percent of its research and development resources to follow the analyst firm's guidelines to reflect Arcserve's efforts towards growth, and saw positive results from the changes it implemented, El-Hilali said.
"When we wanted to correlate the changes to the MQ process, we had multiple meetings with Gartner to discuss the salient points," he said. "But we found the process wasn't followed. They told us we were one of a couple companies who showed double-digit growth, but we didn’t see changes in the MQ. The results put into question their entire process."
Arcserve identified about 40 inaccuracies in Gartner's analysis of the company, including errors in the descriptions of some of Arcserve's functionalities and errors in the growth numbers of its partner ecosystem, El-Hilali said.
Just as important, Arcserve displayed all its roadmaps and projections to the Gartner analysts involved in the Backup and Recovery Solutions Magic Quadrant just before many of them departed the analyst firm, El-Hilali said.
"We didn't see any protections for that data," he said. "There were NDAs (non-disclosure agreements) in place. But if an analyst under NDA goes to one of our competitors, I would think it would be hard to imagine they didn't bring some level of comparison to their new employers."
Several solution providers told CRN USA that clients look to Gartner's Magic Quadrants for guidance, but typically do not depend on them as a major part of their decision-making process.