Gartner recently released their Magic Quandrant report for internal social software vendors. The below images are courtesy of Read Write Web. The first image is for 2010 the second is from last year. You can clearly see that this year there are a lot less vendors featured, you can read the RWW article on that to find out how the criteria has changed for this year’s report. I’m wondering if we might be putting too much weight on these types of reports. This post is more geared for folks who read/purchase these reports in the hopes of making some sort of a business decision as a result. I get that the space is still new and I feel that organizations looking to educate themselves about social software might be quick to put all their eggs in this so called “report basket.”
More in a minute, first the images.
I think these types of reports to a good job of providing an overall big picture of what’s going on in the vendor space and they help many people understand what the “market” looks like, but I’m hard pressed to see how a report like this can help a client. Recently Chess Media Group was involved in helping a large non-profit through the vendor selection process and none of the vendors we narrowed the list down to were mentioned above. We spent weeks breaking down use cases and understanding what every department needed from a vendor and how that would all work together. Only then were we able to put together a solid RFP. Basically what I’m trying to get at is these reports should not be used to make a short-list, vendor recommendations, purchase decisions, or anything else of that nature.
Keep in mind that these reports also serve as a great lead gen tool for the organizations that are creating them, after all, who wouldn’t want to be a “leader” or a “visionary” in social software right? Furthermore, when we’re in such a rapidly evolving and changing market such as internal and/or external social software, how much relevance do these reports really have? I can’t be too negative on them though because I think many of the analyst firms out there (Gartner is one of them) do great work, and heck, I read all of these reports as well.
In my opinion (and of course I can be wrong) these reports:
- Are too broad and general to have any implications to any specific client
- There is no distinction between vendors for vertical or company side, i.e. enterprise and mid size, financial sector and pharma
- The space is changing too rapidly for this to be relevant
- Several vendors themselves are having issues with these reports and their validity and method
Chris Yeh from PB Works, left the following comment on the RWW post:
“One quick note–at PBworks, we specifically requested that Gartner stop including us in these Magic Quadrant reports. When we discussed the quadrant criteria with the analysts, we discovered that some of the criteria (e.g. offering both hosted and on-premise solutions) would always penalize 100% SaaS companies like PBworks.
Given the fact that we (who had been featured in MQ several times in the past) were unaware of these criteria, we figured that potential customers might be unaware as well, and would rather our product be judged on its own rather than misjudged based on a 3rd-party report.”
Aaron Fulkerson, the CEO of Mindtouch, had the following to say:
“Consistently over the last four years MindTouch has had difficulty getting a briefing with the MQ principals. It’s always last minute and is only for 15-30 minutes. No demo is conducted. No workbench evaluation of the software. Seriously, how do you assess software without using it? In previous years, to my knowledge, they only spoke with three customers. For an analysis it’s short on the analysis. Buyers referencing this report should seriously consider this fact. As for why MindTouch was not included this year, I’m told MindTouch had a short period of time to get five customers with 5,000+ users into a briefing. Apparently we did not get all five customers scheduled in the appointed period of time with the appointed calendar openings. “
Reading over comments like this from vendors really makes one wonder whether anybody should be reading these reports at all. Are the images pretty? Yes. Can any organization look at these reports and make any type of useful, relevant, actionable decision? Probably not. Richard Hughes the Director of Product Strategy at BroadVision had this to say and I think it sums things up nicely(taken from RWW comments:)
Yes, I think they do. I have heard many, many times of buyers using them to create their shortlist. The danger is buyers using them “lazily”, simply taking the leader quadrant, and ignoring everything else. As a Gartner analyst once said to me “there are only two axes on the MQ – broadness of vision and ability to execute”. This “ability to execute” axes naturally favours big companies like Microsoft and IBM and penalizes smaller, specialist vendors who often have much better products.
Again, I don’t want to make it sound like these reports are useless but I think that when people read them they should really understand what they are reading and why. It also wouldn’t hurt these analyst firms if they did a little bit around educating readers as to how these reports should used and what value they provide.
Please do not make your decisions based off these reports, use them for more of a very high level big picture idea of what’s going on. What do you think? Am I way off the mark on this one? How would you change or improve the reports?
Agreed! The world is changing too fast for even Gartner to keep a pace. The quadrants, while a snapshot, should be seen as just that. There is so much more diversity in the ecosystem.
I commented on the RWW blog as well. As CEO of IGLOO Software, my company has been a niche vendor in the Internal Facing Magic Quadrant 2010 report for the past two years.
As a former EVP of Marketing at Open Text Corporation, I have been through this process many times before with Gartner, both when Open Text was a small company, and when they became a leader in the ECM space.
As you can guess, it is a lot harder to get on the Magic Quadrant as an emerging technology vendor. But, I guess it should be… as it is up to us to prove ourselves against the big boys.
The real question for me is – what is the report trying to accomplish. According to Gartner… this is a vendor report and not a software/technology report. I personally find it difficult to believe that you can separate the company from their technology. Customers in my experience, buy both – the technology and in the company.
I would like to see Gartner do a bit more categorization in the report. It seems to me that Gartner is comparing apples and oranges. In the report you have vendors like Microsoft, IBM and Open Text who offer older (mature, if you like), enterprise content management platforms. Lets be honest here… none of them are that innovative, web 2.0 or social. Sorry, but true. Then you have vendors who develop and market point solutions; others that do suite solutions – some who offer on-premise, cloud and a combination of both. As a customer, if you aren’t a guru in this space… how would you ever figure out which vendor is right for you.
To make a long story… short… it is, what it is – for now.
Jacob,
This year MindTouch never received a briefing. We began requesting a briefing in April through formal and informal channels. Most we got were email correspondence, survey (in XLS), and a last minute request for access to customers.
Hi Dan,
Personally I think it’s an easy way for analyst firms to make a splash in a new and relatively undefined marketing. If they can been seen as the authority on whatever is they are covering then more $ for them. I’m sure they do quite well on the lead gen as well. I find the Forrester reports to be more valuable and thorough. Gartner likes to be more controversial and get the attention, but at the end of the day to anyone that’s familiar with the space, the information is really not that useful.
It’s always funny to see the vendors/companies that they put in the same quadrant, I remember their social CRM quadrant had vendors such as Radian 6 sitting next to Jive, those companies shouldn’t even be on the same page together. I guess something can be said for the fact that they are making an “attempt” but it seems like many vendors are getting quite irritated with what is going on and the client side value is slim to none. I think Igloo is one of the few companies I haven’t spent much time with, we should fix that. Are you going to be at E2.0, drop me an email jacob[at]chessmediagroup[dot][com]
Thanks for the comment.
@Aaron,
Long time no see/chat man, you know the more vendors I talk to the more shocked I am to see how this all plays out. It’s really a bunch of BS from what most people are saying. Unfortunate to hear what happened to you guys, I know Mindtouch does some very interesting things, hope to see you guys at E2.0!