Transparency in analyst analysis/ratings

Doug Henschen highlights (here) one of the mysteries of the analyst business — just how are those ratings calculated?  To Doug’s eye, Gartner’s 2009 BI platform Magic Quadrant’s rating of the combined SAP Business Objects portfolio does nothing more than split

the difference between its 2008 positions for SAP and Business Objects, as if SAP dragged Business Objects down from its front-runner status. In its “cautions” for SAP, Gartner cited… problems [that] were all evident (and cited by Gartner) last year, so I’m not quite sure what to make of the new position.

I share Doug’s hang-up with these types of rating systems — there’s not enough transparency about the “whys and hows.”  I also share his increased admiration of

Forrester’s Wave Reports, which include scoring grids that detail how the vendors fared and compared on as many as a dozen different attributes.  I know it kind of takes the “magic” out of the rating methodology, but in an age when visibility, auditability and good governance are valued, it just seems like a better approach.