MIT formula for uncertainty: pad your estimates

I don’t have the time to delve into the entire article, so I may be misrepresenting the thrust of Spyros Makridakis, Robin M. Hogarth and Anil Gaba’s Winter 2010 article in the MIT Sloan Management Review (full article here).   Or more properly, I hope that this Forbes excerpt — Why Forecasts Fail. What To Do Instead — misrepresents the piece.  For instance, what do the authors mean here?

Say you’re a publisher and have an unknown author selling her first novel. Publishers should look at the sales track records of first-time novelists in general. The uncertainty surrounding your author shouldn’t differ from the wider population of new authors. You should be able, therefore, to estimate how low or high the sales might go. That range probably covers 95% of all possible outcomes. The next step would be to take the estimated range and increase it. 

Wow…I guess the WAG now has the MIT Sloan seal of approval.  Or perhaps it is CYA that’s now approved.  Other than allowing a forecaster to say — “I told you so” —  is there really all that much use in simply increasing the range of estimates? 

Maybe we should spend our time learning how to distinguish the outliers quickly — e.g., from distinct initial sales patterns or unusually intense media coverage — so we can take appropriate action?


Black Swan author on the limits of statistics

Turkey -- day 1002

Turkey -- day 1002

An excellent and accessible on-line article by Nassim Taleb (author of The Black Swan) about how to protect against improbable events.  It is always refreshing to see a stats expert warning of the discipline’s limits. 

A lot of his focus is on the current financial mess, which of course has at least some of its roots in the misuse of statistics.  Even worse, when everyone uses the same half-baked statistical analyses to assure themselves that there is little risk, they are unknowingly and unwittingly increasing the likelihood and magnitude of the pesky risk events. 

Taleb also has a way with the colorful metaphor:

A turkey is fed for a 1000 days—every day confirms to its statistical department that the human race cares about its welfare “with increased statistical significance”. On the 1001st day, the turkey has a surprise.

Why truly-great performers struggle as leaders

Following up on my previous post on expertise, success, and leadership (here), I’d like to explore the notion that being accomplished in one’s field is an advantage in leadership and management roles.  As I noted before, such a positive relationship is clear — up to a point. 

My belief is that being a truly elite performer — not simply a star — is actually a detriment when leading teams, groups, and organizations.  This disadvantage is driven by at least two factors:

  • Superstars have limited empathy or understanding of the challenges merely good performers face.  Whether the gap is driven by differences in motivation, dedication, or talent , I don’t know.  But there is abundant evidence of former elite athletes failing in two ways: losing patience with apparent effort or preparation shortfalls OR running a very loose ship and expecting the team to simply know what to do.
  • Elite skills are often tacit and innate.  The true superstar sees/feels the game differently than others.  Wayne Gretsky was asked how he was in the proper position so often.  His answer: “I go where the puck is going to be.”  Who taught him that?  How would one teach that skill?  If it could be taught, wouldn’t that advantage be already arbitraged away?

One of the reasons that merely excellent players can be excellent coaches as well is that they can bridge these gaps.  They are close enough to the elite to empathize with the pressures and demands of being a great performer.  Continue reading

Expertise, Success, and Leadership — Study

Bob Sutton at Work Matters (blog here) has an excellent post on this topic (here), along with comments on a provocative study on the relationship among expertise, success, and leadership.  I agree w/ Bob’s basic premise:

[L]eaders aren’t as easily fooled by hollow smart talk, and are more likely to talk in ways that help their followers succeed (rather than simply sound impressive), when they have past deep experience in the industry along with years of experience doing, managing, and succeeding at the kind of work their people do.

As part of his discussion, Bob points to a study (PDF here, Andrew Gelman’s stats blog commentary here) that purports to prove this point.  Which I guess it does to a certain extent.  It appears that coaches with NBA playing experience have more regular season and playoff success than those who don’t.  It also appears that being a better player — all-star game selection being the proxy — has a bearing on regular season records and a weaker correlation with playoff success (the ultimate measure of achievement in a professional league).

However, the study and related posts throw around “great” and “superstar” when characterizing these ex-all-stars, which stretches the point too far.  I’m not sure anything in this study contradicts the received wisdom that truly great players struggle as coaches or managers.  How many truly great players — looking at Top 50 All-Time in the NBA or Hall of Fame in major league baseball — have had any success as coaches or managers?  Continue reading

%d bloggers like this: