Testing the Evidence

  |   Evidence-Based Management Print Friendly and PDF

Jeffrey Pfeffer and Robert Sutton had an excellent piece
in the NY Times titled “Trust the
Evidence, Not Your Instincts
” which began this way.

“Consider this hypothetical situation: You have a serious illness. Your
doctor prescribes an intrusive, painful and costly treatment. What she doesn’t
say — because she hasn’t consulted the research — is that most studies find the
treatment ineffective and fraught with negative side effects.”

That’s a provocative beginning to an article calling for bosses to “consult
the evidence.” I applaud the objective, but if you’re a boss, you have a problem
that the hypothetical physician doesn’t have. You have to figure out which
evidence to follow.

It ought to be easy. After all, over ten thousand business books are
published every year. There are also textbooks and articles of all kinds, both
popular and academic. And let’s not forget blog posts and tweets and
newsletters. What’s a boss to do?

First find a good source, and then look for evidence. Bob
Sutton is a primary source for me because I like the way he explains the
research findings he shares and the way he connects them to the world I live in.
A book he wrote with Pfeffer, Hard Facts, Dangerous Half-Truths And Total Nonsense: Profiting
From Evidence-Based Management,
and his solo book, Good Boss, Bad Boss are both excellent.

Read more than the headline. Headline writers are not
rewarded for being accurate. They are rewarded for increasing readership.
Sometimes you may have to read the research the article is based on.

Last month there were many headlines about how “mean” bosses get paid more.
Jim Morgan (another favorite source) noticed one of them with the headline:
“Life isn’t fair: Play nice, get paid less.” Jim read the article which used the
terms “jerk” and “rude” to describe well-rewarded bosses.

Then he dug into the research and discovered that the actual findings were
far more nuanced and less provocative than the article or headline implied. Read his full report here.

Be skeptical. Most of the findings that seem
counterintuitive turn out to be wrong. Reports on research should be suspect if
they’re used to sell something. Even if the research is solid, the reporting may
be selective and biased. You have to investigate to know.

If a finding is important to you, check out the methodology used in
the research.
Was it observational or survey research? Who was studied?
In what circumstances? What questions were asked? One reason I read the Evidence Soup
is for the tips on evaluating evidence.

Do your own experiments. If a research finding makes sense to you and
suggests you should do things differently, try it. Note the results.

Boss’s Bottom Line

It’s a good idea to base your practice and decisions on the evidence, as long
as the evidence is good.

Additional Resource

After the NY Times article appeared, Bob Sutton posted “Our New York Times Piece on Evidence-Based Management: The Uncut
” on his blog.

Leave a Comment

What People Are Saying

There are no comments yet, why not be the first to leave a comment?