Why personal behaviors impact testing

My last post used testing to illustrate the consequences of questionable personal behavior on a business situation.  Quality is susceptible to personal and professional gaps that interact to amplify each other’s effects.

Why is that so?  Let’s start with the examples I used.  Recall that business process owners simply copied the unit tests of the developers to serve as user acceptance tests.   I characterized this approach as a failure of accountability: the process owners didn’t believe it was their “real” job, even though they knew they would have to certify the system was fit for use.  Less charitably, one could have called it laziness.  More charitably, one could have called it efficiency. 

And indeed, an appeal to efficiency underlay the rationalizations of these owners: “Why should I create a new test when the developer — who knows the system better than I do — has already created one?”  How would you answer this question?  As a leader, do you know the snares such testing practices lay in your path?  Off the top…

  1. Perpetuating confirmation bias:  By the time someone presents work product for formal, published testing, he or she has strong incentives to conduct testing that proves that the work product is complete.  After all, who doesn’t want his work to be accepted or her beliefs confirmed?   This issue is well-known in the research field, so one should expect that even the most diligent developer will tend to select testing that confirms that belief.   An example is what on one project we called the “magic material number”, a material that was used by all supply chain testers to confirm their unit and integration tests.  And the process always worked…until we tried another part number.
  2. Misunderstanding replicability:  “Leveraging” test scripts can be made to sound like one is replicating the developer’s result.  I have had testers justify this short cut by appealing to the concept of replicability.  Replicability is an important part of the scientific process.  However, it is a part that is often misunderstood or misapplied.  In the case of copied test plans, the error is simple.  One is indeed following the process test exactly — a good thing — but applying it to the same test subject (e.g., same part, same distribution center, etc.).  This technique means that the test is only applied against what may be “a convenient subset” of the data.
  3. Impeding falsifiability: This sounds like a good thing, but isn’t.  In fact, the truth of a theory — in this case, that the process as configured and coded conforms to requirements — is determined by its “ability to withstand rigorous falsification tests” (Uncontrolled, Jim Manzi, p. 18).  Recall the problem with engaging users in certain functions?  These users’ ability to falsify tests makes their disengagement a risk to projects.  Strong process experts, especially those who are not members of the project team, are often highly motivated to “shake down” the system.  Even weaker process players can find gaps when encouraged to “do their jobs” using the new technology using whatever parts, customers, vendors, etc. they see fit.

I hope this example shows how a personal failing damages one’s professional perspective.  No one in this example was ignorant of the scientific method; in fact, several had advanced hard science or engineering degrees.  Nonetheless, disagreement who owned verifying fitness for use led to rationalizations about fundamental breaches in testing.

The lasting impact of poor quality

My favorite corporate soap opera: As The Paint Peels

My favorite corporate soap opera: As The Paint Peels

This past weekend I read an article — sorry I can’t find the link, this Forbes article is pretty typical though — about how it is a shame that the past quality problems still haunt America’s car companies. In particular, the author made the case that car buyers need to get over it.   IMO, poor quality is a betrayal of trust for a product like autos, which are so integral to our way of life and psyches. I must admit that I’m one of those buyers who, in the words of Daniel Snow,

turned to Asian and European cars after the oil shocks [and] found they didn’t require much maintenance over hundreds of thousands of miles. To add insult to the injury, Honda, Toyota and others began manufacturing cars in the U.S. The sterling quality of these products proved American workers were not to blame for quality problems at GM, Ford and Chrysler.

To this day, every time I look at US cars I get cold feet. The memories of the peeling paint and balky transmission of my Monte Carlo and the clutch on my Maverick that lasted 20K miles come back. Let’s not even talk about a teacher’s Vega, my friend’s Pinto, or the leaky diesel McDonald’s GM company cars of the 1980’s. Even worse, the strategy of pawning off crap cars via fleet sales to rental car companies — rather than killing/fixing the marque — means that when I think of Chrysler I think of the embarassing Dodge Caliber I rented last year.

Like they said in the X-Files, I want to believe. But, then I remember…

Serendipity in WordPress — the “Related Posts” feature

WordPress added a feature called “Possibly Related Posts” that identifies posts that may be of interest of the readers of one’s post (explained here).  I’ve left the feature on in my blog, but I forgot about it until I saw two new blogs in my “Clicks” stats (from my Triple Constraint posts here and here)

Sit down and shut up! had an older post on the Triple Constraint (here).  It was nice to see that Extreme programming still pays homage to the ol’ Iron Triangle.  Also, I have to like any post that references Spock’s lament from The City on the Edge of Forever.

Suburban Fizz posted on projects, life, and how the Triple Constraint is part of everyday living (here).  I try not to impose PM jargon on my family, but I smiled with recognition at many of the same trade-offs that have gone through my head.

My wife — a refugee from Wall Street — does occasionally use her past experience, however.  For example, when I ask about what might be good for lunch, she’ll reply:

Well, we’re way long hot dogs…

Follow

Get every new post delivered to your Inbox.

Join 1,402 other followers

%d bloggers like this: