Why personal behaviors impact testing

My last post used testing to illustrate the consequences of questionable personal behavior on a business situation.  Quality is susceptible to personal and professional gaps that interact to amplify each other’s effects.

Why is that so?  Let’s start with the examples I used.  Recall that business process owners simply copied the unit tests of the developers to serve as user acceptance tests.   I characterized this approach as a failure of accountability: the process owners didn’t believe it was their “real” job, even though they knew they would have to certify the system was fit for use.  Less charitably, one could have called it laziness.  More charitably, one could have called it efficiency. 

And indeed, an appeal to efficiency underlay the rationalizations of these owners: “Why should I create a new test when the developer — who knows the system better than I do — has already created one?”  How would you answer this question?  As a leader, do you know the snares such testing practices lay in your path?  Off the top…

  1. Perpetuating confirmation bias:  By the time someone presents work product for formal, published testing, he or she has strong incentives to conduct testing that proves that the work product is complete.  After all, who doesn’t want his work to be accepted or her beliefs confirmed?   This issue is well-known in the research field, so one should expect that even the most diligent developer will tend to select testing that confirms that belief.   An example is what on one project we called the “magic material number”, a material that was used by all supply chain testers to confirm their unit and integration tests.  And the process always worked…until we tried another part number.
  2. Misunderstanding replicability:  “Leveraging” test scripts can be made to sound like one is replicating the developer’s result.  I have had testers justify this short cut by appealing to the concept of replicability.  Replicability is an important part of the scientific process.  However, it is a part that is often misunderstood or misapplied.  In the case of copied test plans, the error is simple.  One is indeed following the process test exactly — a good thing — but applying it to the same test subject (e.g., same part, same distribution center, etc.).  This technique means that the test is only applied against what may be “a convenient subset” of the data.
  3. Impeding falsifiability: This sounds like a good thing, but isn’t.  In fact, the truth of a theory — in this case, that the process as configured and coded conforms to requirements — is determined by its “ability to withstand rigorous falsification tests” (Uncontrolled, Jim Manzi, p. 18).  Recall the problem with engaging users in certain functions?  These users’ ability to falsify tests makes their disengagement a risk to projects.  Strong process experts, especially those who are not members of the project team, are often highly motivated to “shake down” the system.  Even weaker process players can find gaps when encouraged to “do their jobs” using the new technology using whatever parts, customers, vendors, etc. they see fit.

I hope this example shows how a personal failing damages one’s professional perspective.  No one in this example was ignorant of the scientific method; in fact, several had advanced hard science or engineering degrees.  Nonetheless, disagreement who owned verifying fitness for use led to rationalizations about fundamental breaches in testing.

How personal shortcomings undermine recovery (Mini Case Part 2)

Unfortunately, our quality control processes didn’t fare so well. We did get sufficient testing resources for the first rollout, but a couple of process owners only delivered under protest. For you see, they believed that testing of their processes — even user acceptance testing (UAT) — was not their job. To put it another way, they did not hold themselves accountable to ensure that the technical solution conformed to their processes’ requirements.

This personal shortcoming — an unwillingness to be accountable — triggered a chain of events that put the program right back in a hole:

  • Because it wasn’t their “real” job, some process owners did not create their own user acceptance tests. They simply copied the tests the developers used for unit or integration testing. Therefore, UAT did not provide an independent verification of the system’s fitness for use; it simply confirmed the results of the first test.
  • This approach also allowed process gaps to persist. Missing functionality that would have been caught with test plans that ensured process coverage went unnoticed.
  • Resources for testing were provided only grudgingly and were often second-rate. They often did not know enough about system and process to run the scripts, never mind verify the solution or notice process gaps.

To say it was a challenging cutover and start-up would be an understatement.  Yawning process gaps remained open because they had never been tested.  For sure, we had a stack of deliverable acceptance documents, all formally signed off.  What we didn’t have was a process that was enabled and fit for use.  One example: 

  • Documents remained stuck in limbo for weeks after go live, all because a key approval workflow scenario had not even been developed.
  • And because it hadn’t been developed, the developers hadn’t created and executed a test script.
  • And because the process owners were so focused on doing only their “real” job, they missed a gap that made us do business by hand for nearly two months.

Can personal shortcomings undermine recovery? (Mini Case Part 1)

I concede that projects can recover — at least for a time — without sustainable personal and professional behaviors in place. Heroic measures to catch up on accumulated technical debt, more testers to ensure all tests are completed, new resources that specialize in turnarounds can and do work… again, for a time.

But what happens when the “hero” team needs to take a week or three of down time? What happens when those additional testers go back to their “real” jobs? What happens when the turnaround team leaves? What happens is that the project risks a slide back into the abyss.

Even one gap can be problematic. For example, I was on a troubled transformation program that needed to use all three of these approaches: extraordinary effort, additional testers, and experienced recovery resources. And indeed, the heroic measures did create deliverables that were fit for use , the technical debt had been repaid, and the development team was staffed up to support the remainder of the program. The turnaround specialists put a set of program governance practices in place; even better, the program office continued to execute them effectively.  Quality assurance and testing were other matters entirely….

3P/T — Personal, Professional, and Project Recovery over Time

This perspective on business recovery means we must respect the relationship among personal, professional, and project behaviors.  The three dimensions of recovery behavior take place over time and are highly interdependent: functional or project recovery cannot exist without a foundation of appropriate personal and professional behaviors.

In particular, I maintain that project, operation, or business recovery will not be effective unless one’s personal and professional recovery leads the way. In other words, one must have changed one’s personal and professional behaviors — that is, moved them from “before” to “during” recovery — before one can effectively fix a project or business.

I hope this conclusion makes intuitive sense. After all, who can effectively facilitate a contentious scope prioritization session when one has been on short sleep for three weeks? Or how does one find the right expert to untangle a technical knot when one’s network was ignored once the new job started?

Before, During, and After Trouble

Like projects themselves, recovery from project challenges follows a definite lifecycle. Todd Williams (@BackFromRed), in his Rescue the Problem Project, identifies one prerequisite and four steps in the recovery process.

  1. Realization. Before recovering a project, the project sponsor, executive management, or steering committee must realize that the project has a problem and needs new direction. After accepting that the project has problems, recovery proceeds in four steps:
  2. Audit the project
  3. Analyze the data
  4. Negotiate the solution
  5. Execute the new plan.

A few years ago I put together a project de-escalation outline on Crossderry Blog based on my experience recovering projects and consulting engagements. My approach had a slightly different twist — largely driven by its emphasis on engagement management — but there are definite parallels.

  1. Discovery: How well do you know your project?
  2. Decision: To escalate, or not to escalate?
  3. Definition: What must be done?
  4. Dialogue: How to explain?
  5. Delivery: Into action.

I’m not sure there’s much value in reconciling project recovery lifecycles; in fact, I’m afraid it will distract from my focus on personal and professional factors in project failure and recovery. Therefore, this series of posts will use a simplified lifecycle of “Before, During, and After” project trouble.

Bad Choices Underpin Most Problem Projects

In my last post I asserted that failed projects often have at their roots “neglect or damage in [the leaders'] personal and professional lives”, not ignorance of project management principles. Rather than let that question begging stand, we should look at that claim. If the claim is true, then shouldn’t we find evidence in problem projects of “good thought, bad action?”  Bad projects don’t just happen, we make them happen.

Indeed we do.  At Crossderry Blog I’ve blogged about project failure and success many times, including the work we did at SAP on project escalations. The most glaring examples are around stakeholder and communications management.

While these stakeholder and communications management techniques appear simple, we’ve found that many project managers can’t muster the motivation or confidence to execute against them consistently. These “symptoms” are great non-obvious indicators for project health:

  • Projects that consistently linked stakeholder analysis, communications planning, and plan execution stayed out of trouble.
  • Project managers who did not fit into, or could not adapt to, customer cultures, mission-critical projects, etc. were reluctant to escalate projects quickly enough.
  • The most frequent communications mistake was failure to execute planned executive-level messaging, which eroded the project manager’s position in the eyes of sponsors and other leaders.

These stakeholder management findings correspond with recent external studies noting that communication breakdowns – especially “keeping quiet” about known risks or issues – are a primary driver of project failures. To my main point, isn’t “keeping quiet” about project-threatening issues and risks a sign that there’s something amiss with one’s mind, body, spirit, or professional life?  More soon…

Personal and professional lives fail before the project does

Well, I’ll confess.  Not only has failure happened to a friend, but I’ve seen it, I’ve heard it, and I’ve lived it.  Personal, professional, and project problems grow and cascade into a torrent of trouble. In other words, project problems don’t remain confined to the project; they inundate every part of one’s life.

This challenge is what I’m going to address in this series. Many excellent writers and thinkers have laid the foundation of project management and project recovery. And indeed I’ll outline many of those standards and practices, along with my take on them.

However, my experience is that projects rarely fail because the initiative’s leaders were ignorant of tools and techniques.  They usually “knew the good,” but did not “do the good.”  This inability to translate knowledge into effective action is often driven by neglect or damage in one’s personal and professional lives.

What do I mean by personal and professional?

For now I’ll keep it simple. The personal life refers to one’s mind, body, and spirit and the balance among the three. I’ll borrow from Eastern and Western traditions to illustrate what and how the personal life gets out of balance. The professional dimension is more like a complex Interstate or Autobahn interchange. It’s where one’s career goals, relations with one’s team and bosses, and professional network come together. We’ll look at both in more depth later.

Things fall apart

Another sleepless night.  It’s 4:25 AM and you’re wide-awake. Check that. You’re awake, sort of, after another sleepless night.  Yes, another sleepless night, because you had two last week, and another two weeks before that.

As you shuffle into the day — guzzling a cup of this, wolfing a bite of that — you barely put one foot in front of the other. You’re so busy that the only time you have to yourself is in the shower. As you slump under the spray, you ask “How did it come to this?”. That moment’s reflection leads to an answer.

You can’t sleep because your life is a mess, you can’t sleep because your job is a mess, and you can’t sleep because your project is a mess. “I’ve gotta take a step back and… Arrrgh!”

A bit of soap in the eye does the trick and now you’re well and truly awake…and late!  And rushing unawares into another day.

Does this sound familiar to you? Are you living this right now ? Has it happened to a friend of yours? 

More later…

Follow

Get every new post delivered to your Inbox.

Join 2,518 other followers

%d bloggers like this: