I cited in my last post a case where Counting Wasted Time. I ended by giving my favorite response when someone is asking to collect data about the project:
...And how will you use that information?
Don't get me wrong. I really want a great answer to that question. I seek metrics that help understand the project. To paraphrase the NRA:
Metrics don't kill projects; people do.
But I digress. Some additional examples of useless information gathering will help you see the importance of the question, "And how will you use that information?"
Example 1: Revised Estimates
In a recent fixed price project, we dutifully re-estimated stories at the start of an iteration. We had release level estimates with which we did our original planning. The iteration estimates were supposed to set our "commitment" level for the iteration; they were supposed to set client expectations.
We over-delivered by 100%. We repeated the exercise, and over-delivered again. So on the third cycle, we asked the question, "How did we use the estimates we made?" The team, including the project manager, acknowledged that we hadn't looked at them since we collected the numbers. So we dropped them.
(BTW as I cited in yesterday's post about Counting, I have rarely found use in resizing just in case things have changed.)
Example 2: Defect Tracking Database
We haven't tracked bugs in the last three development projects I have participated in. How can this be?
One of the key Lean practices is feedback; feedback is most effective when rapidly given. This leads to a positive metric of story cycle time, a measure of the time from the start of analysis until story acceptance by the customer. So if a bug blocks a story, then the clock is running on that story until it is fixed and moves forward. Developers on a Lean-oriented team will attack bugs rather then work on development of new stories. The net results is few bugs live for more than a day. In tracking a recent project, over half the days were finished with zero open bugs.
Instead, we post the bug's existence on a red task card on our standard card wall. When it is fixed and verified, we take the card down. I would like to say that we throw the card away, but we still like to put them into a stack and admire them (for some reason). I do have to admit to a certain curiosity myself. But in the last couple of projects, I have yet to see anyone looking back at those cards!
There are great bug tracking tools out in the market. Many IT shops have standardized on one, and have process guidelines requiring their use. An Agile/Lean project shouldn't need any of them.
[In a bit of cruel fate for a vendor, I was recently asked to give a statement on a recently announced bug tracking tool that purports to solving all your tracking needs; indeed, it is the next generation of tracking tools, rendering all other obsolete. Needless to say, I commented that it was a solution in search of a problem as far as I was concerned. If you need that level of sophistication, your project is going to fail anyway.]
Bad Reasons to Collect Information
There are two reasons for collecting metrics that are, in fact, good reasons not to collect metrics:
Because I might need them: I consider this a euphemism for "I don't know." I lump into this same category, "We always do it this way" and "We have to". In the case of "I don't know", we drop the counting for an iteration or so, and see if we miss it. It is always hard to argue against a short-term experiment. In the case of "we always do it this way", start the discussion to drill down and figure out why we did it that way originally, and see if the reason still exists (especially if you have introduced Agile and/or Lean practices). This is just good development practice, but in particular is a good Lean practice. In the case of "we have to", it is time to engage the process architects and gauge if this is a battle worth fighting. I tend to relish that idea having done the process role in IBM many, many years ago, but we won't discuss my personality defects here. Regardless, the tactic is the same: Find out what it was useful for and see if the need still exists.
I need to understand how well everyone is doing. Translation: "I need to pin the blame on someone when things go wrong." Now you have hit a hot-button of mine. Blame is not a way to run a team. Blame creates fear, and fear kills motivation and productivity. And kills fun! Agile teams, especially collocated Agile teams, have many positive ways to address inhibitors to productivity. This includes many ways of addressing skill deficiencies which will always exist in software projects because of the wicked pace of change. So if someone attempts to collect information for the primary purpose of assigning blame, well, let's just say they have to get past me first.
There Are Good Metrics
Less I have led you astray (and you will use these posts to justify collecting no information), there are many good uses of collected information. I will cite my favorite Agile/Lean project metrics in a future post.
But for now, do you have examples of gathering useless information?