Keystrokes are NOT Gold

Programmers tend to treat their keystrokes as gold. Perhaps they are driven to counting by project managers. Perhaps their typing skills are atrocious. Regardless, there is a great reluctance to discard code that is wrong. It is wrong because it doesn't work.

A programmer hates being wrong; at least in that respect, a programmer is like every other human being. But in the case of code that doesn't work, there is an almost irrational drive to make it right by adding more code, even if:

• It means abandoning the design,
• Coding practices must be abandoned,
• Complementary changes must be injected into completely irrational areas of system, or
• Changes to key interfaces must be renegotiated.

Traditional phased process (aka waterfall) tacitly accept and encourage these practices. Testing is done by someone else, usually at some point in the future; this lulls the programmer into thinking the code probably works, and it is someone else's job to validate that assumption. Bugs (the real name of code that doesn't work) maybe fixed by someone else. The chances that I, as the original programmer, might have to make it work are reduced.

But then comes Agile, upsetting this comfortable, lazy acceptance of bad code. Two practices in particular inhibit letting the bad code slide: Test-first development and pair programming.

Test-first has the programmer write the test first, then enough code to make sure it works. Then write another test. The effect of this is that code that doesn't work is immediately recognized. Since all the tests are run before check in (or at least run by automatic continuous integration tools), code that injects a bug somewhere else is detected. But what if the programmer skimps on the tests? What if the boundary conditions are ignored so that personal/team/manager schedule expectations are met?

The second practice, pair programming, makes that more difficult. Now two programmers have to conspire to ignore poor test coverage. But that focuses too much on the negative. Rather with a partner sharing the problem, the second pair of eyes helps identify the marginal cases, and helps find alternative solutions. It becomes almost a mutual quest to solve.

The New Problem of Keystrokes

Agile creates an environment where bugs are rapidly detected. So programmers can be lulled into taking bad code and continuing to change it in the hope that it will (magically) start working. In the ThoughtWorks programmer training course, one objective is to get programmers to overcome their reluctance to throw code away and start over.

The first trick to teach programmers to work in small cycles. In the class, we give a problem to the students (always working in pairs). The students are to write one or more tests and get the tests working. And they have only 15 minutes! Invariably, students start with a test that will prove everything works. When the timer goes off at 15 minutes, they are actually stunned that all the time has passed. They are nowhere near getting it to work. They ask for another 15 minutes, certain that they will finish in that time. Invariably, almost all fail again. Sometimes I give them more time, but it is only to get them to realize that their quest is fruitless. Then you finally have pity on them and show them the secret: write smaller tests. Get something working in the 15 minutes. That "resets" the 15 minute clock.

By the second week of the course, pairs are competing on how many cycles they got done in 15 minutes. They discover that in these tiny steps, they are marching to completing the work with absolute confidence that the solution works when they get there. Indeed, they are stunned when they write that ultimate test of the overall behavior, and it takes just a few lines of code to get it to work. A stark contrast to the frustration they felt when biting off too much.

Now the second trick kicks in. If you can't get the code working quickly, throw it away and start over. In these small cycles, they haven't written that much. The old code worked; the clock is ticking; rather than trying to get the bad code to work, they back up and start over. Code repositories are a huge tool in this; modern tools allow you to revert all the code in a couple of mouse clicks. Now the programmers don't treat that bad code as gold.

The top reasons to discard the code are:

• It doesn't work. So why are you spending more time creating a bug?
• If the reason seems to be a muddled design, throw the code away and re-think the design. Kent Beck said, "Code is where design meets the harsh reality of dawn." If the design is bad, the code will scream it out. Listen to the code. So why are you investing more time implementing a bad design?
• The programmers don't understand the problem yet. Getting stuck can be a sign of a lack of understanding. Step back and discuss the problem some more. So why do you think you will understand the problem better by continuing to write code?

See the trend? Continuing to write code is a waste of time. Discard and start over. It is not gold, but rather .

If you are a programmer, try this out: If the code seems to be going nowhere, throw it out and start again. Let me know how it goes...

Ideal Developer Ratios for Pods

In the last post, I postulated that an ideal team structure exists for Agile, a pod. The pod is a collocated, self-contained team able to execute the project. Let's drill down into the structure of this "ideal" Agile team a bit more, focusing on the programming skills the pod should have. These observations specifically apply to a pod using OO programming techniques (see the secret assumption of Agile) to write code that is easily modified. It is a language independent assumption, whether using Java, C++, C#, Ruby, or any of their brethren.

I will describe the team composition using the skills taxonomy I introduced earlier, the Master-Journeyman-Apprentice model. Briefly, an apprentice is the typically trained programmer; the journeyman has acquired the vocabulary of objects, including principles and design patterns; the master readily crafts object solutions to business problems.

The first rule of thumb for the team is that apprentices when paired together are not productive. Indeed, such a pair can be negatively productive, developing code that must be re-opened and reworked before delivery. (Of course, you do pair within the pod; a solo apprentice is truly a destructive force in the code base.)

So the guideline against apprentices pairing together leads to the first rule of an agile team:

Masters + Journeymen >= Apprentices

Simply put, the number of Masters and Journeymen needs to outnumber the Apprentices so that the Apprentices have a more experienced person with which to pair. That keeps everyone on the team productive. It also provides the mentoring to move the Apprentice to a Journeyman (see Building a Better Programmer). With care, the Apprentices gradually become Journeymen, providing more pairing options or the ability to absorb additional Apprentices.Master programmers provide continuous architectural and design direction to a project. They recognize the need for new components to support emerging stories, and pioneer these new components themselves to ensure a sound base exists for the rest of the team. How many Master programmers do you need? Fortunately, not that many:

Masters >= 10% (Masters + Journeymen + Apprentices)

In my experience, you can get away with as few as 10% of your developers as Master programmers; it is safer if you can work that percentage up to 20%, particularly in the early project stages. That may be the same number of Master programmers, but just keeping the team smaller (with fewer Apprentices early as recommended above.)

Pitfalls of a Team Out-of-Balance

The most common destructive mistake for an Agile team is to have too many Apprentices. Not only do they not produce in pairs, the best you can hope for is that they didn't damage the code base today. As individuals, it is an almost certainty that damage has been done. But the damage goes further than the code base; an Apprentice who hasn't positively contributed to the project is going home not feeling good about themselves. That creates an unnecessary morale problem for the individual and the team.. I don't know of any programmer who goes into work saying, "Today I am going to create some bugs." That this happens is not a failure of the programmer, but in the team creating the situation. So bring on apprentices only at the rate they can be productive to the team; otherwise, be courageous and defer the staffing, ignoring what your spreadsheets are telling you.

Even if you can tolerate 50% Apprentices in theory, you should not start a project with that mix. The early few weeks are the time in which the initial application architecture and design become visible. Apprentices can rarely contribute to these efforts, and can feel useless. There is one exception to this guideline: An Apprentice who has domain knowledge is very valuable to this initial team; providing key insights in what will need to be done in the long run. So start your projects with few, if any, Apprentices, and stage them into the project as the rate you can productively absorb them.

If you are attempting a reasonable size project without Master programmers, you are running a great risk of failure. Master programmers are the key to building the application architecture and design incrementally. Agile works by having such talent embedded in the team. Without it, significant up-front architecture and design is required to provide the road map for the Journeymen.

Many Agile projects seem to start fine, then seem to hit a wall in a couple of months. It is almost like Agile is not working. Refactoring is common, and is being cited more and more often in stand-up meetings. This is actually a sign that the design is weak, a reflection of the lack of continuous guidance a Master programmer provides. So ensure you have sufficient Master programmers on the project, particularly in the formative stages.

The flip side of the Master programmer requirement is that they may not be required in small or enhancement projects. A small project for certain short-term benefits may not reach a crisis point before it is complete. Similarly, if enhancements are being made to an existing, healthy code base, Journeymen are fully capable to following the tracks laid down by previous Master programmers. So Master programmers are not required for small enhancement projects.

You Have the Wrong Team

As a project manager, you have just staffed your project. Sorry to be the bearer of bad tidings, but you have the wrong team. If you an experienced project manager, you already know this. But you did the best you could from the resources you had available. It is the wrong team because:

• You are missing the experience you need in some key areas, and
• The team members don't necessarily work well together.

Agile practices help here, particularly the practices of collocation (and you do sit with your team, don't you?) and pair programming. If you are missing experience, collocation and pair programming lets you grow that experience by judicious pairing. Knowledge transfer is quite rapid and efficient in this environment. Collocation also dampens down the "finger pointing" that often plagues teams. ("He doesn't understand!", "She is so stubborn!", and the like.) It is much more difficult to talk behind someone's back when she is sitting in the room with you.

If team members still have difficulty working together, your being in the room provides insights to the solution. Is it a matter of tone (challenging, dismissive, personal rather than business)? Then coach on interaction style. Is it disagreement on technical matters? Ensure swift resolution before it becomes personal.In this area, Agile practices can be a disadvantage:collocation and pair programming will exacerbate negative relationships. Whereas you might be able to ignore or defer action when each team member sits in their own cubes, such deferrals are quite destructive in an Agile team with its close proximity.

Net-Negative Producing Programmers (NNPP)

Agile luminaries have coined the term NNPP to identify team members whose overall impact to the project is negative. That is, the work they do is less than the harm they do to their colleagues' productivity. An NNPP may be productive outside of an Agile environment. In a cube, their griping has no listeners. There challenging, terse tone goes unheard. But given their visibility in a collocated environment...

I recently saw on the Web a report purporting to show the impact of one bad apple on the rest of the team. A single individual typically can negatively impact other team members numbering in double digits! So indeed, one bad apple can spoil the whole bunch.

There are several ways to identify such individuals:

• Observe. Being in the room, you can see the behavior. Even more important, you can see the impact on overall team. The situation can be corrected by coaching, or ultimately, removal.
• Ask the team. The team knows who is acting incorrectly. Borrow a lesson from Survivor, and let the team vote someone off the island.
• Who gets picked last? If pairing is practiced, who is the person no one wants to pair with?

If you fail as a project manager to act quickly on identified NNPP, morale issues will quickly ensue. Your inaction, in particular, will be glaringly obvious to the team. Your leadership, dedication, and even competence will come under question.

Once more for emphasis: Failure to address an NNPP problem will rapidly destroy team productivity.

The Continuing Issue

The real fatal flaw in staffing projects, and Agile projects in particular, is not realizing that the project staffing needs change throughout the project. Projects that acquire their staff, and then stick with that staff throughout, will incur excessive costs if not fail outright. Staffing must be adjusted throughout the project.

A few years back on an eight-month Agile project, I made 10 staff changes in the first 6 weeks. The core development team had 16 different participants, but only 4 were on the project for the entire time. Rather than being a disaster for such team instability, the project set high water marks for productivity and quality that have yet to be touched.

Why does staffing need to be adjusted throughout a project? It is for several reasons beyond staff quitting and needing to be replaced:

• The needs of the project change throughout the project. DB tuning will be important at some stage, but not others. Knowledge of systems with which the project much integrate will be needed at some time. The general load of analysis, development, and testing will drift during the project.
• Team members will grow during the project, particularly on projects lasting more than six months. Exploiting this growth is part of good team management. Again from the above example, I reduced the number of developers to six toward the end of the project. They produced so much code, however, it took 17 testers to keep up. The developers skills had increased that much.
• An inbred team will cease innovating. Studies of innovation reveal a significant drop in innovation in a team that has been together for as little as six months. New blood is needed, even if only to challenge the patterns of current thinking.

As a project manager, you must constantly be alert to tweaking the team. You may raise eyebrows by constantly shifting your team composition; ignore them. When the need for an expert has passed, roll her off the project. Bring in a specialist when the need arises; exploit collocation and pair programming to harvest the knowledge and release the specialist. Perhaps that resource you really needed was not available at the start of the project. Bring them on board when they are available.

Successful team tuning requires building a strong relationship with two groups. First, be willing to share your experts with your colleagues. Hopefully, they will reward your good will with the loan of a needed expert of their own. Don't hoard resources when the need has passed; rather, release them to reduce your costs and maintain the morale of the resources. Second, build great relationships with your staffing or resource management team. Send them flowers or buy them a case of beer (or both); they will start to call you when a resource becomes available that you might use.

Application development is chaos; react or face corresponding chaos in your team.

All Programmers are NOT Created Equal

All programmers are not created equal. There. I have said it. It is our equivalent to saying, "The emperor has no clothes."

There is much evidence that, indeed, all programmers are created equal. My exhibit for this: hundreds of thousands of spreadsheets built by project managers. Okay, maybe I will count that as hundreds of thousands of exhibits. I have even contributed at least several hundred of them myself. It doesn't make it true, no matter much it makes the spreadsheet easier to use.

Okay, so it may not be true that all programmers are equal. But it is it approximately right? In my experience (many, many years) on real projects with real delivery deadlines, there is an order of magnitude difference between programmers (that is, 10x difference). Even after throwing out the incompetent programmers (who produce zero), the really good programmers are ten times better than the really mediocre programmers. So even in approximate terms, all programmers are not equal.

Ask a programmer you know about this. (You may be a programmer yourself.) Do they know a really good programmer? How good is she/he? You will likely hear of legendary feats, impossible tasks done in a weekend, feats of heroism worthy of superhero status.

So as a project manager, you should seek these resources out. The "one" on your spreadsheet can become a virtual "ten". That is contingency, risk management, and a cheap insurance policy, all rolled up into "one".

How do you find these programmers for you project? Sorry, more bad news here for the project manager: These resources don't automatically correlate to most of your company metrics.

• Some correlation with experience: Look for years programming; discount years of being an architect or any other non-programming role that doesn't involve personally writing code.
• Negative correlation with years since directly writing code for the job.
• Slight correlation with title/level: Does your organization have tiers of programmers (Dev 1, Dev 2, ...)? Higher levels are more likely to be stronger programmers. If your company has a true meritocracy (versus pure seniority system), a top-tier programmer must have been good at some point to be promoted. So there is a bit better chance that it is still true. Caveat: Negative correlation with years since directly writing code for the job.
• Negative correlation with non-programmer titles: Speciality titles are signs that (1) the skill is now gone, or (2) "Those that can't program, [fill in your favorite job title]". Be wary of architects, particularly if they come with adjectives (web-services architect). Designer titles are a little less risky, but have the same flaws. Ask the key question, "Do you write code?" Run away from any answer like, "I am past that" or "I leverage others".
• Some correlation with salary: The better programmers are probably paid more. Someone has recognized that they are worth holding onto. Most organizations are not as dumb as we think about matters of survival.

So, if all these indicators are not so good, how do I as a project manager know I am getting at least my fair share of these super-programmers? Easy. Ask the programmers. You may not recognize their legendary feats, but they know about them. You may need to prod them a bit; you may need to tolerate some geek-speak. Nod knowingly, and listen for the names. Another way to prod: Ask them, "If you were marooned on a island with the natives that are cannibals, and you had to program your way off the island, who whould you want to be there to help you?"

How many of these super programmers do you need? As many as you can get. What happens if you don't have them? Well, that is a topic for another day.

[This is part of the People thread of this blog. See The 3 P's of Agile: Process, People, and Pods for the overview.]