And why is over committing a problem?

One of the base Agile Principles is that we try to set things up so that Teams operate at a sustainable pace. When Teams hear about this there is general excitement that it might be possible, that they might be able to restore some kind of work / life balance.

Then there is the belief that “they won’t let us do that around here.” This is a rational concern in many organizations as Management often believes that being busy is a sign of being productive and useful, and that management tools like “stretch goals” and “mandatory overtime” will result in increased delivery of value.

Coaches are aware of this, and so coach management that they need to be very positive about how they want to have a sustainable pace. Management (often though coaching) understand that they have an interest in a plan based on reality to improve the predictability of work delivered. They begin to understand that to improve how much is delivered (throughput) they cannot overload the Team as this increases the amount of work-in-progress. They begin to realize that one reason the organization are annoying their customers is that we say we will do something, and then fail to meet that commitment simply because there is too much work to get done. In other words Management has a reason to be positive about the sustainable pace and can be authentic in their delivery. They are looking forward to the day when the Team makes and meets commitments more often than not.

But still it inevitably happens. All the work is now visible so we know Teams are trying to take on more than is realistically possible. We try to help the Team, but they don’t seem to be able to do anything about it.

Perhaps we leave it this time. “They’ll learn when they don’t make it.” But the next time it happens, and the next time, and the next time. But now it’s worse. The original excitement that Agile will deliver a work / life balance is gone since the Team has to work hard to meet the commitments they made.

I’ve seen this pattern happen over and over again. I once ran a survey of 65 Teams and found that the average overcommit of the Teams was 42% and that about 69% of Teams over-committed on a regular basis.

The question we have to ask ourselves is “why?” After all, no one is forcing the Team to do this, right? Management thinking is correct - if we don’t address the problem we will remain unpredictable in our delivery, and have reduced throughput due to high work in progress. Over time, we will further reduce throughput due to a reduction in engagement of the Team. Worse, our customers will see no improvement in our ability to make and meet commitments.

My view is that there are a number of factors which contribute at the individual and Team level:

• Image management: We want to look good, both as an individual and as part of a super Team. We therefore want to be seen to be doing more, to provide more value.
• Not understanding how to say “not now”: Many organizations (especially IT organizations) have a policy to say “yes” to all work. This history has not taught Teams how to say “not now” to their customers and stakeholders. Even if they know they cannot do the work.
• Optimism bias: People overestimate the likelihood of positive outcomes when they look into the future. There is benefit in having a positive outlook, but in this case we are letting this attitude adversely affect our ability to make rational decisions.
• The retrospective bet: The last retrospective was very positive and the Team determined a problem and a solution. The Team bets on this result. This means that we assume we will get better before we actually get better.
• A genuine want to help: Knowledge workers really want to help and so when someone says “I just need this …” they will take it on themselves to get it done, especially if they know the person asking.
• Hero culture: The organization has traditionally rewarded (one could even say developed) the heroes of an organization. This is so ingrained that people are not even aware they are operating this way.
• Uncontrolled work intake system: Many Teams, especially when starting, do not have control of their work intake system. This means they are unable to plan and predict the work, and are impacted more than expected by “break ins”, for example.
• Invisible work: The Team thinks it has a view of all the work, but in reality a lot of work is hidden. This hidden work usually impacts the ability of the Team to deliver on their commitments.
• Poor estimations: Early in a Team’s life the Team will not have a good understanding of what it takes to deliver value which often results in over-committing.
• Return work: When work is completed, there are often issues associated with the initial delivery, and time must be taken by the Team to address these issues. This is often not factored into the capacity of the Team.

And sometimes Management do not help the situation:

• Management talks, but does not walk, Agile: For example, there is a conversation on over-committing, but no real action. Message to Team “situation normal: just keep saying yes.” Or management says “Thank-you Brent for taking this on.” Message to Team “we will continue to reward heroes.”
• Management unwillingness to make the hard calls: Sometimes management are presented with a problem and are unable to make a decision. Even though the Team has evaluated various positions management response is “If we just did it like this … we can do both of these …”. Spoken like a person who doesn’t actually have to do the work.
• Management unwillingness to protect the Team from the consequences of “not now”: While Teams make the call, and Management agree, you will often see Management fold when their customer or stakeholder complains about the decision. Don’t get me wrong, there is often a need to change plans, but if that discussion does not include a discussion about taking work off the Team, all management is really saying is “you have to do it.”

In many ways these are all examples of short term thinking that has huge consequences in the long term.

The question is “what can we do to improve the situation?”

There are some obvious things we can do. Firstly, we can help our Teams understand that these drivers exist and that we often create problems for ourselves. Sometimes a simple discussion raising awareness will help. This is true of both the above individual and environmental factors which lead to over committing. For example:

• Many people are unaware of the optimism bias we all have. You can make people aware of this bias by having a discussion around the Prudential Ad About Thinking Positive in the Future
• You can talk about the Hero culture in the context of books like the “Phoenix Project”.
• We could talk about using “Yesterday’s Weather” approach to determining how much work to take on instead of assuming we can do more.
• A capacity buffer could be added or increased to reflect the unknowns in the work intake system. The idea here is to force the Team to commit to less “planned” items until the intake system stabilizes.
• Retrospectives could be run on the estimates, to improve the Team’s understanding of the work.
• Team Members could ensure that “everything is on the board” to reduce the impact of hidden work.
• Teams could do a value stream mapping exercise to understand percentage of work that is accurate and complete, and so work to reduce the amount of work that returns.
• Management could show awareness of their fallibility by admitting to past errors and discussing how they will address going forward.

Under-commit; Over-deliver.

We also might try a more “culture oriented” approach. For example, we could adopt a mantra like “under-commit and over-deliver”. We would set the cultural expectation that it is OK to under-commit so long as we meet the resultant expectation. Some Management will worry than this will mean that Teams will slack off. The data above shows that we do not have this problem in reality. And wouldn’t you love to be in the room when a Team reports to the customer saying “we were able to complete the committed items and, since we had spare capacity, we also delivered this high priority item you were wanting.” BTW: I have actually seen this happen.

Often these approaches have been tried and you still find a pattern of over commitment. In these situations the Team might want to set up a single subject Retrospective to discuss approaches to improve. This might be a place to discuss the issue, raise awareness of factors, and review possible approaches. I suspect that when you ask the Team, there will be other, more specific factors and potential approaches to take.

For example, one factor that we often see is that while overcommit data is available, it is also ignored. It’s just a number on the screen. One idea to make this more visceral is to hand out tokens which represent capacity and as people plan work, take the appropriate number of tokens away. Watch for an interesting discussion to develop when people run out of tokens and still have more demand (“we have to get this done!”)

And finally we could also consider other more direct and experimental approaches. For example:

• 1/2 Velocity experiment: Team sets up an experiment by saying “Why don’t we set up an experiment where we plan to half our normal velocity as an artificial limit just to see if it helps.” The idea here is that by reducing the commitment the Team will actually produce more and feel better about the result.
• 1/2 Iteration experiment: Team sets up an experiment by saying “Why don’t we set up an experiment where we plan to half an Iteration just to see if it helps.” The idea here is that it easier to plan a short term future than a long term one.

Note: These ideas can be applied at all levels of Iteration - a Team 2 week cadence, a Program quarterly cadence.

The bottom line is that over commitment is a problem that needs to be addressed if we really want to achieve a sustainable pace for the Teams, manage the expectations of our customers, ensure good work / life balance for our people increasing their engagement, and increase throughput.