The Hardest Challenges of Implementing Scrum, #4: Realistic and Practical Estimation

May 30, 2014

Let’s continue examining the twists and turns on the road to Scrum perfection.

So far, in this series we have covered three difficult hurdles standing in the way of truly effective Scrum adoption:

The fourth challenge Scrum teams will invariably encounter is the difficult, sometimes contentious process of estimating backlog item sizes during sprint planning.

The canonical definition of estimation is that stories’ sizes are measured in “points”, which is a non-linear measure of effort required to complete a task and/or the complexity of an item. The total number of planned and pulled forward story points completed in a sprint is the “velocity” of the team, which is a key measure that Scrum teams use to evaluate their progress, in both adoption and in productivity. It does not have a direct correspondence to more traditional “productivity” measures, such as number of lines of code, numbers of man-hours, or number of features.

Measuring productivity by “story points” (rather than these other measurements) is essential, because it forces an emphasis on delivering meaningful output — working product and valuable enhancement — and takes the focus away from production statistics which are beloved by industrial managers but mean little to customers and users.

Why is Story Estimation by Points Important?

By decoupling estimation of items from time-based measures such as man-hours, and using historical velocity as the benchmark for future capacity, Scrum lets the team take full ownership of how they plan to use their working hours to achieve business goals. It makes time a limiting — not a defining — measure of the team’s work.

Better teams will achieve more points in the same amount of time, and teams that learn how to improve their processes and tools will find that their velocities increase over time. Moreover, because each team has to estimate its backlog independently, there is no absolute basis of comparison between different teams and different projects. This liberates the team from being held to unreasonable benchmarks, or unsustainable expectation of productivity and resilience. Instead, the team can focus on the only measure that matters: how to build more and build better everyday.

The Bad News? Estimating Correctly is Hard

Because stories are not directly related to simplistic measures such as time taken, however, it is very hard to actually estimate these stories! At first blush, the problem with estimating is that it can seem really arbitrary. How can we say that this feature is 3 points while the other is only 1 point? How granular do we get when estimating stories? 0.5 or 0.25 points?

Because human beings just aren’t that good at estimating, most teams will struggle with sizing stories, especially at the extremes of complexity. This issue also manifests in how the total effort to create the integrated whole is often significantly higher than the sum of the parts, because to link disparate pieces together, and to ensure that they are well integrated, there are often countless small, insignificant items that tend to add up and take away from the team’s main work.

Unless the work consists of a limited number of tasks (which is very unlikely in any organization sophisticated enough to aspire to adopt Scrum), there will be a lot of variability in the nature and complexity of tasks. Every new sprint, there will be items that have never been done, problems that have no precedent, solutions that have never been previously attempted. It takes a lot for even a single person to normalize such variation cognitively so that they can apply a consistent estimation rubric over time, and it is a lot harder for a team of 3-5 people to grapple with such variation and come to a consensus.

The Most Difficult Trap to Avoid

Hardest to avoid is falling into the trap of estimating by hours and developing a full estimation system around the idea that story points correspond with the amount of time spent. There is an enormous temptation for teams to adopt this after they have struggled to estimate reliably with points and been able to realize the true value of the measure. To those teams, measuring by hours — especially after they have already had sufficient experience with the type and scope of work in the project — is straightforward, natural, and non-contentious. However, it undermines the core tenet of measuring progress in agile — the team is not focused on delivering meaningful outputs (true increments or valuable products). Instead, they have reverted back to reporting on the amount of work they do.

This can only get worse later, as the team falls back into a water-scrum process where they still follow short iterative sprints, but for all intents and purposes, are driven by rigid timeline and valueless productivity measures. That way lies madness. And worse, succumbing to the drudgery of mindless face-time and crushing inhuman workloads.

How to Improve Your Story Estimation Process

This is not an unknown problem, and there are a lot of best practices that you can leverage to address the estimation process. For example, it is recommended that you limit the number of possible “sizes” so that there is less variation in estimation and subsequently, less contention over story sizes. Many teams use a very stripped-down ranger of t-shirt sizes or Fibonacci numbers to estimate stories. Using the Fibonacci sequence, which grows very quickly, is also a very effective way to get team members to realize that a) large stories are impossible to measure, and b) that it is unrealistic to expect good precision for such complex items.

Indeed, there should be a upper and lower limit for the acceptable size of stories, so that very large stories should be broken into smaller sub-stories or tasks that can be estimated more reliably, while very small stories should be avoided all together if possible.

I would urge you to check out some other resources on smart estimation process. Of all the challenges we have discussed, this is not necessarily the less intractable, but recognizing the difficulty. the temptation to give in, and why you should not is already a major, major part of getting Scrum right (eventually).

Additional Resources

If you find this topic interesting, please take a look at my other posts in the same series on the hardest challenges for successful Scrum adoption:

  1. Introduction: The 5 Hardest Challenges of Implementing Scrum
  2. Challenge 1: Developing a truly empowered, self-organizing, and self-examining team
  3. Challenge 2: Embracing rapid iteration
  4. Challenge 3: Adopting prioritization by impact
  5. Challenge 4: Applying realistic but practical estimation
  6. Challenge 5: Managing Effective Scrum meetings

Photo by: Barabara Krawcowicz

Chief Business Officer at UserTesting

Tien Anh joined UserTesting in 2015 after extensive financial and strategic experiences at OpenView, where he was an investor and advisor to a global portfolio of fast-growing enterprise SaaS companies. Until 2021, he led the Finance, IT, and Business Intelligence team as CFO of UserTesting. He currently leads initiatives for long term growth investments as Chief Business Officer at UserTesting.