Waterfall to Scrum: How to Measure the Value of Agility with Sam Falco

Agile Coaches' Corner

This week, Dan and Sam are answering another fascinating listener question. This listener wrote in, “We are in an organization that has been going through an Agile transformation. Our leaders have been asking for metrics to compare Waterfall vs. Scrum. They want to know if we are delivering more with Scrum. How do we measure this?”

Navigating how to measure values when transitioning from Waterfall to Scrum can be a difficult challenge. How do you measure whether something is a successful initiative or not? What are the key differences in what metrics you should be measuring between Waterfall and Scrum? How do you measure the value that Scrum brings? What are some of the key metrics you should be measuring? Tune in to find out!

If you have a question of your own (or would like to share your thoughts on today’s episode), you can email the podcast at [e-mail] or Tweet @AgileThought using #AgileThoughtPodcast!

Key Takeaways

How do you measure the value your organization gets from Scrum vs. Waterfall?

Typically what is measured in Waterfalls is “iron triangle” stuff (i.e. “Were we on time? Did we stay on budget? Did we complete the scope?”) However, these things don’t really indicate whether or not you are actually getting value out of the effort

“Too often, what we’re actually measuring in Waterfall was: ‘Did we get all the work done in a certain amount of time and within budget?’ And that’s not what we’re interested in, in [an] Agile effort.” — Sam Falco

Find different ways to measure the value, whether it’s in actual dollars earned or cost savings or goals achieved

It is more important to measure the return on investment and the value it brought to customers rather than “Did we push it across the finish line?”

Measure based on value rather than on velocity

It’s not uncommon for those from a Waterfall world to measure how many tasks were completed rather than delivering valuable increments (which is critical with Scrum)

Regardless of whether it is Waterfall or Scrum, you might measure: generated revenue, saved spending, etc.

Evidence-based management as a method for measuring the success of an Agile effort:

Reference The Evidence-Based Management Guide from Scrum.org

It will provide you with a number of key value areas to help determine whether you are actually adding value, as well as some suggestions for key value measures for each of those areas

Listen to episode 107 of the Agile Coaches’ Corner, “Evidence-Based Management 101 with Sam Falco”

Think about the metrics you have in place now and whether or not they apply now (i.e. “Does this metric we’re using help us determine the current value for our products?”, “Does it help us determine time-to-market?”, “Does it help us determine our ability to innovate?” And if not, ask, “Do we really need it?”, “Are we measuring something that matters and drives results?”)

Measure lead time (if it shrinks dramatically, this is a good sign that this is going to work for your organization)

“Take a look at what metrics you’re measuring now. And if those were valuable in telling you whether or not your efforts were worthwhile, they might be appropriate now.” — Sam Falco

Even though your process is different, you can still measure using a lot of the same metrics — you just have to look at them in a different way

The most important things to measure are around the value (“Is the stuff we’re producing actually valuable to people?”)

What are some of the ways we can measure value?

Quality metrics (Has your quality increased?) — Agility tends to come with increases in quality because of a limit of work-in-progress and helping the teams focus

Look at trends; not single points in time

Have your escape defects gone down? Has your technical debt gone down?

Look at things like, how long does it take you to get a release ready for production? Is it taking less time?

One of the values of an agile way of working is that you build in feedback loops for looking at your own processes more frequently than Waterfall

Through this, analyze trends and track process improvement

There is no single way to measure output from Waterfall to Scrum

You have to always be aware of what the metric is actually telling you and evaluate not only your processes but whether you are using the right metrics (and whether they are telling you things that are helpful to you)

Releasing in Waterfall tends to be infrequent, painful, and fragile. Releases should be frequent and near-automatic in Scrum. Measuring the pain of releasing is an indicator of how easy or difficult it is to capture value

Measure deployment frequency (“How often do you actually deploy to production or customers and users?”)

Measure: “Are customers happy with the number of releases? Are they happy with the features they’re getting?”

Measure “mean time to first failure” — though it is an old metric in software development, it can still be valuable (i.e. “How long does it take for you to break something?” This should be getting longer and longer as your quality goes up)

Keep track of process improvements

Mentioned in this Episode:

Evidence-Based Management Guide | Scrum.org

Agile Coaches’ Corner Ep. 107: “Evidence-Based Management 101 with Sam Falco”

The Hard Thing About Hard Things: Building a Business When There Are No Easy Answers, by Ben Horowitz

Want to Learn More or Get in Touch?

Visit the website and catch up with all the episodes on AgileThought.com!

Email your thoughts or suggestions to [e-mail] or Tweet @AgileThought using #AgileThoughtPodcast!