How to Measure the ROI of BDD and TDD

July 03, 2019

Many developers feel the benefits of behavior-driven development (BDD) and test-driven development (TDD). There are fewer ambiguous requirements, late nights troubleshooting errors or deploys that have everyone on edge. Everything runs better when business and technical teams are on the same page.

The problem is that it’s challenging to quantify these benefits. Without demonstrating a return on investment (ROI), stakeholders may be hesitant to invest in processes that involve upfront costs, such as training developers, implementing new processes and having conversations with the development team. They may see it as a tax on developer time and their own time.

Let’s take a look at some strategies for quantifying the return on investment for BDD and TDD to justify these upfront and ongoing costs.

Why Measure ROI?

The best businesses make data-driven decisions. For example, an ecommerce shop may notice that visitors prefer one product over another by analyzing their Google Analytics data and move the preferred product higher in search results. These decisions are easy for stakeholders to get behind since the data makes it a solid bet.

BDD and TDD are development processes that impact the primary cost centers of a software business — developers. Businesses want to minimize these costs while maximizing quality and output. Without data supporting the improved quality and output of these processes, it can be difficult to persuade some leadership teams to adopt them.

The best way to persuade stakeholders to, at least, test BDD and TDD processes in the business is by showing them academic research. In 2008, Microsoft and IBM (Nagappan et al.) conducted a study of four teams using TDD and four teams with similar size and velocity that were not using TDD. The experiment was conducted on ‘live’ projects.

There were two key findings in the study:

TDD teams had 40% to 90% fewer defects.
TDD teams spent 15 to 33% more time writing code.

When testing BDD/TDD on your team, it’s important to quantify the net benefits — or calculate the return on investment. The ROI equation is simple: (Benefits - Investment / Investment). The greater the net benefit over the investment, the greater the return on investment.

Benefits & Costs

The first step in measuring return on investment is identifying the benefits and costs and then finding ways to quantify them. Before diving into the quantitative metrics, let’s explore the benefits and costs at a high level to understand how they could impact your software quality and development workflow.

[content_upgrade cu_id="5757"]Download our free list of research studies supporting BDD and TDD implementation. [content_upgrade_button]Click Here[/content_upgrade_button][/content_upgrade]

There are a few key BDD and TDD benefits:

Less Time: BDD processes can save time by getting everyone on the same page before writing a single line of code. At the same time, TDD processes can save time by eliminating the need to track down troublesome bugs.
Fewer Defects: BDD and TDD both cut down on software defects that reach production. In practice, this translates to fewer customer support issues and a better user experience.
Better Product: BDD and TDD keep a product laser-focused on business goals to reduce technical debt, or code that requires refactoring, which translates to more time spent developing features.

There are also some costs to consider:

Learning Curve: Developers may require BDD or TDD training that involves an upfront cost and time investment. In addition, it takes time to learn how to facilitate effective BDD conversations.
Testing Automation: It may be necessary to invest in test automation infrastructure if it’s not already in place. If there’s a large legacy code base, these costs can be prohibitive to adopt without a wider strategy in place.
More Involvement: BDD teams challenge stakeholders with in-depth questions about their requirements, which requires that they stay on their toes and participate in conversations to ensure everything is running smoothly.

Measuring the ROI

The second step is coming up with quantitative ways to measure these benefits and costs, creating a basis for comparison and making the return on investment calculations.

[content_upgrade cu_id="5757"]Don’t forget to download our free list of research studies supporting BDD and TDD implementation. [content_upgrade_button]Click Here[/content_upgrade_button][/content_upgrade]

There are three types of quantitative metrics to consider:

Number of Production Defects: The number of bugs that reach production, which is an important indicator of code quality. The goal of TDD and BDD is to reduce the number of bugs reaching production over time.
Cycle Time: The amount of time that it takes a new story to move from idea to production. While BDD may initially increase cycle time due to the learning curve, the long-term benefit comes from fewer misunderstandings of features.
Planned vs. Unplanned: The percentage of time spent on planned (e.g. feature development) versus unplanned work (e.g. troubleshooting bugs), which is a measure of project planning efficiency. BDD can help improve this efficiency.

The right metrics to analyze depend on your specific software project. For instance, a real-time chat app may be more concerned with availability than defects. You may also care more about developer happiness if you’re operating in an industry where recruiting is challenging. The only prerequisites are that the metrics can be quantified and compared, and stakeholders must be able to translate them into tangible benefits (e.g. a dollar value).

After determining the change in these metrics before and after BDD or TDD adoption, stakeholders can start to determine the value of those changes. The value may be somewhat subjective (e.g. how much is a decrease in defects truly worth?), but at least they have some quantitative inputs from which to base decisions. They can compare those benefits to the ongoing costs to come up with the net benefit.

The final step in the process is comparing the net benefit to the upfront costs to get started (e.g. training and costs). This comparison will help stakeholders justify the investment and potentially expand the practice throughout the organization.

The Bottom Line

Behavior-driven and test-driven development have become increasingly popular, but the practices aren’t ubiquitous across all organizations. If you want to promote these practices in your team, you may have to demonstrate the return on investment to persuade stakeholders to invest in the upfront and ongoing costs.

In addition to calculating ROI, you may want to consider using tools like HipTest to simplify the collaboration between business and technical teams. By lowering these barriers, you make these changes more approachable for everyone. HipTest also generates living documentation for stakeholders to see progress on features via tests in continuous integration.