Behavior-driven development, or BDD, has become very popular over the past several years. While it’s easy to gather around the table and come up with Given-When-Then scenarios, going through the motions doesn’t necessarily lead to tangible benefits. Defining solid metrics and making progress visible is necessary to truly realize the benefits of BDD.

In this article, we will take a closer look at the metrics that matter and how to make that progress visible.

Setting Expectations

There’s no question that behavior-driven development has a learning curve for development teams. While your team may experience a drop in defect rates early on, productivity often suffers as the team learns the ropes. The payoff comes over time as the processes become more ingrained in your culture.

John Ferguson Smart and Jan Molak describe five stages of BDD adoption in their 2016 blog post:

Five Stages of BDD Adoption - Source: John Ferguson Smart

Many organizations fail to surpass the “Appreciation” stage of BDD adoption, where they are at risk of becoming an expert beginner. The best way to ensure that you continue to progress is to implement simple and practical metrics that can ensure your team is moving in the right direction.

The authors note what to expect at each stage of development:

• Disruption: Productivity drops as the team learns and adapts to BDD, but there’s typically an early drop in defects.
  
  
• Appreciation: The breakeven point in productivity and throughput is reached and quality gains continue.
  
  
• Comprehension: There are solid gains in throughput.
  
  
• Adoption: Gains in productivity continue.
  
  
• Extension: Optimum performance is reached, defects drop to near zero, and throughput dramatically increases.

Development teams that effectively implement BDD practices can reduce their defect rates to nearly zero while improving their productivity by upwards of 50 percent.

Code Coverage

Code coverage, or the percentage of code covered by unit tests, is a contentious topic among software developers. Some developers argue that they create perverse incentives by encouraging shortcuts, while others argue that it gives development teams a quantifiable goal.

[content_upgrade cu_id="5368"]Download our free list of BDD development tools to help you maximize the benefits.[content_upgrade_button]Click Here[/content_upgrade_button][/content_upgrade]

Unlike unit testing, BDD scenarios are specifically designed to test use cases or functionality that’s available to end users. Passing test cases from scenarios create what’s known as living documentation, or a record of user-facing features that are verified to be working properly.

Living Documentation on HipTest - Source: HipTest

Code coverage for BDD scenarios can be measured in two ways:

1. Feature Coverage: You can look at the percentage of user stories and/or features that are covered by BDD scenarios. You should strive to see a gradual increase in the number of features covered by tests (developed after the adoption of BDD) in order to realize the benefits of BDD and generate living documentation for stakeholders.
   
   
2. Code Coverage: There are plenty of tools designed to show the percentage of code covered by unit tests, but it’s less common to generate code coverage metrics for acceptance tests. That said, it’s possible to generate these metrics using specific tooling. For example, in Java, passing `-javaagent` to Gradle will show all classes run by a Cucumber scenario to provide coverage metrics.

Defect Rate

The defect rate is the number of defects—including misunderstood requirements—that are raised after a user story has been delivered. For example, a user story may have covered signing up for a new account, but developers may have forgotten to fire off a welcome email as part of the process.

Defect rates are typically measured as the number of bug issues opened divided by the total lines of code. Steve McConnell, the author of Code Complete, found that the industry average defect rate is around one to 25 bugs for every 1,000 lines of code. But these are just averages across a wide range of industries.

Defects are inevitable at some point during development, but there are many different strategies to minimize them. Behavior-driven development helps avoid defects by better communicating user stories through concrete examples that are converted into executable specifications.

You should look for defect rates to immediately fall around 20 percent following the adoption of BDD and gradually move toward a 90 percent or greater reduction as your team becomes more experienced. It’s not unheard of for experienced BDD teams to have zero defects hit production.

Throughput

Throughput is the number of completed user stories over a set period of time (e.g., per week or month). If you’re using point estimates, you can measures throughput as the number of story points that are launched into production. These figures help you assess your team’s overall productivity.

Throughput is also expressed as the project’s velocity on a burn down chart. When sprint planning, velocity can be a very helpful metric to estimate how much work a team is capable of finishing and delivering over a period of time. The goal is always to maximize velocity without increasing defects.

While behavior-driven development may seem counterproductive—since there are more meetings and tests to write, the improvement in communication gradually helps improve throughput over time. You don’t have to worry about missing or misunderstood requirements.

It’s common to experience a slight reduction in throughput immediately after the adoption of BDD practices, but over time, you will see throughput accelerate. Experienced BDD practitioners experience a 50 percent or better increase in throughput over time with far fewer defects.

Getting Started

There are many different tools that you can use to measure code coverage, defect rates, and throughput. For example, SmartBear’s Collaborator enables development teams to review defect metrics and generate reports to see trends over time and assess the effectiveness of behavior-driven development.

[content_upgrade cu_id="5368"]Don’t forget to download our free list of BDD development tools to help you maximize the benefits.[content_upgrade_button]Click Here[/content_upgrade_button][/content_upgrade]

In addition to measuring the outcomes, you may want to consider tools that simplify implementation and reduce the learning curve. HipTest is a great web-based tool for BDD testing that’s complete with a scenario editor, living documentation tool, and automated test generation.

Try HipTest for free today and level up your BDD practices.