Flowbird

The Problem

Flowbird, Tieto, HSL and all the other partners involved in the project were spread across 6 different countries. They had to work on a large ticketing system that involved around 50 configuration and monitoring modules, 300 sales and service points as web servers, and over 6,000 on-board and street-level devices working on Android. With over 8 distributed technical and business teams in 3 different countries, Flowbird recognized that a deeper level of collaboration and communication was a requirement to bring the teams closer together to have a successful venture.

The first step would be to ensure that the business and technical teams were on the same page with respect to the different features of the vast ticketing system that was being worked on. This is where CucumberStudio and BDD come into the picture. Behavior Driven Development or BDD is an agile approach to software development, that creates a shared understanding of the features of a product by defining simple test cases that mimic common user behavior through collaboration. CucumberStudio is a cloud-based agile test management tool that helps teams co-design test scenarios based on acceptance criteria, plan test execution and maintain both manual and automated test results. With native BDD frameworks and Gherkin support that can directly integrate with CI/CD pipelines, CucumberStudio enables real-time discussions and collaboration between all the project stakeholders and helps reflect the most up-to-date state of the project through living documentation.

In addition to using CucumberStudio for Agile testing and BDD frameworks, Flowbird and its partner companies also used Jira for project management, Cucumber for test automation and Jenkins for continuous integration.

Solution

Flowbird tackled this huge project with a threefold approach. They first collaborated with international teams to determine all the different units that would make up this complex ticketing system. They then went about defining the behavior of these individual units or ‘modules’ with BDD using the Gherkin syntax to create test scenarios. Raphaël Citeau, the delivery manager at Flowbird said, “The native support of BDD and the Gherkin syntax in CucumberStudio brings clarity to the design of test scenarios.” CucumberStudio allowed Flowbird to have a smart testing framework and to develop BDD test case scenarios at scale.

They also realized that there was a need for a common business terminology that could facilitate a shared understanding between their teams and customers to develop new features. “The reuse of test steps in CucumberStudio allows us to standardize the terminology and improve team collaboration,” explained Raphaël, as Flowbird was looking for "one method that can fit in to all types of teams regardless of the technology they are using." BDD was the golden opportunity to create this universal business terminology as various modules were defined. This business terminology was used to create and define more modules in the future. In this way, functional unit test scenarios could be created, reused and then promptly automated; saving considerable time.

The next step was to create integration tests from the previously defined functional unit tests. These integration tests would be the basis for assembling different modules together and were automated. Finally, the complete system was built by assembling all the modules together. For this, Flowbird used the integration steps to make end-to-end tests that verified the integration of the required hardware components with the assembled ticketing system modules.

In addition to using consistent business terminology to promote shared understanding and using a Gherkinsyntax for BDD, Flowbird also practiced test refactoring with CucumberStudio to create quality software. By refactoring their tests, they were able to identify duplicates and create a single point of maintenance for these tests. It was also possible to automatically refactor the test scenarios, meaning that by changing one test step, all the tests with dependent test steps are automatically updated, thereby saving time.

Apart from creating test scenarios with BDD and automating them, CucumberStudio also provided Flowbird a place to manage all their features as test cases, store them in a central repository and share them across the organization. This reflected the most up-to-date versions of the test cases and helped business stakeholders, developers and testers be on the same page, thereby simplifying test maintenance. This was the true value of living documentation. Flowbird also practiced a bi-weekly, iterative framework for software development, known as Scrum. This enabled Flowbird to make incremental improvements to the project through collaboration, adaptability and the focus on delivering customer value.

Benefits and Key Results

Flowbird used CucumberStudio and BDD to create 1,635 functional unit tests, 990 module integrations tests, 475 systems tests and 200 end-to-end tests. However, in business terms, they created only 1,690 test steps. The remaining 1,610 test steps didn’t need to be manually created as they could be reused from the previously developed steps. “Thanks to the test refactoring and step autocomplete features, we have automated 3,300+ functional tests by implementing just 1,700 steps! This was definitely the key to accelerating automated testing at scale.”, says Raphaël. Reusing previously-created test scenarios and automating over 300 man-years worth of manual work saved Flowbird and its partner teams years of time.

Additionally, there was never more than a week of delay on any deliverables from the team’s side. This is also attributed to the time savings associated with the automation of the test cases or user stories written in the BDD framework. Furthermore, any type of rework in development or testing constituted less than 3% of all the work done by the teams.

The living documentation feature of CucumberStudio also ensured that features could be changed and the respective test scenarios could be seamlessly updated. Flowbird found that 20% of the features out of the initial 800 could be changed without any impact on the project. This reinforces the Agile mindset that the different teams strove to achieve; one that allowed small incremental changes to be made which delivered the maximum value to the user.

When asked about Flowbird’s favorite feature in CucumberStudio, Raphaël says, “CucumberStudio is very easy to use and integrates smoothly with Jira and into our continuous integration process. It fits well for manual testing as well but is clearly oriented for automation."

So next time you visit Helsinki and ride on the local commuter rail along the scenic Finnish coastline, know that your exp