Ouest-France

Business Needs and Challenges

With more customers taking to Ouest-France’s digital platform, the company saw sense in completely overhauling the website in 2017. This decision came more from a technical necessity, than merely from a functional point of view. Some high-level tests were executed manually, and software teams practiced a little bit of Continuous Integration (CI) on the old website. However, the lack of feature documentation and shared knowledge caused the website to suffer in terms of quality. This led to multiple hotfixes that needed to be applied frequently, which were costly and potentially risky for the company.

Therefore, there was a need to address software quality in a way that would meet Ouest-France’s business needs. The company determined that quality could only be attained through comprehensive product documentation, sharing and designing executable requirements and test automation. Additionally, the desired confidence and product quality pointed towards a possible solution that would utilize the principles of Behavior Driven Development (BDD). A further analysis revealed that a BDD approach would lend its benefits perfectly to Ouest-France’s desired organizational goals.

There were challenges however, as embracing BDD would mean completely overhauling the existing software development process. Most of the product owners were not accustomed to the BDD philosophy. Therefore, ensuring that the BDD approach bled through the entire organizational structure was the primary hurdle that had to be overcome. To bolster the adoption of this approach and to actively ensure that their product owners started to think ‘agile’, Ouest-France decided to use a software tool that would would act as an adhesive to create the quality website they envisioned. SmartBear Software’s CucumberStudio was this tool.

Targeted Solution

At the time, Ouest-France did not possess complete documentation of legacy products, which posed a problem at first. CucumberStudio helped them create a comprehensive list of every functionality of the new website, which was now ready to access and analyze on demand. This allowed the company to update, create and enrich their set of executable requirements every day. These requirements in the form of simple test scenarios were stored as living documentation in CucumberStudio and allowed anyone to quickly understand every feature.

This streamlined BDD approach also facilitated discussions between the product owner, developer and tester which reduced the time taken to design the new executable requirements from many hours to a few minutes. This quick turnaround time for setting the requirements was possible as Ouest-France developed and leveraged standardized, reusable action words when creating their test scenarios.

“A major hurdle we faced was the granularity of the executable requirements.,” says Florent Vaution, QA manager at Ouest-France. To combat this, they decided to bifurcate the BDD approach into a functional level and a technical level. The former comprised of mainly user interface (UI) testing which proved more difficult and expensive to automate and maintain from Ouest-France’s point of view. The latter consisted of tests directly carried out on the APIs that embed most of the features. These API tests were faster to create and execute, and far less cumbersome to maintain, given the current skillsets of their team.

Another challenge initially was the relation with the developers that had to implement action words (ie test steps). Because they started from scratch, building the dictionary of action words was a little expensive during the nascent stages, since there was a lot of refactoring involved. But, by building on to their action word repository and making it more robust, the current cost of creating a new automated test is next to nothing.

Initially, Ouest-France’s QA team consisted of only two people who were setting the rules for writing executable requirements and creating action words. They then added three more testers who had no previous experience with BDD. Nevertheless, everyone on the team easily picked up the BDD philosophy and were entrusted with adding new features independently.

Starting out, the QA team members at Ouest-France were the only users of CucumberStudio itself. Slowly, the product owners and the developers were shown demos of how the QA team used CucumberStudio, with the objective of training them to use the tool themselves to write new features. The QA team members were the only ones responsible for the documentation of features and requirements. However, now the product owners and developers do not hesitate to challenge, correct or even change the features written by the QA team.

Another advantage of using CucumberStudio, is that it allowed the QA team to check if a feature has been tested, how it was tested and if an action word already exists for that test scenario to avoid duplication. This especially saved valuable time for Ouest-France which was dealing with more than 400 test scenarios and 500 action words.

Key Results and Takeaways

Since implementing BDD using CucumberStudio for the website redesign project, Ouest-France went from the “idea” to the “production” phase in the first half of 2017. The old website and processes consisted of only some manual tests with no test automation. The new website and processes consist of several hundreds of automated tests and very few manual tests.

Before implementing BDD, they were averaging about one hotfix per day, but now, they can keep up with instantly migrating changes to the production environment for the new website.

• Some key CucumberStudio features that Ouest-France found the most useful were:
• Autocompletion during scenario design
• Ease of use of variables in scenarios
• The speed of test run creation
• Living documentation
• CucumberStudio publisher with links to an SCM and Jenkins