Test automation is gaining popularity and is frequently being adopted by organizations as a part of their software testing activities. However, when moving from manual testing to test automation, people often overlook the common pitfalls associated with automating tests. Although there is no “right way” to go about automating your testing efforts, there are some useful practices that can help you build a test automation process that scales. Read on to learn more about what we think is important to keep in mind when automating your tests.

#1 Every Test Should Tell A Story About Your Application

One of the most popular questions we get is, “Which tests should I automate?” And our answer is always the same, “Why have you created this script? It doesn’t matter if the execution is manual or automated. Tell me what the intent of this test is.” While this may not be what everyone wants to hear, most of time their answers are not clear. People are often talking about test automation and making big investments in their automation efforts, however, the intent is not always clear.

Before deciding what tests to automate, it is important to ensure that your tests are readable and have been created with a clear purpose in mind. Every single test should tell a story about the application. This is where Behavior Driven Development (BDD) comes into the picture. BDD is an agile approach to software development where business stakeholders, product owners, designers, developers and testers collaborate to define product features through examples that mimic user behavior. These examples or user stories act as both the living specification of your software and as acceptance tests for developers.

Once you’ve made it clear why you have a test and are aware of its value, then you can start thinking about test automation. If your organization performs continuous delivery like we do, it is advisable to try and automate as many scripts as possible, because immediate feedback is invaluable.

#2 Test Automation is About Velocity

A common misconception about test automation is that you will have no more manual work to do. Automating tests is expensive and requires some degree of manual work. Scripting automated tests is a complex and sometimes time-consuming activity. Maintaining them is a whole different battle, as even automated tests need to be updated as your application changes. If automating tests involves a lot of work and resources, then why do people do it in the first place?

This is because test automation pays back at a higher velocity. Weeks of manual testing condenses to a couple of days when you begin automating your tests—an immediate recognition of the time you get back. This means that testers can get back to developers with their feedback faster, fixes made earlier, and software shipped quicker, thereby saving your company time and money. The work done to setup automated tests and maintain them is an initial investment that gets paid back quickly by the benefits of accelerated software testing through test automation.

Automation requires some upfront work, but there are also tools available that can help you. Automation tools can streamline the process by simplifying test creation and automatically updating tests as changes in the application are detected. Adopting agile approaches, like BDD can also accelerate testing and ensure that organizations develop quality software that meets user needs. Adopting BDD will take some initial investment. Team structures will change as people from different teams will need to collaborate on feature definitions based on user behavior. This investment of time and effort will however be rewarded with faster software releases, better understanding of product features, and smoother cross-team collaboration.

Several companies have found success in test automation. Parkeon, one of our customers and a leader in the smart-cities industry, automated over 300 man-years worth of manual work to develop a colossal system to manage ticketing and transportation for Helsinki, the capital and most populous city of Finland. They adopted agile practices and built thousands of automated tests using BDD. Parts of the tests were executed at the API level, some at the GUI level and a few of them on real devices with robots giving inputs.

As you can imagine, building such an infrastructure was expensive. However, it was worth the investment. The value that Parkeon was looking for was the speed of feedback. With test automation, they were able to detect unanticipated impacts due to changes in the code very quickly. This enabled the team to iterate faster, add new features and modify existing features with confidence.

#3 Record-and-Playback Tools Are A Great First Step

Record-and-playback tools are great for teams that are just starting their transition to test automation from manual testing. It enables team members of all skill levels to start automating. It is especially useful for testers who can’t create complex test automation scripts. While record-and-playback tools can help you automate your tests, at some point, test maintenance will be a factor to consider. Sometimes, these kinds of tests are fragile, and cannot give you the full test coverage you’re looking to achieve. Record-and-playback can be useful as a quick start strategy but it’s important to modify and maintain the scripts directly at the code level after a point. So, it is important to select a test automation tool that provides the flexibility to use record-and-playback to get started quickly and easily, but also offers scripting capabilities as your test automation needs become more advanced.

#4 Collaboration is Key.

Teams are made up of different kinds of people with diverse skillsets. These people may include test automation engineers, developers, or SDETs (software developer engineers in test).

If you want to start practicing test automation, it’s not just about the skill-level of your team; you must also ensure that people are communicating openly. For example, when you automate a test, you need to decide if the behavior should be driven at the GUI level, API level, or if there are any redundancies with the unit tests. This necessitates an open channel of communication between all team members. Miscommunication between testers and developers is common. A testing leader when questioned about the stack his developers used replied, “I don’t know, they are in India, and I have no idea about the language and frameworks they use.” This lack of collaboration spells disaster when teams try to embrace test automation. Collaboration helps teams find bugs quicker and release high quality software faster through automation.

#5 Build a Test Framework in Layers

If you want your test automation framework to scale, then it’s important to use a multi-layered approach. What do we mean by this?

At the high-level business layer, you start by defining your test cases using consistent business terminology broken up in “steps.” This is where tools with auto-complete features and refactoring capabilities help test design. Auto-complete features ensure that you don’t end up with separate “Sign-in” and “Login” steps that have the same meaning but are written in two different ways. Therefore, by using distinct “action-words” to create steps, teams can avoid redundancies.

The technical layer is relevant once you have designed readable test cases with a clear intent in the business layer of your test automation framework. The next step is running these test cases or steps in test fixtures using page object models or journey patterns, and then testing their interactions with browser automation tools like Selenium WebDriver.

Having a multi-layered framework will enable you to limit the impact of a change when it occurs and will help make your test maintenance so much easier.

If you really want your business to scale with test automation, there is no shortcut. Tools with record-and-replay features can help you get started, but you’ll also need hard work, open collaboration between developers and testers, and a working test automation framework to succeed.