Hi everyone, Before digging into the first blogpost of the year, we wish you a Happy New Year! Now, let’s start our Test management tool comparison. The software development process evolves continuously and we have identified 3 main contexts for testing activities.

1^st context: Waterfall process

In organizations using Waterfall-like process, QA teams usually work in a silo (not embedded in development teams). With a new release every 3 months or more, test managers spend a lot of time in planning activities which require strong tool support. A lot of tests/checks are manual. Key features needed:

• Comprehensive planning features
• Comprehensive reporting widgets

 

2^nd context: Agile teams using mix of manual and automation for test execution

QA department (if any) support agile teams with coaching. Testers are embedded in development teams. With a release every 2 weeks, agile teams do little planning and need to automate regression tests and do exploratory testing. Test coverage is also ensured by unit and integration tests.   Key features needed:

• Simple planning
• Test automation

 

3^rd context: DevOps teams using BDD, automation and integration to CI/CD pipeline

Teams doing continuous deployment deploy couple of times a day in production. Devs/Testers/Ops work as a single team. Every test/check is defined prior to development and is automated. This is part of the shift left movement and the testing process is integrated with the CI. Key features needed:

• Integration with CI/CD
• Native support of BDD
• Living documentation

 

Best test management for Agile and DevOps teams

How we build software is complex and changing. Feature requests change overnight, and timelines are accelerated. So for this 2018 test tools comparison we decided to focus only in context 2 and 3 for Agile & DevOps teams. Our list represents the most popular tools in the Atlassian Marketplace ecosystem. 1. Zephyr Zephyr Jira add-in provides a real-time test management tool enabling to design, execute manual tests and create comprehensive reports. Zephyr integrates with CI tools and has REST APIs. That requires additions tools and plug-ins. Zephyr is available in the cloud and on-premises. The company also provides an enterprise version of its platform. 2. Hiptest Hiptest is a continuous testing platform enabling Agile & DevOps teams to test their software from idea to production. It has a native support of Behavior Driven Development and Gherkin syntax. With Hiptest, you can create, organize, execute, automate your tests. Hiptest has a Jira add-on and can be integrated with 15+ test automation frameworks. The plus? You can generate your own living documentation!   3.TestRail TestRail is a test case management tool that integrates with several issue tracking tools. With it, you can create, execute tests and create comprehensive reports. TestRail also supports CI/CD integration and has organization capabilities (by project, version,…). Testrail integrates to Jira with an add-on.   4.XRay Only available in a server version, Xray is a test case management tool that supports BDD and enhances design, organization, execution and automation of tests.  XRay is a Jira add-in.  

Here is a comparison of the tools capabilities.  If you have any suggestions, just send us a quick email and we'll polish this table.