If you want to manage a release process that is easy to follow, we find that Jira with Xray can be a great fit for most organizations that want to keep quality at the center and implement agile best practices.”
Gianni Pucciani - QA Specialist - TCS
With over 1.5 million members, Touring Club Suisse is the largest mobility club in Switzerland. Founded in 1896, TCS provides assistance, road safety education and accident prevention to all travelers and vehicles along the entire Swiss road infrastructure.
TCS adopted Xray for their test management in 2015, which allowed them to manage all the testing activity in Jira and have a simple and solid QA and release process all in Jira. They saved time in Test Definition (through re-usability), Defect Analysis (through the link Defect-Test), and Reporting (no need to collect input data via email and aggregate test results from several Excel files).
In December 2019, TCS transitioned away from writing test automation code using Selenium. Instead, they would adopt Boozang to automate their tests, a tool that allows non-developers to participate in automation. The integration with Xray was easy thanks to the well-documented REST API’s that allowed them to export test definitions and import test execution results.
In this Success Case, they share why they outgrew their automation process with Selenium and Jenkins, and how their new automation process with Boozang helps them scale their test automation.
In 2015, TCS started investing in test automation with a custom made solution based on Selenium, Cucumber, Xray, and Jenkins. The objective was to create a regression tests suite to cover the main business processes on the Salesforce platform, used not only as CRM but as a product catalog, sales, and contract management tool.
Features and Scenarios were documented in Jira via the Xray and executed nightly on 2 test environments via Jenkins. Results were available in Jenkins (via the cucumber-reports plugin) as well as in Xray.
As their maturity evolved and application complexity had grown they wanted to: reduce dependency on developers; do end-to-end testing over several applications; keep existing ways-of-working, including Xray and Jenkins.
Simple and smooth process: The definition of automated test cases in Xray via the Test Type Automated Cucumber was well adopted by business analysts. The integration with Jenkins allowed the nightly schedule of automated tests and the possibility to analyze quickly the test results in the morning, with a drill-down view from features to scenarios and steps.
Automated and Manual tests all in the same place: allowed to report the integration of the QA activity in a single dashboard. Tests for new features and bug fixes were mainly manual, and regression tests automated. They use the Overall Coverage Gadget to show the progress of testing.
Developer capacity: Developing new scenarios and maintaining the current ones was taking a considerable fraction of the time allocated to develop new features. Only 2 out of 6-7 developers were able to maintain and extend the framework which was solid but not the easiest to master.
Limited to a single web application: Selenium worked ok as long as the system under test was a single one, in this case, Salesforce. Automating and end to end process going over three web applications (E-Commerce site, Salesforce, and backend applications) was too complex with Selenium.
Outside of coding and maintenance of Selenium tests, the existing process at TCS was working well. The challenge was therefore to replace Selenium while maintaining their current way-of-working.
They wanted to continue to use Xray and Jira for Cucumber test management, which was working very well. They also would like to keep Jenkins as their CI servers and maintain the Cucumber reporting format.
The team decided to look for a codeless solution that fulfilled these main requirements:
In the search for the right tool, they encountered Boozang: AI-powered test-automation that runs in any web browser, and saves the data in the Cloud.
A proof of concept went smoothly, the tool was easy to learn and they did a pilot to further explore it. The pilot phase lasted 2 months, with approximately 20 man-days. The pilot was meant to challenge the solution on real-life scenarios, covering 3 web applications, and integrating with Jenkins and Xray.
In December 2019 after a successful pilot, Touring Club Suisse adopted Boozang to automate their tests and allow non-developers to participate in automation.
The integration with Xray was easy thanks to the well documented REST APIs that allowed them to export test definitions and import test execution results.
“We like that Boozang focuses on web test automation and it does it well. By leaving the scheduling to Jenkins, the requirements management to Jira, and the reporting to Jenkins, we can keep our current setup and just replace the challenging part: implementing tests in Selenium.”
Gianni Pucciani - QA Specialist - TCS
“Test automation is an important factor to increase the quality and time to market. For test automation to be efficient and able to cover end to end business processes we find that a BDD like approach coupled with Boozang is a great solution that integrates well with Xray and your CI/CD infrastructure.
Gianni Pucciani - QA Specialist - TCS
The new process was therefore quite similar to the previous one. The most important change was becoming an agile team that works more efficiently on defining and implementing the scenarios.
Clean design: For QA practitioners Xray feels like home. The design is clean and the available objects allow for full quality control during the release process, supporting from simple single applications to complex multi-application and multi-environment enterprise context.
Manual and Automated tests both in the same place: With Xray, you can manage both manual and automated tests using the same process and tool. If during a Sprint part of the Test only part of the test was automated, you can still have a unified report that shows the test coverage of both automated and manual executions.
Documentation: Keeping the documentation up to date is a known challenge for software editors, and the Xray team does this job very well. Be it an API description or a new feature, all you need to know about Xray is always available on their Confluence page.
Great support and clear roadmap: Xray support is reactive and very efficient. Their roadmap is clear and really tailored to users’ needs.
As TCS’s test automation matured and efforts increased, Xray’s flexibility helped them scale their test automation. Xray allowed them to easily adopt a new workflow, reduce dependency on developers, do end-to-end testing over several applications, and keep existing ways-of-working with Xray and Jenkins.
Now, everyone involved in the software development process can write automated tests, making them faster and even more agile. Now that’s the kind of flexibility you want out of your SDLC toolset.