The Company
The client is a leading technology company in the insurance industry. They manage the IT operations for their parent company, supporting thousands of employees and overseeing millions of insurance contracts in Central and Eastern Europe. The organization provides a wide range of personalized insurance products and services through its large network of subsidiaries.
In recent years, enterprises have adopted new tools to improve test automation. Tricentis Tosca, with its AI-driven approach for Cloud and DevOps, and Xray, our test management tool integrated with Jira, are examples of this. This success case shows how connecting these tools led to a 90% reduction in error analysis time for a leading insurance tech player.
The Challenge
The company needed to unify testing processes across various departments. With hundreds of testers performing manual testing tracked in Xray, results from Tosca's automated regression testing - approximately 15.000 tests executed weekly - were manually merged into Xray reports. This process limited visibility and created inefficiencies for teams lacking direct access to Tosca.
To address this, a custom connector was developed by Automators, designed to synchronize test cases between Tosca and Xray, as well as optionally in the reverse direction. Let’s dive into the solution.
The Solution
The client implemented the Automators Xray Tosca Connector, establishing a seamless connection between Xray and Tosca and a one-view reporting for all executed test cases. With the connector, all test cases, steps, results, and screenshots are automatically transferred to Xray, making them easily accessible to all business users.
The solution goes further by leveraging AI-based error classification, which helps distinguish between business-related errors and technical failures, significantly reducing the time spent analyzing failed test cases.
Additionally, the connector translates logical test cases developed in Tosca into fully detailed test cases in Xray, enhancing Xray’s test management capabilities and fully integrating automated test cases. This integration allows Tosca test results to be linked to user stories, providing teams with comprehensive coverage analysis that includes both manual and automated testing outcomes.
How the integration works
The Tosca-Xray Connector was developed to synchronize test cases between Xray and Tosca. This two-way integration ensures seamless communication between the tools, enabling users to receive comprehensive test results from all routines effortlessly and fully automated.
How to use Tosca-Xray Connector:
The connector streamlines the synchronization of objects between Tosca and Xray without manual intervention, offering an efficient way to manage testing. It supports 3 key areas of integration:
- Test Case variant sync: Web application test case variants in Tosca can be synchronized with Xray in seconds, transforming Tosca’s attributes into Xray-readable test steps. This reduces setup time and effort, helping users fully leverage Tosca's design capabilities for manual tests in Xray.
- Synchronizing complete Test Cases: Both manual and automated test cases from Tosca are transferred to Xray with all steps and information intact. The connector ensures that every detail is maintained, creating a unified test element in Xray, which simplifies the testing process and improves accuracy.
- Centralized reporting: With it, large enterprises gain a comprehensive view of their testing activities. All Tosca test results are displayed alongside manual test outcomes in Xray, offering a consolidated reporting view that enhances visibility and testing performance across the business.
Editions of the Connector
The Tosca-Xray Connector is available in 3 editions to meet different business needs:
- Standard edition: Includes base synchronization features and is designed for easy self-installation.
- Extended edition: Offers additional customizations such as individual field mappings, with assisted installation and operation.
- Tailor-made edition: A fully customized version to meet specific requirements, tailored by Automators to suit unique business needs.
Key Benefits
- Save time by designing test cases in Tosca while collecting results in Xray;
- Benefit from Xray’s advanced dashboard and reporting features;
- No expensive adaptations or extensions are required;
- Simple and fast installation process;
- The connector is compatible with all versions of Tosca and Xray, ensuring ease of use across different environments.
The Results
The integration streamlined testing processes and enhanced cross-departmental collaboration. Key outcomes include:
- 50,000 Test Cases synced: All test cases are now unified and accessible within Xray, providing a single source of truth for test management.
- 15,000 Automated Tests executed weekly: Automated test executions are seamlessly integrated with manual test results, offering a complete overview of test performance.
- Error analysis time reduced by 90%: Thanks to AI-based error classification and centralized reporting, the time required for analyzing errors has been eliminated, accelerating the testing cycle.
The outcomes were:
Improved team collaboration: Around 100 business users now have direct access to biweekly regression test results, enabling faster decision-making and better visibility into testing outcomes. This access was previously unavailable to most business users, greatly improving collaboration between IT and business departments.
Enhanced Agility: The client’s processes became more agile by improving the analysis of failed executions. Reports are now available in higher quality and earlier, enabling faster feedback loops and quicker issue resolution.
Numerical improvements: The integration has reduced the effort required for test analysis. Comprehensive reporting is now available in Jira each Friday, 2 to 3 days earlier than the previous process of sending out PDF reports via email the following week.
Impact on business performance: While the integration’s impact on business performance is not easily quantifiable, it has led to higher satisfaction and better cooperation between IT and business teams. The results of approximately 10.000 automated test cases (reduced to a business-relevant subset) are now easily accessible in Xray.
About the Partner
Automators is a leading consultancy in automated, continuous testing of critical systems and processes. They make enterprise software more productive, reliable, and profitable. Since 2018, they have continued to grow and innovate.
Automators is a dynamic group of individuals passionate about their work and dedicated to delivering the best results for their clients. This also includes products like the Connector and DataMaker, a new comprehensive tool to handle and create Test Data. With operations spanning continents, they maintain a strong global presence. Their headquarters are in Vienna, with additional offices in Brno and Zürich in Europe, and Dubai in the Middle East, enabling them to serve clients worldwide.
