Before, I would spend over 5 hours getting an overview of our testing progress, what was being executed, and what was missing. It was time consuming and tedious. With Xray, I’ve reduced my time in gathering information about our testing progress by 50%.”
Kenni Thomasberg | Software Test Engineer at DEIF
About DEIF
Established in Denmark in 1933, the DEIF Group is a global market leader in green, safe and reliable control solutions for decentralised power production on land or at sea. In simple terms, they make sure power is always available. DEIF Group is staffed by more than 550 employees and represented in 50+ markets in all regions of the world.
All production facilities are located at the Danish HQ in Skive and 20% of their employees work in R&D identifying new environmentally friendly ways of increasing overall performance, reducing maintenance intervals and fuel consumption for the world’s power generating industries.
The Challenge
DEIF’s industry is heavily regulated. The supply power for industrial equipment like ships and hospitals are audited by big insurance companies that have strict guidelines. They need to be compliant to all of the regulations and rules, as well as show proof of documentation for all of their testing.
What does this mean for their test management? The test strategy has to be completely foolproof!
- DEIF needs to be able to track every part of the testing process;
- For every test case, complete requirement traceability needs to be provided;
- And lastly, deliver documentation for each test case and defect.
Previous to Xray, they were using Word and Excel to manage their vast number of test cases. Requirements traceability was hard to manage and wasn’t always 100% accurate. They also spent an enormous amount of time on tracking testing progress, holding back their execution.
Their top challenges in test management were:
- No requirements traceability;
- No formal test management tool;
- Too much time spend on reporting;
- Lack of connection to CI/CD pipeline;
- No visibility into coverage, defects and execution.
They needed a test management tool that would cover all of these points. Since they already used Jira for software development, they looked into the Atlassian ecosystem to find their solution.
The Solution
In test management, we want to get testing as soon as possible. The sooner we can get results back to the developer, the cheaper it is to solve an error. In our new workflow with Xray, we find errors earlier because of the shared visibility.”
Kenni Thomasberg| Software Test Engineer at DEIF
Previously, the amount of data and tests that they produced got too large to handle only with Word and Excel. They needed a test management tool that would ease the process and save them time on reporting and documentation. They chose Xray when comparing to several other tools in the Atlassian Marketplace including Zephyr and TestRail.
Once they adopted Xray, it was easy to implement their own processes. DEIF does a lot of test automation, and they have their own tool for test automation that produces XUnit files from the test run. Using Xray’s powerful REST API, they connected their test automation tool, and are now able to execute automated tests, and show the results in Xray. It was a seamless experience.
Currently they use AHA as a system to do release and milestone tracking between their requirements. There’s a link between AHA and Jira where they map features and functionality in the products from AHA to Jira in stories and epics.
They use Xray, to map all stories as requirements, then map test cases against the stories. If there is a change in a requirement, from AHA, the changes will be pushed over to Jira, and the changes will be mapped to new user stories.
In terms of processes, they follow a SCRUM framework. Testers are system test developers, and they create and execute test cases. They have 2 or 3 week sprints, working in an agile workflow to deliver releases and features.
The Results
It’s been 4 years since DEIF started using Xray, and they’ve seen significant efficiency gains. Here are their top 4 benefits from using Xray for their test management.
1. Optimized test case management + visibility
DEIF’s QA team has about 500 test cases per module or project. They use Xray’s Test Plans board to organize their tests and track their progress per project. Before Xray, the QA team would spend more than 5 hours getting an overview of testing progress, what was being executed, and what was missing. It was time consuming and tedious.
Teams were constantly arguing about spending too much time on giving feedback. Data collection was tedious and it slowed down their test execution. With Xray, they've reduced their time in gathering information about their testing progress by 50%.
2. Foolproof requirements traceability & documentation
Meeting their industry’s strict regulations and audits for requirements traceability was a big bottleneck for DEIF. Xray provides full requirements traceability due to its Jira native integration. Now they’re able to create a traceability report with just a click, and deliver documents that would have otherwise cost them hours of time.
Also thanks to Xray’s enhancements on the Jira Agile Boards, everyone on their team, including developers, has visibility into the status of their requirements and test cases.
3. Integrate automation into the CI/CD pipeline with Jenkins
DEIF heavily focuses on test automation. Some teams have almost 99% automated tests, while others do 50/50. At the systems level, they had a lot of manual tests, and now with Xray they try to automate even more test cases.
Thanks to Xray’s REST API they connected their own automation tool and use it to run automated tests and report the results in Xray. When they have all the test cases developed and test suites are running, they create a test plan in Xray, and map that Test Plan in Jenkins. Every test case that's run is automatically created in Xray, and everything is correct from the first-go.
4. Saved time on reporting & management
During a release test, they see the Test Plan Report to get a quick snapshot in time. The report gives a clear understanding of how many test runs they ran in different test plans, and gives them a fast overview into the different requirements they need to meet.
Test automation runs at least daily, so they use the Test Execution Report to check the status of those tests and which ones are failing. They have much better test coverage because requirements are linked - this was all done manually before.
I really like the Overall Requirement Coverage Report because it gives you a clear picture of what requirements are not covered with test cases, it gives us a fast overview of what I need to do in the project. I try to hit 100% as fast as I can.
Kenni Thomasberg | Software Test Engineer at DEIF
The Future
Since DEIF adopted Xray, they saved hundreds of hours on documentation and test progress review. When it comes to a release, they can quickly access their status of testing and readiness to deploy.
Xray’s requirements traceability report keeps them 100% compliant with industry regulations and audits, giving them certainty in their quality efforts.With Xray’s REST API and integration with leading automation frameworks like Jenkins, they’ve set up a unique and efficient CI/CD pipeline which keeps them agile.
Their commitment to quality and software testing, combined with the right tools, pushes them to be more reliable, agile and competitive in their industry.
