The Company
A leading institution in the education sector successfully migrated its extensive test management data to Xray Cloud, achieving significant improvements in performance, user satisfaction, and operational efficiency. This case study explores the challenges, solutions, and results of this complex migration, highlighting the value of thorough preparation and strong partnerships in managing large-scale data migrations.
The client is a prominent institution in the education sector, encompassing a network of campuses, medical centers, and affiliated national laboratories. The central administrative office coordinates activities that allow this complex system to operate efficiently as a single entity, advancing its public interest, academic, and research missions.
With 3,300 users across various testing processes, the institution had been using Xray Test Management for over 7 years across 24 projects. The complexity and scale of this environment presented unique challenges during the migration to Xray Cloud.
The Challenge
The migration to Xray Cloud was driven by the need to reduce operational efforts, lower maintenance costs, and scale more efficiently, especially with the impending end of their server license. However, the migration presented several unique challenges:
- Data volume management: The testing ecosystem involved hundreds of thousands of test steps (431,994), results (502,301), and activities, which translated into an extremely large data set that had to be migrated (over 119,466 test runs were conducted). Managing such a large volume of data posed a significant challenge, especially ensuring that all this data was transferred accurately without loss or corruption.
- Cross-project dependencies: One of the most critical aspects of the migration was the need to retain cross-project dependencies. The company needed to migrate a large volume of data, including 24 Xray projects, 11 of which had critical cross-dependencies, which meant that all 11 projects needed to be migrated together in a single phase to maintain data integrity and relational links between Xray entities. A change or update in one project could have ripple effects across multiple other projects. During the migration to Xray Cloud, it was essential to move all associated projects simultaneously to ensure these dependencies were preserved.
- Ensuring data integrity and consistency: Given the complexity of the testing framework, maintaining data integrity and consistency during the migration was a major concern, especially because we needed to ensure that all connections between Xray entities remained intact in the cloud environment. The migration had to guarantee that all test steps, plans, and results were mapped and preserved in the new system. The team collaborated closely with Xray support and refined their migration strategy to achieve a successful outcome.
- Other applications: The environment was integrated with several other applications, such as Scriptrunner and JMWE, which were heavily customized with workflows and automations. These customizations needed to be managed carefully to ensure a smooth transition to Xray Cloud while preserving essential functionalities.
The Solution
To address these challenges effectively and ensure a successful migration to Xray Cloud, Xray's partner, Empyra, implemented a strategic and collaborative migration plan. The process was divided into 4 well-defined phases. The first 3 phases focused on migrating non-dependent Xray projects, while the final phase concentrated on the 11 projects with cross-dependencies. Recognizing the complexity and importance of this migration, the team took a systematic approach to minimize risks and ensure a seamless transition.
Step-by-Step Migration Process
1. Preparation and Evaluation
The company’s team began by evaluating the migration’s complexity, focusing on managing large data sets and maintaining the integrity of critical cross-project dependencies. This preparation phase involved detailed planning to account for the interconnected nature of the projects. To mitigate risks and anticipate potential issues, the team conducted 5 rounds of dry run migrations: essential to assess data compatibility, identifying potential problem areas, and refining the migration approach before the production cutover.
2. Test Migrations and Outcomes
During these initial dry runs, the team encountered some challenges with the data migration: in the early stages, some data did not migrate correctly to the cloud, and relationships between Xray entities needed further refinement to remain intact. These early observations highlighted the need for an adaptive approach. The team worked closely with Xray support to investigate these issues, gather detailed insights, and implement necessary adjustments. This collaboration helped ensure that minor issues were quickly addressed, and the migration strategy was refined for smoother execution.
3. Collaborative Support from Atlassian and Xray
Throughout the migration process, the company relied on strong partnerships with both Atlassian and Xray support teams. When partial migration issues arose, the team engaged with Xray support, who provided valuable assistance, expert guidance, and custom builds to resolve specific concerns. The continuous support and expertise from Xray were crucial in refining the migration process and ensuring a successful outcome.
4. Managing Production Cutover Challenges
The production cutover phase was critical and presented opportunities for agile problem-solving. The team anticipated potential hurdles and was prepared to manage them effectively. When some data did not fully migrate to the cloud, the team demonstrated adaptability by deleting existing projects in the cloud - a process that required careful management due to its long duration. They then renamed project keys on the server and performed a second round of migration, which allowed them to perform the migration without conflicts, resulting in a smooth process the second time around.
5. Post-Migration Optimization
Following the successful migration, the team focused on fine-tuning the environment in Xray Cloud to ensure all data was correctly integrated and optimized for performance. An extensive cleanup was conducted to confirm that no unmigrated projects remained in the cloud, maintaining data integrity and optimizing system functionality. This phase emphasized continuous improvement and collaboration between Empyra, Xray, and the client, highlighting the importance of thorough preparation, support, and adaptability in navigating complex projects.
The Results
The migration to Xray Cloud resulted in significant improvements in performance and efficiency for the client:
- Efficiency gains: The new cloud UI in Xray Cloud led to a 15-17% increase in overall efficiency for test management tasks compared to the server environment. In the cloud environment, this increase was even more pronounced at 30%.
- User satisfaction: Users reported a 16% increase in satisfaction scores after the migration to Xray Cloud, thanks to the improved user interface and faster navigation.
- Error reduction: The cloud UI's improved design and functionality reduced user errors by 5%.
- Performance improvements: The average load times for the Xray Cloud environment improved by 1 second per hit, equating to a 20% reduction in overall load times. Users acclimatized to the new cloud environment reported an 11% improvement in navigation and search efficiency compared to the server setup.
- Test execution throughput: The average time for test execution decreased by 2 minutes, an 18% reduction in time, resulting in an increase in test execution throughput from 28 to 32 executions per day.
These improvements significantly enhanced the client's testing processes in Xray Cloud, making the development cycle more efficient and less prone to errors.
Leveraging Xray Cloud for superior test execution and performance
The migration of the education institution’s extensive Xray data to Xray Cloud was both a challenging and highly successful endeavor. The complexities involved in handling large data volumes, maintaining cross-dependencies, and ensuring compatibility with various customizations required a well-thought-out strategy, proactive collaboration, and strong support from Xray. The team’s meticulous planning, multiple rounds of dry runs, and quick decision-making during the migration process demonstrated the importance of a flexible and positive approach.
The positive results post-migration - including improved performance, increased efficiency, and higher user satisfaction - underscore the benefits of migrating to Xray Cloud for large-scale organizations with complex environments. This case highlights how a combination of thorough preparation, strong partnerships, and a focus on solutions can lead to successful cloud migrations, enabling organizations to achieve a more streamlined and scalable testing environment.
About the Partner
Empyra is an Xray Partner and a full-service software products & services company. They are an ISO 27001:2013 certified organization with over 25 years of experience in Software Development and IT solutions.
"Customer First" is an integral part of their corporate DNA. As a Global Software Company with decades of experience, they are able to create close, meaningful relationships with their customers that give them insight into their unique needs and pain points. They evaluate short-term and long-term impacts to develop targeted, cost-effective solutions that optimize business processes and create lasting value.
