Game Developer uses Xray for PlayStation, Xbox and Nintendo - Xray Blog

“Xray is used in-house at Abstraction to manage all test cases for PlayStation, Xbox, Nintendo and Stadia. Gone are the days of trying to manage technical requirements in an excel sheet. One of the best investments we've made for testing and I'm happy I got this added to improve our test case management processes.”

Mark Pittam, QA Lead at Abstraction

About the Company

Abstraction is a leading game development studio that has delivered over 200+ products across 60 games over the last 13 years. They have more than 80 employees based all around the world and are renowned for taking on technically challenging projects for some well-known customers including Funcom. 

The studio works with some of the biggest names in the industry including PS4 & PS5, Xbox One|X, Xbox Series X|S, Nintendo Switch, Google Stadia, PC, and Apple.

The Challenge

“With Google Sheets and Excel, we could not share decent results with the clients, internally or with management, about how well a product had been tested.”

Mark Pittam, QA Lead at Abstraction

Prior to Xray, the studio used Google Sheets and Excel to manage all of their testing, including functional and technical tests. This was an unreliable system that left their team overwhelmed with information, lacked any traceability, and most of all proper reporting. For them, using Excel and Google Sheets was a low-cost solution and they had easy access to it. 

However, with Excel and Google, they faced many challenges including: 

• Poor reusability;
• Task management;
• Easy to lose focus due to information overload;
• Very poor at tracking & sharing results;
• Everyone hates working in spreadsheets.
  
  

Abstraction needed a powerful test management tool that would centralize their testing, organize their multiple projects and tests, and provide them with powerful reporting so they can easily communicate the results of testing to key stakeholders. 

*Want to see this success case in audio or video? Watch the presentation as a webinar. 

The Solution 

“Xray allows us to give managers and producers quicker at-glance reporting, to know how many tests are failed and passing. Now we’re able to show the client the current state of the product and compare the testing against the platform holders’ own internal QA teams.”

Mark Pittam, QA Lead at Abstraction

Abstraction needed a new tool to help them organize their complex testing and optimize their test management. They had a strict selection process and they wanted a tool that had the ability to: 

• Discard old practices of working in Excel or Google Sheets;
• Manage and organize a large number of detailed test cases for each platform;
• Edit test cases on the fly while test executions are being run;
• Easily read and follow Test Steps;
• Present all/current test results in an easy to understand format;
• Improve chances of the product passing submission the 1st time;
• Boost overall confidence & knowledge in the QA team;
• Make management and the client happy;
• Provide reporting that is quick and easy to understand.

Xray matched all of their requirements and provided them with a powerful test management solution that would meet their growing needs.

The Results

“With Xray, it’s more efficient for us to do testing certification. The new process allows us to react much quicker, we can create the defects directly from bugs,  give the feedback to developers, and turn things around much quicker.”

Mark Pittam, QA Lead at Abstraction

Favorite and most used Xray features

Since adopting Xray, Abstraction has learned to rely on the tool for a smooth and efficient test management process. Some of their favorite and most used features include: 

1. Xray Academy to onboard new members: For Abstraction, the Xray Academy is the perfect introduction to Xray and they use it as an onboarding tool for current and new team members;
   
   
2. Test Repository to easily organize tests: Abstraction prefers to use the Test Repository which is a folder-like structure that helps them efficiently organize their multiple projects for different clients like PlayStation, Xbox, and Nintendo;
   
   
3. Native Integration with Jira for better collaboration: With Xray integrated into Jira, the issue management platform, developers at Abstraction now create ad-hoc tests from bugs reported by QA. This builds upon a Quality culture and fuels a quality first mentality;
   
   
4. Parameterized Tests to test large data sets: Abstraction takes advantage of test parameterization and data-driven testing to test large amounts of data. They use data sets to generate iterations that cover many different values such as weapons, clothing, skills, options, etc.
   
   

Takeaways and benefits of using Xray

“The ease of use is very important to us. We want people to come in from any skill sets, producers, managers, QA, designers, and quickly get up to speed. Using the Xray Academy we are able to quickly onboard new team members.” 

Mark Pittam, QA Lead at Abstraction

Abstraction has found clear benefits that went above and beyond their expectations. 

3 main takeaways from Xray are: 

1. Ease of use where any team member can use and work with the tool;
2. Low levels of maintenance especially on Xray Cloud;
3. Cost-effectiveness especially compared to other test management tools.

4 benefits of Xray for Abstraction are: 

1. Everyone in the company has visibility into test runs and overall testing progress; 
2. Better estimates for clients on how long certification testing can/will take;
3. Improved overall certification knowledge amongst all project team members; 
4. Higher retention rates of clients for future projects.

From Excel to Xray, test management just got easier

Want to see this success case as a webinar? We recently had a webinar with Mark Pittam from Abstraction who shared how and why they’ve been using Xray for test management. 

You can watch the complete presentation and a valuable Q&A section where Mark answered many questions that you might be curious about too.