The impact of model-based test case design on automation - Xray Blog

Research has shown that 35% of companies find manual software testing to be the most time-consuming part of the software development life cycle. 

Because manual testing is so time consuming, test automation has been growing as a solid way to improve ROI. From reduced tester fatigue and greater productivity to a better product and faster time to market – building a robust test automation framework can add numerous benefits to your team. 

However, selecting a proper automation approach for your organization can be daunting. A lot of analysis is often focused on the automation itself and the downstream, but, from our perspective, optimizing earlier steps in the testing process can significantly simplify the journey. 

In this blog post, we’ll show you how to use  model-based test case design (demonstrated by Xray Enterprise’s Test Case Designer) as a way to reduce the overhead of test case creation and automate your efforts.

Challenges of adopting Test Automation 

Some of the common barriers to adopting more test automation are:

Choosing the “right” strategy

Lack of a clearly defined implementation strategy is one of the biggest challenges in test automation today. It includes both “how” and “what” aspects:

“How” - There are plenty of frameworks, languages, and tools for all kinds of automation goals. And an important angle to keep in mind is how they will interact with other areas of your business/SDLC process - e.g. how they will generate useful test reports and metrics for all stakeholders.

“What” - Prioritizing your automation targets should happen with repeatability in mind. Will this automation test suite have a long lifetime? What are the expected volume and execution frequency? But between massive manual regression suites and continuous feature development, picking the highest ROI candidates is not easy. 

And if you decide wrong, a strategy pivot can significantly damage your productivity.

Implementation and maintenance effort

Scripting automated tests is often time-consuming and expensive, you need skilled automation engineers. Furthermore, traditional test automation becomes expensive to maintain. With frequent requirements changes, having a script or a recording per manual test case becomes impossible to manage and scale.

Visibility and collaboration

Effective, transparent communication across teams is crucial, but it is more complicated in test automation than in manual testing. 

Business analysts and project managers should participate in discussions about the test cases that need to be automated and the necessary functionality scope. But they are typically not well versed in coding languages or backend syntax. So, reviewing automation scripts one-by-one is a non-starter. Creating a “mirrored” set of readable documentation adds overhead, and, if you strive towards in-sprint automation, you shouldn’t always rely on the manual tests as a “collaboration backup”.

How model-based test design helps break the barriers 

To be on the same page, we will use this definition:

Model-based test design is a technique where, instead of writing test cases for each acceptance criteria in isolation, we first create an artifact that holistically represents the behavior of the system-under-test (or its part relevant to the current project). Then, we derive a more comprehensive test suite from that artifact.

Model-based design helps with the adoption barriers 2 & 3 mentioned above, as it:

• Simplifies maintenance because model adjustments are centralized which leaves little room for ambiguity about individual script changes. As an added bonus, if you need to switch frameworks/languages, your models would still be valid;
  
  
  
• Makes collaboration easier since reviewing a model artifact (e.g., a mind map, a decision table) does not require special technical skills. That, in turn, accelerates the overall process and increases testing accuracy; 
  
  
  
• Is often associated with data-driven automation (but is not exclusive to it). If the interest in model-based design prompts you to adopt data-driven automation, you no longer need to create and maintain an individual script for each test case, which is typically a time-consuming process. 

However, implementing model-based design without intelligent tools will limit your potential and just shift some of the barriers to a different stage - manual creation and maintenance of models can be only marginally less cumbersome than those of individual scripts.

We can find a better alternative to manual model management. For example, with the feature from Xray Enterprise - Test Case Designer (TCD) users find: 

Let’s see how optimized data-driven automation can be accomplished with model-based testing and TCD. At a high level, steps are similar regardless of the target system or the automation framework:

1. Create the setup in Jira with the requirement story and the Xray Test Plan.
2. Create a model for the target system in TCD.

a. You can add the model URL to the requirement story;

b. Model-based testing is often considered abstract, but it can absolutely create fully executable test cases with the “right” value names;

3. Export the data table as CSV from the Scenarios screen in TCD;
4. Create the data-driven execution script in the selected framework/language, utilizing CSV iterators;
5. Run that code and generate the report format that is supported by Xray;
   a. Optionally, you can configure the level of evidence in the report and add links to the requirement and/or test plan from step 1;
6. Import the report file via REST API (directly or through a CI/CD tool);
7. Enjoy full visibility of the results in Xray.
   
   

As a bonus, if at any point you need to switch to the “traditional” script-based approach, you can easily do so by leveraging the data-driven BDD/Robot Framework templates under TCD Scripts. Create 1 Scenario/Test Case block, export or sync it to as many issues as you have iterations. You do not need to rebuild your models, and you still benefit from all 4 points above.

For more implementation details, please see this tutorial.

Accelerate automation with Test Case Design

Nowadays, companies feel more pressure to optimize their time-to-market metrics, maximize resource utilization, and scale their team’s productivity - all at the same time. Test automation therefore is expected to evolve beyond “faster regression.” It should help companies achieve these business goals by striking the balance between speeding up the entire product delivery process and maintaining quality.

Model-based test design, if done with the right tools (like TCD), can be part of the solution to accelerate automation, helping reduce costs by optimizing efficiency and quality:

On top of that, QA teams should use a robust test management and reporting solution like Xray in order to achieve faster feedback and enhance the overall productivity.

Outpace the competition and improve your automation efficiency by getting Xray Enterprise and accessing all the features from Test Case Designer.