Software testability measures how simple it is to test both the system as a whole and each individual component. Testability might mean different things for different members of the software development lifecycle.
Remember how deeply frustrating it was to rewrite dozens of UI tests when a front-end developer made a small change that broke all the locators? Or the several restless nights spent fixing minor performance issues that aren't reproducible on the local machine? What if we told you these issues might be resolved quickly by keeping testability in mind?
Everybody in the software development lifecycle performs testing at some level. Developers test on the unit test level and a bit above. Managers or Product Owners perform acceptance testing from time to time. Users test your software by using it and subjecting it to conditions you can't envision.
In this blog post, you’ll learn how developers, testers, and managers view software testability and how each team member can benefit from adding more testability into the software development lifecycle.
Testability in software testing. Why do testers need testability?
Tim is the tester on your team. Daily, he spends time on the maintenance of a pretty extensive UI automation test suite. At the same time, he needs to take care of the new features produced by his fellow developer - Dave.
The test suite keeps expanding, becoming flakier due to the high priority given to new features. Additionally, he spends hours setting up environments and testing the system in the ideal condition.
Software testing issues and how can testability help testers?
- Problem 1
The main reason for UI test flakiness for the tester is fragile locators and never-ending refactoring of the frontend part.
Testability Tip
The developer can add unique automation-related properties for each element used in the tests. For example, an element can look like <div id="login" automation-id="login-btn"/>
The tester, after all, can create such properties by himself by contributing to the frontend repository directly. (without bothering the developer).
- Problem 2
Tim, the tester, spends hours setting up the environment and test data for an entire test suite of automated tests. Each precondition involves numerous difficult processes that are performed through a UI.
Testability Tip
Tim and Dave can create an API that allows creating the test data by making HTTP REST calls to the backend. Dave can also develop reusable SQL scripts that add generic test data before all testing activities begin.
- Problem 3
Tim needs to perform a set of repetitive actions each time he executes a test (E.g., it can be playing the whole level from the start until the final boss, spinning wheels in the slot machine, generating transactions in a chain of upstream systems to get results). These procedures take time, are monotonous, and are prone to human errors.
Testability Tip
Tim and Dave can create a tool (or a service) that allows them to modify the system state - at the end of the level or a complex end-to-end flow. Tim can almost instantly put a system in the desired state using the tool and do exploratory testing for a particular feature.
Developer testing and the need for testability
The developer, Dave, puts in a lot of effort. He writes design documents, discusses needs with business analysts, works on new features, and makes code improvements. However, making reasonable estimations and setting up a reliable delivery schedule is impossible because of production issues. Dave needs to spend a fair amount of time investigating and fixing those nasty bugs that make users unhappy.
How can testability help the developer solve these problems?
- Problem 1
A part of the investigation is searching for the root cause of the issues using logs. The problem is that while certain system components offer a ton of logs, others only write logs when there is a serious internal malfunction (otherwise, such logs are empty). Dave can’t configure the log level and can’t distinguish good logs from bad ones. Thus, he spends hours and hours scrolling through the walls of text.
Testability Tips
During the next sprint, Dave can add more precise logging that allows seeing the whole path of the data through the system at each point in time.
He can add the ability to configure log levels easily without the need to restart each service or the entire system manually.
Dave also can examine how easy and understandable it is to use production logging (for him and other engineers) and explore other options. Additionally - he can teach Tim how to use logs effectively and gather only those logs that matter.
- Problem 2
To fix production, Dave needs to reproduce it, ideally - locally. But he can’t do it because many dependencies, like databases and message queues, are tightly coupled with implementation. It is tough for him to instantiate a small database copy on his local machine. So, the investigation leads him to debug on production (or pre-productions) and to play guessing games to find the root cause of the symptoms.
Testability Tips
Dave can invest his time into creating a toolset for testing the system (or a single service). Maybe he can put dependencies into Docker containers and start along with the system. Or he can write instrumentation code that can instantiate a part of the system in a very isolated environment. As a result, Dave will be able to isolate and investigate such issues really fast - so the fix won’t take too long to be delivered to production.
Why does management need testability?
Myroslava, the manager, always wants to build features with a reliable and happy team. The team provides results with a good level of quality and within a predictable time frame.
But she observes that Dave, the developer, is struggling with investigating and fixing issues. Tim also has a hard time with untestable software and unreliable test results.
How can testability help the manager solve her problems?
- Problem 1
The team puts a lot of time and attention into analyzing and fixing production-related issues. It consequently costs money and the company's reputation.
Testability tip
Myroslava can insist on including testability tasks in each feature that the team is working on. She can ask the tester and the developer to provide feedback on how easy to test features during the development process or maintenance in production. Such questions or tasks can be included even as a part of the acceptance criteria for the feature.
- Problem 2
Tim, the tester, wastes too much time attempting to repair UI tests that keep failing. Dave and other team members can’t rely on such test results. In both situations, the development process takes more time to deal with new problems than it actually does to produce anything.
Testability tip
Myroslava can encourage the team to work on software testability and improve everyone's lives and procedures by making it easier to add locators or simple-to-use testing APIs to an application.
Additionally, she can explain to higher management the necessity of the team devoting its valuable time to testability tasks. As a result, Myroslava and her team can create quick and predictable results thanks to predictable and manageable tools. This leads to a motivated and content staff.
Why does software testability matter?
In a nutshell, testability's core objective is to make your testing easier. Depending on who conducts the testing, it can take on a variety of shapes.
Investing in software testability takes time and effort because it is a continuous process, just like creating a strong engineering culture. Whether you're a manager, developer, or tester, testability helps everyone to obtain crucial information more quickly.
This knowledge aids in risk management and problem-solving for code or processes-related issues. Good testability leads to a happy team, manager, and users.
