Exploratory testing has become a more common topic among testers. Exploratory testing provides valuable quality-related insights that otherwise would escape unnoticed.
Unfortunately, there’s still a long way to go until everyone can unlock the full potential exploratory testing. Not everyone has heard about it, not everyone understands exactly what it is, and not everyone has taken the most out of it.
In this article, we will address a few misconceptions, try to clarify them and, if possible, see how to overcome them.
1. Exploratory Testing is just ad hoc testing
There’s this idea that exploratory testing is ad hoc testing in the sense that this way of testing is made without any upfront planning. A way of rephrasing that would be to say that an ad hoc process is about dealing with things as they happen.
It’s true that in exploratory testing, we don’t have detailed upfront planning in terms of the exact steps that we’ll make during testing. However, we usually do some sort of high-level planning where we decide the scope and resources for our testing.
It’s also true that during exploratory testing, we learn continuously and use that to drive our testing, depending on the feedback we get at the moment we get it.
2. Exploratory testing is random testing
One major misconception is that exploratory testing is about testing randomly; this is completely untrue.
We don’t test without a “method” or without making conscientious decisions. In exploratory testing, every single decision is driven by our knowledge of the product, its context, and the feedback we get whenever exploring or trying it out. We also start our test by having in mind a goal, some risk(s), and a test charter. Therefore, exploratory testing is driven by knowledge and not randomly performed.
To uncover important information about the systems that we target by testing, we need to use testing skills and techniques wisely. We also need to consider our knowledge about the product, its users, the business context, etc. All this drives our initial testing decisions but also the decisions we make during testing, depending on what we experience.
3. Exploratory testing is a testing technique
Framing exploratory testing as a testing technique depends on what we understand by testing technique.
If we look at it as a way of performing testing that requires skills, that is true. If we say that it is a procedure to complete a specific task, then it becomes trickier as there’s no specific task to achieve in exploratory testing (unless the task is seen as finding relevant information about the quality of the target system).
Known testing techniques (such as BVA (boundary-value analysis), pairwise testing, and many more) exist to address a very specific goal (like generating test cases with combinations for all pairs of variables).
We can use several testing techniques during exploratory testing, depending on what we see as the best fit at a given moment. That is a decision of the tester based on knowledge, skills, and findings while testing.
4. Exploratory testing is an alternative to "manual" testing and "automated testing"
Some people see exploratory testing as an alternative to “manual” and “automated testing.” There’s some discussion about what those mean exactly, but for many people, it represents manual scripted test cases and automated test scripts, respectively.
Exploratory testing is an approach to testing, perhaps its deep nature, where we test, learn, and so on. All of that during a time period, no matter if we call it a session or something else.
While testing, we use our eyes, our hands, our ears,
all our senses, and most importantly, our brain.
We may be assisted by tools, including automated test scripts running in the CI or even run manually.
Exploratory testing is more of an alternative to traditional scripted testing. It’s about exploring the software with our “eyes wide open”, and our brain fully active.
5. In exploratory testing, we don't use tools
If people see exploratory testing as something “manual,” as explained earlier, then the context for tools is not clear.
However, tools are always present during testing activities. Testers use tools to obtain, record, and share information. Tools can also be used to augment testing. Therefore, it’s always relevant to be up to date with tools and assess how they can be beneficial or not.
Overall, testers use tools to:
- Obtain information (e.g., context in the form of documents or other);
- Collect/Take and record evidence in the form of text notes, mind maps, screenshots, videos, logs;
- Create defects;
- Link findings with some existing story or requirement;
- Share findings with the team;
- Create ideas for new test charters or automated test scripts;
- Perform types of testing that otherwise would be hard or impossible (e.g., load testing);
- Make reports.
While some of these are used during actual testing, some are used before and after it as part of the overall testing process.
6. Exploratory testing doesn't require documentation
To explore, we don’t need documents. That’s true. But does that mean that there’s no documentation involved in exploratory testing? Not exactly.
Documentation exists as a source of information. It can exist in stories, formal requirements, wikis, comments, emails, etc. Existing documentation provides context; we should use it critically as we know that it may be incomplete, inaccurate, outdated, contradictory, etc. We can explore without that, and that also provides a useful perspective, but the documentation has information that we cannot ignore.
During exploratory testing, we also document our findings, so we can share it with the team while at the same time it also serves the purpose of evidence.
Therefore, even though we don’t spend a lot of time writing or reading test cases with exploratory testing, there’s some level of documentation involved. The good news in this type of testing is that we spend most of our time actually testing, in its true meaning: exploring the product and finding issues/gaps in it that affect the value perceived by the different stakeholders.
7. Anyone can perform (good) exploratory testing
Anyone can draw, right? But what about producing artworks?
Well, anyone can test; we all test and explore the world on a daily basis. However, given some systems and constraints (e.g., time, budget, resources), can we all provide valuable insights about quality that can eventually support the team to make the product better?
In the same way that many can write code, but only a few of those can write code with good quality, the same happens with testing. We need a bunch of different skills, knowledge, tools, and experience. In exploratory testing, we are not driven by the hardcoded knowledge on detailed test cases, which ignores the real-time findings we get while testing.
As we test systems, the feedback we get supports our next steps. We decide what we do in a given moment and in the next moment based on our skills and what the product hints to us. Only that way can we expose important risks or gaps that would escape otherwise.
Investing in learning and experiencing tools and systems is key to improving the value we get from exploratory testing.
8. Exploratory testing can’t be used together with Test Management tools
Some see exploratory testing as a way of performing testing where testers perform it totally “freestyle”, taking notes if they want, the way they want, in a very light way. Sometimes, the idea of disconnecting it totally from issue trackers and test management tools also comes to mind.
The fact is that most people use issue trackers, such as Jira, to manage their projects and related issues (e.g., stories, defects, tasks). Some also use test management tools to create and manage test cases, track testing results along with coverage, track and trace defects, make quality-related reports, etc.
Even though you don’t need test management tools for exploratory testing, it can provide some benefits, especially if your team is already adopting one.
As an example, with the Xray Exploratory App you can upload the findings of your exploratory testing sessions to Jira and Xray. Thus, you can track your test results in one place (Jira), no matter how those results were obtained (either from manual scripted test cases, test automation, or from exploratory testing). Additionally, you can combine these results and use them to assess the coverage status of your stories or requirements, which in the end, are the deliverables whose quality you’re concerned with.
Test management tools can also be used to plan the charters of your exploratory tests in a way that you can have an idea about the features/areas/risks that you plan to address, assign them for execution, report to them, link defects to them, and share with the rest of the team.
