Although test automation is the biggest trend in software testing right now, only focusing your strategy on automation isn’t going to guarantee you a fool-proof QA process.
By diversifying your testing strategy with different methods, you’ll be able to cover more ground (i.e. untested code) and more unexpected discrepancies in your code and product.
Exploratory Testing should be a part of your testing strategy because it will test the effectiveness of your existing tests, discover code discrepancies, and alleviate bottlenecks where bugs hide the most.
In this post, QA Software Engineer, Pekka Pönkänen, tells us how he effectively performs Exploratory Testing and his advice on making the most of your sessions.
Why Exploratory Testing is important
In 2018 I walked the length of Japan, and one of my goals for the trip was to explore the country and its rich culture. During the trip, I decided to walk in different routes other than just following Google Maps, and as a result of that, I had many exciting adventures. Undoubtedly, without having the curiosity to explore, I would have missed many unique places and interesting conversations with locals.
I firmly believe that in order to find new ways of seeing and experiencing the world, you need to be brave and curious about the unknown. The same goes for software testing. Always following the same paths, or tests, will get you expected results. On the contrary, if you go on an exploratory session of your application, you’ll be amazed to discover just how many bugs could be hiding or how much you can improve the functionality for the user.
If you want to try Exploratory Testing yourself, consider this your starting point.
What is Exploratory Testing?
The term "Exploratory Testing" means that you are exploring the application and how it performs after different actions.
To truly understand the concept, it is essential to dig deeper into the roots of the terminology. The term “Exploratory Testing” was introduced in 1984 by Cem Kaner.
"Exploratory software testing is a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the value of her work..."
James Bach's 2003 paper, "Exploratory Testing Explained"
Through exploratory testing, your goal is to find out how a specific application area works while using your skill set to provide valuable feedback to your team. You want to find the nastiest bugs under the hood and ensure that nothing critical or dangerous can happen in the app.
Preparing for Exploratory Testing
Before starting the testing run, it's good to have tools to write down ideas, bugs, and defects. I like classic pen and paper, and bug-tracking software to write down thoughts and plans for the future. During the run, you will see application logs, automation ideas, new approaches for new testing runs, and bugs that need to be fixed. Good software that captures images, videos, and notes can be very helpful in centralizing your findings and sharing your insights with your team.
Keep in mind that your notes do not need to be the most delicate piece of art. What's important is that you can gather a small story around it for your teammates or stakeholders after the session. I have heard very successful use cases with testers doing mind mapping during exploratory testing. This visual aid in the process can be valuable for building on your ideas and creating themes.
Identify your goal
The first thing to start in exploratory testing is to define what to test. It can be a known bottleneck, a possible risk, a new feature, or an area with many bugs.
As a software tester, you may know the places to look, and the development team can define the areas which need more attention. When planning the testing, remember not to make it broad so you don’t lose focus. Here are a few examples to get you started:
What to do:
- Explore catalog page with a screen reader to verify page accessibility;
- Explore login process with iOS gestures to verify that functionality is accessible.
These tasks can be timeboxed and can be done in one session and short enough to be focused.
On the other hand, you don't want to choose a task that has too many options or paths to follow. The best way is to keep it simple and focus on smaller tasks.
What not to do:
- Explore all possible mobile security issues to the system to discover any security-related threat.
Once you’ve identified your goal and prepared your tools, here are the steps you can follow to complete an Exploratory Testing session:
1. Prepare your session
After selecting an area to explore, design the test session. Once you know what to explore, you’ll get a flood of ideas about how to test different aspects - keep track of everything that comes to mind, so you have a solid plan before you start testing. Write down your mission and prepare the notes about the way you want to proceed.
2. Setup the testing environment
Check that you have all credentials and access to enter the testing environment. Testing is pleasurable when you can focus on the testing itself, and when you don't need to worry about usernames or unreachable servers.
3. Timebox and execute
Depending on the task, open the suitable logs, and monitor exploratory testing tools to log your actions during testing. Application logs are crucial to provide valuable information when things go wrong. While executing, keep your objective in mind, write notes, be systematic, collect info, gather ideas for the next sessions, and, most importantly learn and explore your product.
Exploratory testing is more of a mindset than a framework
Exploratory testing is a skill that is always developing while your skillset as a tester is evolving. The beauty of testing is that you never get bored because there is always room for improvement. Be curious about the application, try different approaches to execute testing, learn about your software, and share the knowledge.
At the end of the day, software development is a team sport!
Inform others about the state, risks, and any other concerns. Share your mind map and any evidence you collected like videos, images, and notes. You don’t have to have all the answers but sharing them with your team will help others discover themes and insights that you wouldn’t have known otherwise.
