Exploratory testing is a dynamic approach to software testing that emphasizes your freedom and responsibility to adapt to changing testing conditions. However, without a clear direction or purpose, getting lost in testing without achieving meaningful results can be easy. This is where test charters come in. In this article, we will explore what test charters are and how to use them to optimize your exploratory testing process.
What is a test charter?
A test charter is like a mission statement for your exploratory testing session. It guides your testing activities while giving you room to be creative.
Test charters are often used in session-based testing to help you focus on testing in a specific area and within a certain time frame.
To create a test charter, ask yourself:
- What's my main goal for this session?
- Do I want to learn more about a specific area or get an overview?
- Will I be using any tools or heuristics?
- Will I focus on certain quality criteria or positive/negative scenarios?
Why do test charters make exploratory testing more effective?
Exploratory Testing differs from other testing approaches because it doesn't follow predefined steps or check for well-known expectations. You don't test randomly, either. Instead, you use your experience and knowledge to uncover new information.
Using test charters during exploratory sessions helps you stay focused and inspired, resulting in better outcomes. By defining a specific area of the system to test and setting clear expectations for what you hope to achieve, you are less likely to get sidetracked or waste time on testing that doesn't contribute to your overall goals.
Test charters also encourage creativity and innovation by inspiring you to test from different angles and explore the software in diverse ways. This can lead to more varied and comprehensive testing scenarios, which may uncover issues that otherwise can be missed.
How to write test charters?
Writing effective test charters is essential as it can impact the quality of your exploration.
A charter structure that works for most teams is the following:
Explore <area, feature, risk>
with/using <resources, restrictions, heuristics, dependencies, tools>
to discover <information>
Adapted from Maaret Pyhäjärvi, Elisabeth Hendrickson
A common mistake you should avoid while writing a charter is overcomplicating it. Try not to get too specific, otherwise, you'll write a test case. On the other hand, you should also avoid getting too broad because you will lose focus.
A perfect charter inspires you in your exploration without forcing specific actions or outcomes.
Explore the login page
Using Chrome and different screen sizes
To discover problems with usability
How to get inspired while writing test charters?
Getting stuck in your day-to-day routine may make you feel uninspired to write charters and have difficulty thinking outside the box. Luckily, experts in the testing industry have developed approaches and techniques that can help you test from new angles and find unnoticed defects:
1. Explore using personas
User personas help you explore your system by considering different user behaviors, which leads to finding product issues associated with them. Your user personas can contain information about age, job, accessibility, usage habits, and feelings.
Here are some examples of user personas that can you can consider:
- Hacker Dave: Dave wants to break things and find system weaknesses. He is looking for security loopholes and will use malicious techniques to steal data or get access to unauthorized areas;
- Senior Adam: Old age Adam has vision impairment and often zooms in to see better. He uses an old browser version and is unfamiliar with the software. He will take unpredictable actions to reach the final destination and abandon if he cannot complete the task;
- Traveler Ana: Ana is a digital nomad, changing her work location monthly. She often changes time zones, network providers, keyboard layouts, or even devices. Sometimes, Ana is in areas with poor network connectivity or connects to the local public network.
A charter example that is based on a persona:
"Explore the bookstore site
With a mid-age, non-technical teacher persona perspective
To discover problems around the easiness of the whole process of buying books"
2. Try test tours
Tours are metaphors for testing activities linked with themed journeys that users may take. When you're testing using a tour, you navigate the system focusing on finding defects linked to a theme. James Whittaker uses a tourism metaphor that organizes tours in different themed districts (business district, entertainment district, tourist district, and more).
Here are some of the tours that James Whittaker is proposing:
- Obsessive-Compulsive tour: This tour aims to repeat the same action as often as possible. This tour can reveal bugs like applying the discount twice;
- Bad-Neighborhood tour: Explore the area or feature that has the most errors. In this tour, you spend time on areas known to have issues;
- Supermodel tour: This tour focuses only on surface looks and the impressions it makes. Test the UI interface and design elements like layouts, icons, flows, and colors;
- Museum tour: Test code that developers didn't change in a while. You can experiment with how the code functions in new environments (Android, iOS, etc.).
A charter that uses the supermodel tour can look like the following:
"Explore the login page
Using Chrome, Firefox, Edge, and Opera
To discover problems with CSS styling and JavaScript"
3. Experiment with heuristics
Test heuristics are cognitive shortcuts that facilitate the testing process by helping you to solve problems or make decisions. They can have different forms, such as checklists and mnemonics.
- CRUD (Create, Read, Update, Delete): Test an area using the essential actions a user can take.
- Beginning, middle, and end: Change the element position in a sequence.
- Zero, One, Many (Zero things, one thing, many things): Look out for performance issues as the number of items increases.
- Starve: Limit access to required resources like CPU, memory, network, or space.
Here is a charter example that uses the CRUD heuristics:
"Explore the REST API endpoint for managing bookings and the CRUD operations
-
-
- POST
-
-
-
- GET
-
-
-
- PUT
-
-
-
- DELETE
-
Using the "curl" command line tool and the REST API specification
To discover problems with the syntax"
How to use test charters with Xray Exploratory App?
We created the Xray Exploratory App to help you focus on your exploration session and ease the documentation task by easily recording your findings.
1. Define your charter
Start your exploration session by writing down your mission: the charter. Include in your charter the quality attributes you aim to test and add lists of items you know in advance you want to cover. Remember that a charter needs to inspire you and keep you focused, so try not to get too specific.
2. Start your session and gather evidence
While you explore your system, use the app to record evidence and videos, take screenshots, and mark defects directly in Jira. When you feel you're losing focus, return to the charter and remember the goal for your session. Use the notes to write down ideas for new charters.
3. Review your findings
At the end of your session, export the PDF report of your session or automatically upload the results to Jira and Xray. Once there is visibility over your exploratory session, discuss the charter with your team. Charters can be reused and will generate different findings when used by multiple members.
Learn how to generate a PDF report of your exploratory testing session using the Xray Exploratory App:
Start developing your charter library to stay focused and inspired
Test charters are an important element of exploratory testing as they help you stay focused and inspired in your exploratory testing sessions. Writing charters for exploratory testing is a continuous process that starts with the first discussion of requirements and evolves with each session. To keep your sessions focused and ensure better test coverage, it's helpful to create a charter library. If you feel tempted to explore in ways that are clearly off-charter, it's a good sign that you need to update your charter library.
