Guidelines
Tree Testing
Assess the logic of your user journey and build user friendly navigation and information categories, both crucial for adoption
Image by Storyset from Freepik
Test Phases
Time: 2 days
Planning
- Defining the scope and purpose
- Decision on location and equipment/ Testing approach
- Creating the information tree
- Scheduling & Recruiting
Time: 1 day
Testing
- Assigning tasks
- Motivating users to share their thoughts (if moderated)
- Taking notes (if moderated)
Time: 1 day
Analyzing and Reporting
- Clean up the data & enrich with notes
- Identify patterns of processes or problems across the testers
- Prioritize problems discovered and share report
Planning
2 days
- Defining the scope and purpose
- Decision on location and equipment/ Testing approach
- Creating the information tree
- Scheduling & Recruiting
Testing
1 day
- Assigning tasks
- Motivating users to share their thoughts (if moderated)
- Taking notes (if moderated)
Analyzing and Reporting
1 day
- Clean up the data & enrich with notes
- Identify patterns of processes or problems across the testers
- Prioritize problems discovered and share report
Study Design
Read here about the different options for your study – your research question should always guide you in chosing the right format, approach, and setting!
Where Are You Testing?
Do you need to sit with the users and observe them? Or do you prefer to get behavioral insights from a lot of different, remote testing users?
In Person Testing
- You meet your users physically for the testing – e.g. in your office/meeting room or where your users are
- Is always moderated testing
Why it’s useful
A moderator can observe and record the user’s body language, gestures, and non-verbal cues
good for testing with people that have low access to internet/ digital skills
Potential challenges
requires more time, logistics and budget (also for compensation payments)
timelines need to meet users' availability
How Are You Testing?
Do you need to sit with the users and observe them? Or do you prefer to get behavioral insights from a lot of different, remote testing users?
Moderated Testing
- A real person facilitates (moderates) the testing, either physically or virtually
- Can be done remotely or in person
Why it’s useful
The moderator can ask individual follow-up questions
The moderator can guide and support users during the testing (e.g. with a complex feature)
The moderator can observe body language and non-verbal cues
Potential challenges
Moderator might introduce bias
More time (logistics) and budget needed
Testing
In this section, we explain how to collect data through Tree Testing.
Data Collection
Example Tasks
Your tasks should address key website/App goals and user behavior as well as potentially problematic areas of your design.
Interesting tasks could be:
- An online store could ask users to find how to add a new payment method
- A digital agriculture App provider may request users to navigate to the settings of their App and change their language settings
- An IVR extension service might ask users to navigate to a specific topic they seek help on or to record an information request
Example Metrics
- Success rate: the percentage of users that completed the task
- Directness: the percentage of participants who completed the task without hesitation and got the correct answer the first time
- Time: the time it took participants to finish a task
- Path: the routes participants took up and down the tree before selecting an answer
How to implement your study
An information tree prepared in Excel...
...and ready for testing in a testing tool.
Data Aggregation
In analyzing Tree Testing results, below are some helpful questions. Use the answers to validate your hypothesis about your current design
- Where did participants think they’d find your content?
- Did they find the navigation or wording confusing?
- What paths did they use first?
- Which was the place where they backed up to try a different path?
- How long did it take them?
Best Practices
Find here best practice examples with helpful tips and tricks.
Do’s and Don’ts
Do’s
- Make it easy: Tree Testing can be done using index cards, just like paper prototyping
- For each task you write, you should define the right answers that correspond to where the information is located within the tree, so you can automatically calculate success rates for each task
- Make sure all different aspects of your users are represented among your sample of testers
- Do a pilot test and practice with your team.
- You can run split testing with two different trees and compare the results (see A/B tests LINK)
- You can follow-up the tree test with survey questions to collect more behavioral insights
Don’ts
- Don't overwhelm your testers: One session should not go longer than 20 minutes and include max 10 tasks
- Don't only map the part of the tree that you want to test. Excluding the other sections is risky because it assumes that users will know which section to go to
- In the formulation of your tasks, prevent priming: the task instructions should avoid using terms that give away the answers
Potential Bias To Be Aware Of
Find a detailed overview of potential biases with counter actions here. Below a list of potential bias to be aware of when conducting Tree Tests.
Image by Storyset from Freepik
People tend to give more weight to their most recent experiences. They form new opinions biased towards the latest news, e.g. by focusing only on the problems found in the latest usability session
Image by Storyset from Freepik
Image by Storyset from Freepik
False consensus is the assumption that other people will think the same way as you. May lead to assuming own logic chains and preferences guide users as well.