The Hidden Dangers of Tree Testing

Tree testing and card sorting are often the best types of research for content strategists. We love seeing what our audience thinks about terminology, hierarchy, and categorization. But sadly, I’ve been in many a research phase where we looked at the results and had more questions than answered. “Was that a problem because we used the wrong words, or put them in the wrong places?” or even “You thought that label meant that? I always use that term for this… what were our participants using it to mean?”

This is the hidden danger in content research: confusing the search for terminology with the search for organization.

What is Card Sorting and Tree Testing?

If you’re already familiar with the basics of these activities, go ahead and skip this section. But for the uninitiated, here’s a brief introduction.

Card Sorts

A card sort is a type of user research involving 20-30 digital or physical cards. Each card has a word written on it. Participants (either as individuals or in groups) sort the terms into categories. This shows the researchers what items participants see as related, or would expect to find together.

In a closed card sort, the moderator also provides the categories. In an open card sort, participants create their own category titles. This way card sorts are useful in creating an information architecture or sitemap.

Tree Tests

In tree testing, the moderator gives participants a task to complete. The participant sees a “tree” – the top level of a sitemap. The participant then clicks down through the sitemap until they find the page or section where they believe they would be able to complete the task. With tracking software like Optimal Workshop, researchers can see what percentage of people went down wrong paths, and how many people chose the same wrong path. In this way, the team can understand which labels are being misunderstood and/or what is being placed in the “wrong” category.

You can see why suddenly realizing post-facto that participants and moderators had different term connotations would cause problems.

The Hidden Dangers

Card sorts and tree tests provide the answers to two different questions:

1. How does my audience interpret these terms?
2. How does my audience group this information?

Beware! A single card sort or tree test can only answer one of these questions at a time. If you don’t yet know how an audience will interpret terms, you can’t truly test whether the information has been grouped correctly. Equally, if you aren’t sure whether items are grouped correctly, you can’t truly select appropriate labels.

How to Improve Tree Tests and Card Sorts

Sounds like a catch-22, no? Here’s how to get around it.

• Option 1: Use a card sort for organization, and tree testing for labels
  • Don’t worry about calling things by their potential final labels.
  • Err on the side of verbose and call an article “Article about how to mow the lawn” rather than choosing a catchy title.
  • Similarly, call a blog “Marketing blog,” rather than “Insights” or “Perspectives.”
  • Focus on nothing but how people categorize the different information chunks, and remove any tricky language.
  • Then, once the IA is on its way to final, test out labels in a tree test. You’ll see whether people connect the top level nav labels to the items housed within.
• Option 2: Learn the labels first, and tree test for navigation
  • In this variation, you may not need a card sort at all. Try conducting a user interview and ask the participant to define terms and name certain types of content.
  • From this, the content strategy team will learn what vocabulary participants are comfortable with.
  • Then, with the confidence that comes from knowing participants and moderators have a shared vocabulary, set up a tree test for sample IA.

Don’t waste your time with a tree test based on uncertain labels and navigation! Divide your tasks, and conquer content testing.