User experience and usability

Card sorts are usually used for gathering quantitative data. But they can be valuable as well as a knowledge elicitation technique for qualitative data, especially when you’re working in domains outside your expertise. The main benefit is that in addition to the categories, you get to know a bit more about the criteria that your participants are using and their thinking process. (Although if you’ve never done a card sort before, you might be surprised how much you can learn from a standard computer based card sort.)

If working with one participant at a time, use normal think-aloud instructions. Do a warm-up exercise if necessary. Alternatively, you could pair up your participants and have them talk to each other, or use a larger group.

• Encourage to make a pile for ‘other’ or ‘don’t know this item’. That way the sort will be focused on items that the participants have something to say about. You can always revisit the rest pile at a later stage.
• Ask why questions (that is, if they don’t talk enough on their own). Why does this item belong in pile A? With really strong participants such as true domain experts you can even ask ‘why not’ without influencing the results (for example when asking a medical doctor to categorize symptoms).
• Often, during the sort people will change their mind. Add a new category, change a label, or move an item. This is where it gets really interesting. What was wrong, why is this better?
• Have some blank cards ready, in case a participant can’t choose between two or more piles because he feels the item can belong to all of them. In that case you can duplicate the card.
• When the sort is complete, take a look at the results. Does your participant feel that any items were missing? If that’s the case, it’s easy to create a few extra cards and at them to the relevant piles.
• After the initial sort, you can move up or down into the hierarchy. Ask to split up large categories into subcategories. And ask to group multiple categories together into supercategories. You can strongly encourage splitting up by taking 3 random items, and asking which 2 are most similar (the other one will go into a new category, taking all items that belong there too with it).
• Some members of a category are more central than others. You can ask which items are the best, most representative, example(s) for each category.
• Ask questions about the chosen labels. What makes them a good description? Alternative wordings?
• Ask questions about the participant. How familiar are they with the items? Do they consider themselves an expert in the domain? If so, did that influence their sorting?

With this type of card sorting the final groupings are less important than the thoughts during the process. Your questions may influence the participant’s behavior, so I wouldn’t mix the results from such sessions with those where participants work on their own.