Steven Hannah’s Sorting Out Card Sorting isn’t really about card sorting. It’s an example of using a particular methodoloy to do academic literature review and then a proposal for creating a tool that can be used to augment and extend the knowledge discovered during the review.
But even if the phrases grounded theory and constant comparative method make you feel a bit woozy it’s worth having a look at Chapter 4: Analyzing the Data. Hannah discovers twelve characteristics that are used to describe card sort exercises which roughly reflect how card sorting is done and discussed.
These characteristics make a decent basis for a “things you need to consider while planning your card sorting exercises” check list. I’ve pruned a couple of redundant items and rearranged the rest into a useful order for planning.
- Define the information domain
- Select a target audience
- Choose between open and closed sort
- Choose between individual or group exercises
- Choose a number of cards to be used
- Select the objects to be used on the cards
- Set a time limit for the exercise
- Choose analysis methods.
Two things emerge as consistent recommendations:
- Investigators should provide a little guidance to the test subjects as possible. Only what is necessary to get people piling the cards up.
- Card sorts should be limited to less than 100 objects (cards), or what ever will take less than one hour.
The question of how to analyze the results of the user sorts is, well… murky. The usual dichotomy of qualitative vs., quantitative exists here as every where. The difficulty is that while designers (and others of the editorial bent – this by JJG is food for thought on the subject) are willing to “eyeball” the conclusions there are others who want something more rigorous. They justifiably point out that the masses of data brought to light by card sorts over large sets or the responces of of many participants can yield results that are too large to wrap one’s mind around.
Surely quantitative analysis is a good thing – especially if there is a perceived need for “facts and figures” to justify design decisions. There is however as of yet little information on the specific types of quantitative analysis that are appropriate and how the results would be used in the design process.
One aspect of card sorting that is not considered in Hannah’s survey is the question of computer aided sorting exercises. This software didn’t exist when most of the literature surveyed in this paper was published. These tools exist now and whether or not to use them is an important test design question.
Oh… about the presentation on the web page. I have not been able to make it work. It won’t even attempt to load in Firefox. It tries to load under IE7/Vista but enters a loop and takes up all my bandwidth. I’d say that I’ll go try it on the laptop with XP and IE6 but, you know that I won’t; it’s too much trouble. Too bad.