A technique using closed card-sorting to evaluate candidate information architectures for a web site is described. Participants in an online card-sorting study are randomly directed to one of the architectures being evaluated. Each participant is shown the same cards but different categories to sort them into.
The basic data collected is simply which cards each user put into which groups. For any one architecture being tested, the data show what percentage of the participants put each card into each group. A better architecture is one where the participants were more consistent with each other in terms of which groups they put each of the cards into. The basic ‘score’ proposed for each card is the percentage associated with the ‘winning’ group (i.e., the group with the highest percentage)—the higher that percentage is, the better. A consistency score for each architecture tested can then be calculated by taking an average of these percentages across all the cards. A technique for correcting this score when the different architectures have different numbers of groups is also described.