shiny things in messy little piles

Month: August 2007

TQR — Sorting Out Card Sorting

Steven Han­nah’s Sort­ing Out Card Sort­ing isn’t real­ly about card sort­ing. It’s an exam­ple of using a par­tic­u­lar method­oloy to do aca­d­e­m­ic lit­er­a­ture review and then a pro­pos­al for cre­at­ing a tool that can be used to aug­ment and extend the knowl­edge dis­cov­ered dur­ing the review.

But even if the phras­es ground­ed the­o­ry and con­stant com­par­a­tive method make you feel a bit woozy it’s worth hav­ing a look at Chap­ter 4: Ana­lyz­ing the Data. Han­nah dis­cov­ers twelve char­ac­ter­is­tics that are used to describe card sort exer­cis­es which rough­ly reflect how card sort­ing is done and discussed.

These char­ac­ter­is­tics make a decent basis for a “things you need to con­sid­er while plan­ning your card sort­ing exer­cis­es” check list. I’ve pruned a cou­ple of redun­dant items and rearranged the rest into a use­ful order for planning.

  1. Define the infor­ma­tion domain
  2. Select a tar­get audience
  3. Choose between open and closed sort
  4. Choose between indi­vid­ual or group exercises
  5. Choose a num­ber of cards to be used
  6. Select the objects to be used on the cards
  7. Set a time lim­it for the exercise
  8. Choose analy­sis methods.

Two things emerge as con­sis­tent recommendations:

  • Inves­ti­ga­tors should pro­vide a lit­tle guid­ance to the test sub­jects as pos­si­ble. Only what is nec­es­sary to get peo­ple pil­ing the cards up.
  • Card sorts should be lim­it­ed to less than 100 objects (cards), or what ever will take less than one hour.

The ques­tion of how to ana­lyze the results of the user sorts is, well… murky. The usu­al dichoto­my of qual­i­ta­tive vs., quan­ti­ta­tive exists here as every where. The dif­fi­cul­ty is that while design­ers (and oth­ers of the edi­to­r­i­al bent — this by JJG is food for thought on the sub­ject) are will­ing to “eye­ball” the con­clu­sions there are oth­ers who want some­thing more rig­or­ous. They jus­ti­fi­ably point out that the mass­es of data brought to light by card sorts over large sets or the responces of of many par­tic­i­pants can yield results that are too large to wrap one’s mind around.

Sure­ly quan­ti­ta­tive analy­sis is a good thing — espe­cial­ly if there is a per­ceived need for “facts and fig­ures” to jus­ti­fy design deci­sions. There is how­ev­er as of yet lit­tle infor­ma­tion on the spe­cif­ic types of quan­ti­ta­tive analy­sis that are appro­pri­ate and how the results would be used in the design process.

One aspect of card sort­ing that is not con­sid­ered in Han­nah’s sur­vey is the ques­tion of com­put­er aid­ed sort­ing exer­cis­es. This soft­ware did­n’t exist when most of the lit­er­a­ture sur­veyed in this paper was pub­lished. These tools exist now and whether or not to use them is an impor­tant test design question.

Oh… about the pre­sen­ta­tion on the web page. I have not been able to make it work. It won’t even attempt to load in Fire­fox. It tries to load under IE7/Vista but enters a loop and takes up all my band­width. I’d say that I’ll go try it on the lap­top with XP and IE6 but, you know that I won’t; it’s too much trou­ble. Too bad.

More on Denton’s How to Make a Faceted Classification and Put It On the Web

Dur­ing a recent slog up tread­mill hill I read through Wm. Den­ton’s How to Make a Faceted Clas­si­fi­ca­tion and Put It on the Web.

I feel bet­ter now. After my grad school deja vu expe­ri­ence with Sim­pli­fied Facet Clas­si­fi­ca­tion I was despair­ing of ever being able to bring facets to the mass­es. Or in my case to intel­li­gent but not LIS qual­i­fied engi­neers and oth­er tech types.

You’ll still need some back­ground in clas­si­fi­ca­tion to under­stand the details but it’s a great overview that is under­stand­able by peo­ple with some back­ground knowl­edge mod­el­ing. Most web and soft­ware dev types have, often unknow­ing­ly, done a fair amount of infor­mal knowl­edge modeling.

Den­ton’s straight for­ward style makes his dis­cus­sion clear enough that, bar­ring melt down
over the tech­ni­cal­i­ties of the entity/instance analy­sis and facet cre­ation in steps 2 and 3
you can hand this essay out as back­ground reading.

Sec­tion 1: When to Make a Faceted Clas­si­fi­ca­tion gives a nice overview of where faceted clas­si­fi­ca­tion sys­tems fits into the field of clas­si­fi­ca­tion and orga­niz­ing schemes in gen­er­al. Den­ton pro­vides use­ful ques­tions to ask when con­sid­er­ing faceted clas­si­fi­ca­tion. It’s refresh­ing to see a con­sid­er­a­tion of facets that also dis­cuss­es when facets are not the best answer to your questions.

In the sec­ond sec­tion Den­ton divides the actu­al work of cre­at­ing the clas­si­fi­ca­tion sys­tem (facets and foci) into 7 steps. Begin­ning with Domain Col­lec­tion and end­ing with Revi­sion, Test­ing, and Main­te­nance. (Love to see that word main­te­nance laid out in black on white!) It’s a sim­ple method­ol­o­gy that will get you through the process and give you a work­able sys­tem at the end of the day.

What fol­lows are a few notes on his descrip­tion of the process.

Start­ing with what I believe is the miss­ing first step.

Defin­ing Your Domain

Step 0: Define the Domain

You must have a sol­id and agreed on def­i­n­i­tion of the sub­ject and scope of the domain
before you start. We are all aware that assum­ing that what’s already in the sys­tem is the
lim­it of the domain that needs to be con­sid­ered but also be care­ful that you do not make
the assump­tion that every­thing dealt with by your web­site or soft­ware should be includ­ed in
the domain for which you are build­ing the classification.

Build­ing the Faceted Classification

The next five steps make up the build­ing the facets and foci part of the process.

Step 1: Domain Collection.
Step 2: Enti­ty Listing.
Step 3: Facet Creation.

Note that you’ll have to do quite a bit of iter­at­ing over steps 1, 2, and 3. The process of col­lect­ing, ana­lyz­ing, and defin­ing always brings to light bits and pieces that were missed in the first (lat­est) go ’round. It’s just a fact of life, so plan for it.

Den­ton does not dis­cuss the tech­niques of analy­sis that can be used to get from the enti­ty (things) list to the facets (char­ac­ter­is­tics) list. This sort of analy­sis is com­plex and domain depen­dent and IMHO the most com­mon point of break down for many attempts. I’m always on the look out for mate­r­i­al that describes these tech­niques, as well as “real” world examples.

Once you have a sys­tem of facets and foci you have to decide on how to arrange its pieces:

Step 4: Facet Arrangement

I find Den­ton’s expla­na­tion of this step not entire­ly clear. You may have to explain that
you are now work­ing with both the foci (terms) and facets. The end result of this step will
be a list of facets and and arrange­ment of the foci with­in each facet. You will definitely
have to reit­er­ate that the foci with­in each facet are arranged in a way that best reflects
the sub­ject of the indi­vid­ual facet. Once again — the things inside one facet don’t have to
be arranged in the same way as the things inside anoth­er facet. (Can you tell that I’ve had
trou­ble with get­ting this across? Data­base jock­eys are the worst. Facets and foci do not
map well onto tables, fields, and joins.)

Step 5: Cita­tion Order

Cita­tion order is of less impor­tance in an elec­tron­ic sys­tem than it was in the paper
sys­tems in use when faceted clas­si­fi­ca­tion was first invent­ed. Though you may find yourself
in a sit­u­a­tion in which you aren’t going to be able to take full advan­tage of the
flex­i­bil­i­ty of a com­put­er based sys­tem to mix and match your facets for tech­ni­cal or bud­getary rea­sons. No mat­ter how flex­i­ble your sys­tem is you are going to have to decide on a default dis­play order and behav­ior so don’t skip this step entire­ly but don’t allow any­one to get hung up on it either.

Apply the Faceted Classification 

Step 6: Classification

And now we get to the whole point. Apply­ing the clas­si­fi­ca­tion sys­tem to the stuff. And
it’s about as sim­ple as Den­ton makes it sound. Sometimes…

Before hand­ing this task off to the near­est con­ve­nient, unoccupied,warm body take a clear
eyed look at how many of your facets include terms (foci) that will require judg­ment calls
to get things labeled prop­er­ly. Who’s best qual­i­fied to make these judg­ment calls in a way
that will serve your ‑users-?

Check­ing it twice, Get­ting it on the road, and Keep­ing it running

Step 7: Revi­sion, Test­ing, and Maintenance.

Note that you have been doing iter­a­tive test­ing and revi­sion through out the cre­ation of
the sys­tem. If you’ve got­ten here with­out hav­ing to rethink or redo any part of your system
you are one of: work­ing in a very lim­it­ed well-know domain, very lucky, not paying
attention.

Actu­al­ly only revi­sion and test­ing should be includ­ed in step 7. Main­te­nance should be step
8. Any clas­si­fi­ca­tion sys­tem that does­n’t include main­te­nance as sep­a­rate, on-going phase
is bound to suf­fer ROT.

The final sec­tion: How to Put the Clas­si­fi­ca­tion on the Web tack­les the ques­tion of how to use your new faceted clas­si­fi­ca­tion scheme to help your users nav­i­gate the stuff. I’ll be dis­cussing Den­ton’s help­ful sug­ges­tions in anoth­er essay.

Con­clu­sions:

If I were hand­ing this out ’round the table in a con­fer­ence room packed with devs and coders I’d leave out the third sec­tion titled: “How to store the faceted sys­tem in a com­put­er”. The tech­ni­cal Ways, Whys, and Where­fore’s of stor­ing and access­ing meta­da­ta such as a faceted clas­si­fi­ca­tion sys­tem go far beyond what is cov­ered by Den­ton’s cou­ple of pages of X(F)ML and SQL exam­ples. It nev­er pays to drop a shal­low solu­tion to a prob­lem into a room of peo­ple who are trained to take any prob­lem laid before them and debate the best way to do it.

TQR — Introductory Tutorial on Thesaurus Construction

If the­saurus con­struc­tion is some­thing that comes up only occa­sion­al­ly in the course of your work you should book­mark this tuto­r­i­al cre­at­ed by Dr. Tim Craven of West­ern Ontario Uni­ver­si­ty for his LIS students.

Eight sec­tions take you quick­ly through the basic con­cepts and con­sid­er­a­tions for build­ing a the­saurus. It’s a handy refresh­er that is soft­ware agnostic.

In fact the sec­tion head­ings would make a good out­line for a set of ques­tions to ask the soft­ware ven­dors if you are con­sid­er­ing pur­chas­ing a the­saurus man­age­ment system.

Speak­ing of the­saurus soft­ware, Dr. Craven also has a hand­ful of free­ware pro­grams to assist in index­ing and the­saurus con­struc­tion. I haven’t checked them out yet and so can’t offer an opinion.