2

I would like to conduct a Usability test for an Applications' portal. I know about the separate tasks, the sessions set-up, the card-sorting, the screen capturing and most of the full Monty. I will have to perform everything in a single session (no question about that) with each test Use. I would like to hear from experienced people how they dealt with the sequence of all these different stuff in similar situations.

Thank you in advance for all your input.

flag

5 Answers

4

Sounds like you might be cramming a little too much into this test. I'd suggest running the card sorting separately from the usability testing. Card sorting usually takes an hour and participants tend to feel a bit drained afterwards so I'm not sure they'll be that productive on your test tasks if you make them do a card sort first!

When you get to the test itself, the rule for test tasks is as follows:

  1. Give your participant an easy, no-fail task to start, so that he or she gains confidence in the setting.
  2. If you have tasks that must be done before the others (such as an installation task or a registration task), then put those tasks first.
  3. Give the remaining tasks in a random order. This prevents order effects -- for example, participants may get better/worse as the test goes on, and if you give your tasks in a fixed sequence you may mistake this as a strength/weakness of the interface.
link|flag
3

It does depend on exactly what elements you're testing... generally, it's better to order them "naturally", i.e. how a person would go through them in real life.

I usually start my tests with a scenario, for example: "You are in the process of moving house and are looking for help...", then the tasks follow on from that. So what's the first thing someone would use the portal for?

It's also best to avoid "previewing" future tasks if possible. Start off with something fairly open and fairly simple to get them comfortable with the Talk Aloud technique (assuming that's what you're using), then move onto the tasks that involve deeper usage. If the later task includes something they've already seen, they may be less reactive to it.

How well this applies to your testplan will depend entirely on what you're testing and how! So best of luck :-)

link|flag
2

I would do the card sorting before the user performs tasks on the system. Card sorting is an activity intended to provide data about organization/architecture/labelling outside of any design aspects. So you don't want to expose the user to the interface/prototype before you gather their unbiased feedback.

Then ease the participant into the tasks, starting with the easiest/most achievable task first. (echoing Rose's comment)

link|flag
1

Similar to the answer by Rose, I'd list the key success criterion and determine which of your test methods overlap. Save the most comprehensive (most overlap) for last and start with the ones that have specific, contained criterion result. Otherwise, in one test session, you can skew the results with familiarity.

link|flag
1

What's the goal of the card sorting? What's the goal of the user testing?

I'd normally expect to be doing things like card sorting pretty early in a project - because the results you get about the taxonomy of the domain are going to affect the design. If you've already got a product for user testing - is card sorting the best way to get that information?

Like others - I'd be worried about the ordering. Doing the card sort first means that you're going to familiarise the user with a bunch of terms that the portal is likely to use - biasing the results. Doing the user testing first is likely to bias the user with respect to the taxonomy currently used on the portal.

Because of this I wouldn't have the same person do the same test - even if it meant I got results from fewer people.

Instead maybe consider:

  • With the group doing the card sort - do a quick run through of the site with them post-sort. Ask them about where they'd expect to find the categories they discovered in the sort. Ask them whether they'd considered categories on the site that didn't appear in the card sort. Spend time exploring and understanding the categories together - rather than testing.

  • With the group doing the user testing, consider including tasks that are explicitly aimed at discovering whether the categories used in the portal are valid. If you run the card sort group first you should have an interesting set of labels to experiment with.

Hopefully this makes vague sense :)

link|flag

Your Answer

Get an OpenID
or

Not the answer you're looking for? Browse other questions tagged or ask your own question.