Because we don’t test users, we test designs.
We do remote testing sometimes. We’ve used Vividence, and NetRaker. We’ve NetOp‘ed into the participants computer and talked with them over the phone. But is it good enough? Does remote testing really get good data? Or better data than co-located testing (assuming co-located testing has value)?
Have you used remote testing? How well has it worked for you? Did it depend on what type of testing you did?
I've been wondering about this too. We've historically only done face to face testing, but I've considered Vividence. Do you really get actionable data out of it?
Steve
As with any testing, it's not the data that makes the difference, it's the interpretation of the data. My favorite quote regarding data comes from David Ogilvy: "I notice increasing reluctance on the part of marketing executives to use judgment; they are coming to rely too much on research, and they use it as a drunkard uses a lamp post for support, rather than for illumination."
I agree, JF. But I also think that the methodology used feeds into the data that gets collected, which then feeds into how the data is interpreted.
What is especially true for usability testing is that you have people who are making educated guesses about testing data. We can only hope that the education is valid.
I'd say remote testing is better than no testing, but nothing beats actually watching a site visitor (attempt to) use your creation.
Matthew (MO), I work designing interfaces for a large PDM company and we've had really good success with remote testing. We needed an alternative when travel budgets were slashed and we've found remote testing - using webex or similar products - work quite well.
It's like all testing, it comes down to planning, planning, planning. As long as you plan for all scenarios and make the proper accommodations for latency, refresh and even voice delay you can effectively test. We're still looking for effective recording technologies to capture screen activity, but beyond that we're pretty satisfied.
Have to agree that nothing beats being there in person, but for most testing this gets the jobs done and provides worthwhile data.
Feel free to email if you want to discuss further.
I was just thinking yesterday that I wanted to create some web apps for running Card Sorts and Freelisting remotely. I've got a project with very little to no budget and need some way of getting everything done.
Does anyone know of any quality Open Source, Freeware or cheap applications which will do a good job for this or other usability/user tests?
You can try IBM's Ease of Use area's EZSort.
EZSort is helpful, though the UI needs some serious help. I have used it to complile data from a card sort. So I entered the information later, rather than having the participants do it during the session. I'd just test it first, of course.:)
And, yup, it's free.
I haven't tried or observed remote testing. It would certainly be a big help in terms of some logistics, especially in terms of getting either testers or subjects to the test location. But, I'd be a bit worried about losing some of the things you get from in-person testing. There's a lot that hard data can tell you, but there are more subjective things that can prove to be just as important. Watching someone's reactions, hearing how they're talking themselves through a task, etc. often tell as much as or more than the actual data. Perhaps some of the remote testing solutions offer that sort of experience; I haven't researched them to know.
It will be interesting to see if this is an area that continues to be developed and expanded. The benefits of it are quite obvious. If some of the potential shortcomings can be addressed, I could see remote testing becoming almost de rigeur.
6118 Very well said chappy.
bocigalingus must be something funny.
5272
super bowl
super bowl 2005
super bowl ticket
super bowl history
super bowl sunday
super bowl xxxix
super bowl date
super bowl sunday 2005
super bowl 2005 date
super bowl 39
nfl super bowl
super bowl champion
jacksonville super bowl
super bowl party
super bowl package
super bowl ads
super bowl odds
2005 super bowl ticket
nfl super bowl ticket
super bowl ring
nfl super bowl 2005
when is the super bowl
super bowl schedule
cheap super bowl ticket
las vegas super bowl party
super bowl hotel
super bowl prediction
super bowl game
super bowl weekend
win super bowl ticket
2005 super bowl odds
maxim super bowl party
super bowl ticket package
super bowl party idea
when is super bowl 2005
super bowl football
playboy super bowl party
super bowl logo
super bowl record
past super bowl
super bowl halftime show
super bowl xxxix ticket
leigh steinberg super bowl party
super bowl 05
lee steinberg super bowl party
super bowl mvp
super bowl team
super bowl travel package
super bowl win
super bowl pick
super bowl location
super bowl stats
buy super bowl ticket