fb
Make-it-a-pony-final

5 Reasons to do a UX review before user testing

Shares

A client once proudly came to me to review his “Quora-like app but with live video chat”. He did all the right things: build a prototype, test with users, iterate… but his users weren’t sticking around. With just a quick look, I noticed that many design decisions were poorly made: a confusing navigation, non-standard icons, poor labeling, accessibility issues… the list went on. “Why didn’t you come to me earlier?” I asked. “We could have resolved these issues months ago.”

If an expert hasn’t reviewed an interface first, time and money  may be wasted doing user testing. Apps and websites are likely to fail usability guidelines, accessibility guidelines, best practices, industry standards, and current trends; these are all issues that a User Experience Designer could catch before testing.

UX reviews help increase conversions, brand loyalty, user satisfaction, and reach more customer markets. In this post, you’ll learn 5 benefits of conducting a User Experience (UX) Review.

1. UX Reviews have a wide scope

UX reviews help teams detect major and minor problems, which allows user tests to focus on the more obscure problems. In a UX review, User Experience designers analyze the major aspects of a product: each important page and all the key elements. UX issues surface, and the expert ranks these issues based on severity and redesign priority, using the following scale:

  • Cosmetic problems: not necessary to fix unless extra project resources are available.
  • Minor problems: give low priority. However, note that numerous minor issues add up to one bad experience.
  • Major problems: fix right away. Major issues disrupt a seamless flow and user interaction.
  • Usability Catastrophes: fix immediately. Usability catastrophes prevent users from completing tasks.

This ranking system is widely used and well researched. You can find out more at Nielsen-Norman’s website.

Usability testing alone is not enough to cover the wide range of UX issues found in early prototypes or product redesigns. This limitation occurs because usability tests need to be specific in order to generate reliable results (i.e., you can’t have test users spend endless hours doing all of your tasks). You can read about what goes into each test plan at usability.gov. User testing is typically linked with specific user interactions such as sign up, purchase a product, upload and share your video, etc. In the end, resources are wasted if test users get stuck on problems that could’ve been solved beforehand.

2. There are 30+ years of established psychology, design & usability standards

Expert reviews are especially useful for finding violations of standards and best practices
Jim Ross UX & Usability Guru

UX Designers and Researchers pull knowledge from decades of scientific research, principles, best practices, and industry trends. Experts apply this knowledge to their work every day! These types of insights are very unlikely to surface from user testing alone. Below you will find a list of some of the most important UX principles today (come back later and check em’ all out!).

3. UX Reviews provide early solutions

In today’s fast-paced, competitive environment, it is essential to not only build products quickly, but to build useful, usable, and delightful products quickly. To achieve that last goal, a delightful product, you’ll need great designers, lots of user testing, and some creative and inspirational ideas. To achieve the other important goal, build a product quickly, it’s important to eliminate waste by having a few UX designers run through early sketches, wireframes, and prototypes. The earlier an expert gets in (an in-house or outside expert), the earlier poor design choices will be found and fixed.

On an additional note, I always like to say that user testing helps discover issues, but doesn’t always provide the solutions. Even with extensive user tests, an expert is still needed to provide the solutions.

Have you ever read Malcolm Gladwell’s book, Blink? If so, you’re probably familiar with the story of a rare ancient Greek kouros statue The Getty purchased for $9 million. If not, I’ll briefly introduce it to you.

kouro

They Getty purchased this kouros statue in 1985. One could understand why The Getty thought this was an extraordinary find. The kouros statue was splendid, in near perfect condition. There was also a seemingly reliable paper trail that dated back at least 100 years. Tests were run to determine the age of the marble, and the statue’s authenticity was confirmed. However, as various experts trickled in to examine the sculpture, they knew something just wasn’t right.

Other experts felt the same and pronounced the statue a fake. These experts were able to make rapid decisions on the authenticity of the statue because they had spent thousands of hours sharpening their skills and honing their knowledge. And the deeper those experts looked, the more issues they discovered. The style, proportions, and features of the statue were all slightly off. And so, The Getty’s controversial purchase goes down in the history books as a horrible business decision.

What is the take away from this story? Well, it turns out that experts can make accurate, intuitive insights in a split second. These insights lead experts in the right direction, allowing them to make decisions faster and better than non-experts in their field. These early insights can then be tested, and companies can have a higher degree of confidence in design decisions.

4. UX Reviews provide an outside perspective

When people work too long on one particular project or idea, they are likely to develop a form of “knowledge blindness”. They become unaware of the basic, obvious aspects of whatever it is that they are doing. This even happens to experts! If you have a team of UX designers that have already been working on your project, it’s a good idea to get an outside perspective before moving on to user testing.

And if you ever wonder why users don’t “get the obvious”, that’s because it’s only obvious to you!

5. UX Reviews are a quick & affordable discount usability method

Discount usability methods are relatively affordable and quick to implement, and have been shown to result in better ROI than other methods. Discount usability methods include building user scenarios, simplified user testing, and of course UX reviews (an evolved heuristic evaluation). With only a few expert evaluators, the proportion of usability problems found approaches 75%! These evaluators can also conduct solo expert reviews and still improve usability. An interesting research finding shows that the ratio of benefits to costs is highest when only 3-4 evaluators conduct a review (see the graph below). This benefits to cost ratio quickly drops off with the use of more than 5 experts.

stats-blog1

User testing is still VERY important

Don’t let the information above mislead you. You can’t rely on UX expert reviews alone. It is a myth that companies don’t need to do user testing once they hire an expert. As to how to balance UX reviews and user testing, we’ll let you decide.

Just as “Bad user testing beats no user testing” (Nielsen), a lean UX review beats no UX review.

Get exclusive UX tips not found on the blog!

Email *
Shares
Get UX Tips & Best Practices Straight To Your Inbox
We hate spam as much as you do!

8 Comments. Leave new

One critique, I don’t think “bad user testing beats no user testing”. It is key in today’s world to find the right UX and UI designers for our team and get the job done as effectively as possible. From a business standpoint, I feel that bad user testing is a waste of money and has overall a negative impact rather then a positive impact. When looking for the right expert to Carry on your project, make sure to do a lot of research and find the person that has a very relevant/specialized background to what you are trying to do. Also; Thanks for bringing all the UX/UI facts and principles together. Very beneficial for my learning.

Reply

Good point Steven. I think that you’re right that in some cases misinformation can be a waste of time. I think maybe it comes down to being able to learn something of value. Being able to decipher quality feedback from the other stuff is definitely a tight rope to walk.

Reply

Yes, finding the right specialist for your project type is definitely important! Possibly a mix of a fresh perspective and a specialized expert? That “knowledge blindness” can be pretty powerful.
I think when Nielsen said the “bad user testing is better than no user testing”, he may have been joking that even quick, cheap, dirty user testing is likely to provide some sort of value to improving your product. Users always bring up things you had never thought of while designing.

Reply
Jeremy Schweitzer
April 21, 2015 3:46 pm

Nicely written, and a treasure trove of cool quotes and links about UX and best practices. Looking forward to more content from you!

Reply

Thank you. I won’t disappoint!

Reply

This post has so much great information that’s useful for beginners and experts alike. I can’t tell you how many times I’ve seen people do testing without any evaluation first and then get upset at the amount of time and money it takes to complete the tests, do the analysis, and implement the results. From my experience, it takes so much longer to do UX testing in this case because the initial prep work hasn’t been done and users sometimes end up focusing on the small things instead of what could be more big picture items or issues that they really should be focusing on during testing. Thanks for posting this and all of the resources, so we can get the word out about the best and most efficient approach for evaluating a site!

Reply

Great Article Ryan, loved every bit of it.

Reply

I find it fascinating how the USDA and Tom Vilsak of MONSANTO can stand up and acknowledge smaller farms and Farmers Markets. The USDA just isn’t for tiny farmers. The #1 interest in the USDA is GMOs. The USDA has allowed the poisoning of Organic Farms with pesticide residue and GMO drifting by allowing MONSANTO and many others to sel-eregulatf. This can be a policy of stupidity and destruction which will lead to your demise of most little farms and related agricltural businesses.

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *


Get advice from certified UX experts

Bonus: Enter your email below to receive a download link of this post as a PDF