5 Reasons To Do a UX Review Before User Testing

A client once proudly came to me to review his “Quora-like app but with live video chat”. He did all the right things: build a prototype, test with users, iterate… but his users weren’t sticking around. With just a quick look, I noticed that many design decisions were poorly made: a confusing navigation, non-standard icons, poor labeling, accessibility issues… the list went on. “Why didn’t you come to me earlier?” I asked. “We could have resolved these issues months ago.”

If an expert hasn’t reviewed an interface first, time and money  may be wasted doing user testing. Apps and websites are likely to fail usability guidelines, accessibility guidelines, best practices, industry standards, and current trends; these are all issues that a User Experience Designer could catch before testing.

UX reviews help increase conversions, brand loyalty, user satisfaction, and reach more customer markets. In this post, you’ll learn 5 benefits of conducting a User Experience (UX) Review.

1. UX Reviews have a wide scope

UX reviews help teams detect major and minor problems, which allows user tests to focus on the more obscure problems. In a UX review, User Experience designers analyze the major aspects of a product: each important page and all the key elements. UX issues surface, and the expert ranks these issues based on severity and redesign priority, using the following scale:

Cosmetic problems: not necessary to fix unless extra project resources are available.

Minor problems: give low priority. However, note that numerous minor issues add up to one bad experience.

Major problems: fix right away. Major issues disrupt a seamless flow and user interaction.

Usability Catastrophes: fix immediately. Usability catastrophes prevent users from completing tasks.

This ranking system is widely used and well researched. You can find out more at Nielsen-Norman’s website.

Usability testing alone is not enough to cover the wide range of UX issues found in early prototypes or product redesigns. This limitation occurs because usability tests need to be specific in order to generate reliable results (i.e., you can’t have test users spend endless hours doing all of your tasks). You can read about what goes into each test plan at usability.gov. User testing is typically linked with specific user interactions such as sign up, purchase a product, upload and share your video, etc. In the end, resources are wasted if test users get stuck on problems that could’ve been solved beforehand.

2. There are 30+ years of established psychology, design & usability standards

UX Designers and Researchers pull knowledge from decades of scientific research, principles, best practices, and industry trends. Experts apply this knowledge to their work every day! These types of insights are very unlikely to surface from user testing alone. Below you will find a list of some of the most important UX principles today (come back later and check em’ all out!).

3. UX Reviews provide early solutions

In today’s fast-paced, competitive environment, it is essential to not only build products quickly, but to build useful, usable, and delightful products quickly. To achieve that last goal, a delightful product, you’ll need great designers, lots of user testing, and some creative and inspirational ideas. To achieve the other important goal, build a product quickly, it’s important to eliminate waste by having a few UX designers run through early sketches, wireframes, and prototypes. The earlier an expert gets in (an in-house or outside expert), the earlier poor design choices will be found and fixed.

On an additional note, I always like to say that user testing helps discover issues, but doesn’t always provide the solutions. Even with extensive user tests, an expert is still needed to provide the solutions.

Have you ever read Malcolm Gladwell’s book, Blink? If so, you’re probably familiar with the story of a rare ancient Greek kouros statue The Getty purchased for $9 million. If not, I’ll briefly introduce it to you.

The Getty purchased this kouros statue in 1985. One could understand why The Getty thought this was an extraordinary find. The kouros statue was splendid, in near perfect condition. There was also a seemingly reliable paper trail that dated back at least 100 years. Tests were run to determine the age of the marble, and the statue’s authenticity was confirmed. However, as various experts trickled in to examine the sculpture, they knew something just wasn’t right.

Other experts felt the same and pronounced the statue a fake. These experts were able to make rapid decisions on the authenticity of the statue because they had spent thousands of hours sharpening their skills and honing their knowledge. And the deeper those experts looked, the more issues they discovered. The style, proportions, and features of the statue were all slightly off. And so, The Getty’s controversial purchase goes down in the history books as a horrible business decision.

What is the take away from this story? Well, it turns out that experts can make accurate, intuitive insights in a split second. These insights lead experts in the right direction, allowing them to make decisions faster and better than non-experts in their field. These early insights can then be tested, and companies can have a higher degree of confidence in design decisions.

4. UX Reviews provide an outside perspective

When people work too long on one particular project or idea, they are likely to develop a form of“knowledge blindness”. They become unaware of the basic, obvious aspects of whatever it is that they are doing. This even happens to experts! If you have a team of UX designers that have already been working on your project, it’s a good idea to get an outside perspective before moving on to user testing.

And if you ever wonder why users don’t “get the obvious”, that’s because it’s only obvious to you!

5. UX Reviews are a quick & affordable discount usability method

Discount usability methods are relatively affordable and quick to implement, and have been shown to result in better ROI than other methods. Discount usability methods include building user scenarios, simplified user testing, and of course UX reviews (an evolved heuristic evaluation). With only a few expert evaluators, the proportion of usability problems found approaches 75%! These evaluators can also conduct solo expert reviews and still improve usability. An interesting research finding shows that the ratio of benefits to costs is highest when only 3-4 evaluators conduct a review (see the graph below). This benefits to cost ratio quickly drops off with the use of more than 5 experts.

User testing is still VERY importantDon’t let the information above mislead you. You can’t rely on UX expert reviews alone. It is a myth that companies don’t need to do user testing once they hire an expert. As to how to balance UX reviews and user testing, we’ll let you decide.

Just as “Bad user testing beats no user testing” (Nielsen), a lean UX review beats no UX review.

About the author:

Ryan O'Connor

Ryan has helped build compelling experiences for startups and Fortune 500’s around the globe, a passion he draws from his endless curiosity of psychology, computer interaction, and company culture. A published author and mentor with an M.S. in Human Factors, Ryan is on a mission to help companies discover the balance of creating delightful digital experiences while achieving business goals.