How to run a heuristic evaluation that finds real friction

A team books a heuristic evaluation the week before a launch, three designers spend a Tuesday afternoon walking the new flow, the Notion page fills with sixty-something annotated screenshots, and a "P3" tag gets applied to nine out of every ten findings. Engineering ships, the launch goes out, and a month later the same three usability problems show up as support tickets that nobody connects back to the evaluation. The artifact existed. The friction never got fixed.

This is a working guide on how to run a heuristic evaluation that produces signal a product team will actually act on: what the method is, the three reasons most evaluations stall, the six steps that earn the conclusion, and where the method pairs with real participant research rather than substituting for it. The post sits inside the broader voice user research guide and pairs with the playbooks on usability testing and how to choose a user research method.

What a heuristic evaluation actually is

A heuristic evaluation is a usability inspection method in which a small group of expert reviewers walk through an interface and judge each screen against a fixed set of usability principles, producing a list of usability issues scored by severity. The output is a ranked list of problems, anchored to a screen or a flow, scored on a shared rubric, and pointed at the team that owns the fix. It is a formative evaluation: the goal is to surface what is wrong before a feature ships, fast and cheap, with a handful of trained eyes rather than a recruited cohort of users.

The canonical principle set is Jakob Nielsen and Rolf Molich's ten heuristics, developed in 1990 and refined in 1994 against a factor analysis of 249 real usability problems. They are general principles, not a checklist: visibility of system status, match between the system and the real world, user control and freedom, consistency and standards, error prevention, recognition rather than recall, flexibility and efficiency of use, aesthetic and minimalist design, help users recognize and recover from errors, and help and documentation. The full reference lives on the Nielsen Norman Group canonical page, and the procedural side is covered in NN/g's how to conduct a heuristic evaluation guide.

Heuristic evaluation is one of the cheapest formative methods available to a product team. It does not need participant recruitment, it does not need a research budget, and it returns a ranked list of issues in a day or two. The trade is that it returns expert opinion, not user behavior. A heuristic evaluation will surface what an experienced reviewer thinks will trip a user up, calibrated against a principle set. It will not tell you what users actually do. The reviewer is informed by usability research; the reviewer is not a user.

Why most heuristic evaluations don't change anything

Three failure modes recur. All three are structural, and effort alone does not fix them.

The reviewer set is the wrong one

The most common version. The team books "a heuristic evaluation" and the reviewers are the same designers who built the screens. They know exactly what the flow is supposed to do. The reviewer's job is to spot what an unfamiliar user would not understand, and the people who built it cannot do that job no matter how thoughtful they are. The evaluation runs, fewer issues come back, and the launch goes out with the friction intact.

The fix is to recruit reviewers who did not build the flow and who are calibrated against the heuristic set the team has chosen. Nielsen's original finding was that a single reviewer surfaces roughly 35% of usability problems, three reviewers surface around 60%, and five surface around 75%; beyond five the marginal return falls off sharply. The unit of analysis is the union of independent reviewers' findings, not the average of their opinions.

The findings are catalogued, not scored

The second version. The reviewers run the walkthrough, the screenshots get annotated, the Notion page fills up, and at the end the team has a long list of issues with no ranking. A few critical issues sit next to a hundred minor ones. Engineering reads the list, picks the easy ones, and ships. The load-bearing issues never make the planning slot because the artifact never told the team which they were.

A heuristic evaluation without a severity score is a checklist with no decision attached. Each issue should be scored on three dimensions: severity (how disruptive the issue is when a user hits it), frequency (how often it will be encountered), and persistence (whether users adapt or keep hitting the same wall). The rubric does not need to be elaborate. A 0 to 4 scale on each, summed, and the top of the ranking is where the planning slot goes.

The output never meets a user

The third version. The heuristic evaluation surfaces a list of expert opinions, the team takes the list as truth, and the issues at the top of the ranking get fixed without ever exposing the fix to a real user. Heuristic evaluation is expert opinion calibrated against a principle set; it is not user behavior. A reviewer can be confident that the cancel button placement violates user control and freedom, and the actual users can be entirely fine with it, because users do not read interfaces the way reviewers do.

The fix is to treat heuristic evaluation as the upstream filter on usability testing, not the substitute. The top three to five severity-ranked issues from the evaluation become the load-bearing hypotheses for the next usability test or for a fast participant check at the surface where the friction was claimed. The expert review narrows the search; the user evidence settles it.

How to run a heuristic evaluation, step by step

Six steps. The order matters. Skipping step one (the principle set) produces a ranking that mixes apples with footnotes, and skipping step six (verification) produces a launch that ships polished fixes to issues users never had.

01 · Pick the heuristic set the product can answer to

Nielsen's ten are the default and almost always the right starting point. They generalize across desktop, mobile, voice interfaces, and conversational products, and they are the set most reviewers have already internalized. If the product is doing something genuinely unusual (an AI agent that takes actions on the user's behalf, an immersive interface, a regulated workflow with consent gates), supplement with a domain-specific set rather than swapping Nielsen out.

Two patterns that work:

• Nielsen's ten as the base. Use them verbatim. Resist the urge to rename or merge them; the labels are how reviewers index their findings, and renamed heuristics lose the cross-team shorthand.
• One supplementary set per surface. A conversational-AI surface adds principles for transparency, recoverability, and disclosure. A regulated workflow adds principles for consent, logging, and informed action. The supplementary set is a small addition (three to five extra heuristics), not a replacement.

Whichever set you pick, freeze it before the review starts and share it with all reviewers as a one-page reference. A reviewer who is consulting their memory for the principle names is a reviewer who is going to drift into personal opinion by the third screen.

02 · Recruit three to five expert reviewers

The Nielsen finding holds: three reviewers surface ~60% of findable issues, five surface ~75%, and beyond five the cost grows faster than the signal. The reviewers should be people who have run heuristic evaluations before, are familiar with the heuristic set, and did not build the surface being reviewed.

A working pattern for product teams without a research function:

• Two internal reviewers. A designer and an engineer from a different team. Familiar with the company's design system, not familiar with this flow.
• One product reviewer. A PM from a different surface area. The PM will catch the value-proposition and decision-flow issues the design-trained reviewers miss.
• One external reviewer. A contractor or a peer at another company who can review under NDA. The external eye catches the issues that look normal to internal reviewers because the company always does them that way.
• One optional accessibility reviewer. If the surface has any accessibility-significant interaction (form, navigation, modal, error state), add a reviewer trained in WCAG. Heuristic evaluation does not substitute for an accessibility audit, but a heuristic review without an accessibility lens will systematically under-rank issues that real users hit hardest.

Brief them separately. Reviewers who walk the flow together will anchor on each other's first finding and converge prematurely. The independence of the reviewer set is what produces the union of findings; collapse the independence and the method collapses with it.

03 · Define the scope and the tasks before review

The reviewer needs to know two things before opening the artifact: which slice of the product is in scope, and what user goals they should walk through it carrying. Reviewers given an open scope produce a wandering catalogue of personal preferences. Reviewers given a goal walk the flow the way a user would and surface the friction at the touchpoints that matter.

Each reviewer gets the same brief:

• The scope. The exact set of screens, the start state, and the end state. A reviewer who follows a link off the brief is reviewing a different artifact.
• The tasks. Two to four representative user goals, written as a sentence the user would say to themselves. "Sign up, set up a workspace, invite two teammates, and run the first synced import." Not "evaluate the onboarding flow."
• The heuristic set. The frozen principle list from step one.
• The output template. A row per issue, columns for the screen, the heuristic violated, the description, and the severity score. A consistent template is what makes the union step possible.

The scoping is also the moment to be honest about what is not in scope. Visual polish on screens slated for redesign next sprint is not worth reviewing. Microcopy on an experimental flow that may not ship is not worth reviewing. A scoped evaluation surfaces five real issues; an unscoped one surfaces fifty issues the team already knew about and will not act on.

04 · Score each issue by severity, frequency, and persistence

Each finding gets three scores on a 0 to 4 scale.

• Severity. How much does this issue disrupt the user when they hit it? 0 is a non-issue. 4 is "user cannot complete the task." A confused cancel button is a 2. A signup form that silently fails on certain emails is a 4. The Nielsen severity scale (cosmetic, minor, major, catastrophic) maps cleanly onto 1 through 4 for teams that prefer the canonical names.
• Frequency. How often will users encounter this? 0 is "never under normal use." 4 is "every session." A footer link bug is 1; a primary navigation issue is 4. Frequency is where reviewer estimates start to weaken (a reviewer who has never seen the analytics is guessing), which is why the verification step in 06 exists.
• Persistence. Do users adapt and stop hitting it, or do they keep hitting it session after session? A surprising default that users learn around once is a 1. A confusing terminology choice that surfaces in every error state is a 4. Persistence is the dimension that separates "minor friction" from "drag on adoption" and is often the one that makes the planning case for fixing an issue that scored low on raw severity.

Sum the three for a composite. Sort by composite. The top of the list is where the planning slot goes. Issues with a composite below four are typically not worth fixing in the next sprint; flag them on the artifact, return to them when the surface gets another pass.

05 · Cluster findings across reviewers

Each reviewer hands in a per-issue list. The synthesis step is to cluster across reviewers: two reviewers describing the same friction become one issue with two pieces of evidence; one reviewer describing a friction the other four missed becomes one issue with one piece of evidence and a note that warrants verification. The unit is the union, never the average.

Three rules:

• Cluster by underlying friction, not by surface position. Two reviewers can describe the same root cause from different screens; one issue, two pieces of evidence. A team that clusters by screen position will count the same problem three times.
• Preserve the divergences. A finding only one reviewer caught is still a finding. Tag it as low-redundancy and route it into the verification step instead of dropping it. Some of the most valuable issues are the ones the other reviewers walked past.
• Score the cluster, not the strongest reviewer's opinion. Take the median severity across the reviewers who flagged the cluster; do not let the most opinionated reviewer drive the ranking. The general pattern is covered in how to synthesize user research.

The output is a single sheet: surface, heuristic violated, description, evidence (the reviewer notes), composite score, and an owner. Engineering reads it as a backlog; the PM reads it as a decision frame.

06 · Verify the load-bearing issues with real participants

This is the step that decides whether the evaluation was research or theatre. Take the top three to five composite-ranked issues and put them in front of real participants. Two patterns work, often together:

• Targeted usability test on the issue. Recruit five participants and run a short session against the specific task where the issue lives. The Nielsen "five users surface 85% of usability problems" finding (also from NN/g) is the empirical basis for keeping the cohort small. The full operational treatment is in how to run usability testing.
• A standing async link on the surface. Place a study link on the screen where the issue was flagged and let participants who hit the friction in normal use answer in voice, text, choice, or rating. Voice catches the verbal "wait, what?" pause that proves the reviewer's prediction; rating catches the swing in confidence across many participants; choice and text cover the participants who would not bother to record a voice answer. Probing depth can be set per question: shallow on a quick rating check, medium on activation surfaces where the participant has more to say, expert on the long-form review where the AI keeps probing until it has the same context a senior researcher would dig out in a moderated interview. The pattern is covered in AI follow-up questions for user research.

"I tagged the empty state as a violation of recognition. After watching three participants land on it and say 'oh, that's clear,' I have to retract. The empty state is fine. The path that gets users there is the problem."

The reversal in the pull-quote above is what verification is for. The reviewer was confident, the heuristic violation was real, and the user evidence pointed at the upstream root cause that the screen-level review had missed. A heuristic evaluation that ships without verification ships the reviewer's first answer. A heuristic evaluation that ships with verification ships the reversal.

Where heuristic evaluation pairs with user research

Heuristic evaluation is not a standalone artifact. It works hardest when it sits inside a wider rhythm.

Usability testing is the natural downstream pair. The evaluation narrows the search to a small ranked list; usability testing settles the questions on it. Running the evaluation before the usability test is the cheap order: fewer screens to test, sharper hypotheses per screen, smaller cohort needed to get a decisive answer.

5-second tests and tree tests cover the value-proposition and information-architecture layers that heuristic evaluation is structurally weak at. A reviewer is not a good proxy for first-impression comprehension or for the IA expectations of a user landing for the first time; the participant-led methods are.

Stakeholder interviews and internal reviews are the upstream pair. Before reviewers walk the screens, the product team often benefits from sharing the same scope and tasks with engineering, design, support, and legal to surface the constraints the reviewers should be carrying. Share the same brief in internal channels; the team gets a synthesized view of every stakeholder's input before the review begins, and the reviewers walk the flow with the constraints they need to be honest about.

The wider pattern is the voice user research guide, which positions heuristic evaluation as the formative inspection step in a continuous-discovery rhythm rather than as a pre-launch ritual.

When to use something else

Three cases where running a heuristic evaluation will make a team feel rigorous while producing the wrong artifact.

Strategy-level decisions. Heuristic evaluation answers screen-level usability questions. It will not tell you whether the value proposition is right, whether the segment is the right one, or whether the price band lands. The right tools for those questions are concept testing, jobs to be done interviews, and pricing research. A heuristic evaluation of a strategic question returns a list of polished violations on a flow that should not exist in the first place.

Brand-new categories. When the interface is the first of its kind, the heuristic set has nothing to anchor to. A reviewer walking an interface that breaks an existing pattern will flag every break as a violation of consistency and standards, and most of the flags will be wrong because the standard the reviewer is consulting was set by a different category. The right tool here is participant observation in a contextual inquiry or a continuous-discovery rhythm.

Accessibility-critical surfaces. A heuristic evaluation will surface accessibility issues incidentally, but it is not the right tool to satisfy a WCAG conformance question. A dedicated accessibility audit, run by an evaluator trained against the relevant guideline level, is the right artifact. Combine the two when both are needed; do not substitute one for the other.

FAQ

What is a heuristic evaluation in usability testing?

A heuristic evaluation is a usability inspection method in which a small group of expert reviewers walks through an interface and scores each screen against a fixed set of usability principles, producing a ranked list of usability issues. It is a formative method: the goal is to surface what is wrong before a feature ships, fast and cheap, with three to five trained reviewers rather than a recruited cohort of users. The canonical principle set is Nielsen and Molich's ten heuristics, refined in 1994 and unchanged since.

What are Jakob Nielsen's 10 usability heuristics?

Visibility of system status, match between the system and the real world, user control and freedom, consistency and standards, error prevention, recognition rather than recall, flexibility and efficiency of use, aesthetic and minimalist design, help users recognize and recover from errors, and help and documentation. They are general rules of thumb, not a checklist, and they generalize across desktop, mobile, voice, and conversational interfaces. The canonical reference lives at the Nielsen Norman Group.

How many reviewers do you need for a heuristic evaluation?

Three to five. One reviewer surfaces roughly 35% of findable problems, three surface around 60%, and five surface around 75%; beyond five the marginal return falls off sharply against the cost of the additional reviewer. Brief them separately and synthesize the union of their findings rather than the average. The reviewers should be people familiar with the heuristic set and unfamiliar with the flow being reviewed.

What's the difference between heuristic evaluation and usability testing?

Heuristic evaluation is expert review against a fixed principle set; usability testing is observation of real users completing tasks. The two are sequential, not interchangeable. Heuristic evaluation is fast, cheap, and tells you what trained reviewers predict will trip users up. Usability testing is slower, costlier, and tells you what users actually do. The cheap order is to run the evaluation first to narrow the search, then test the load-bearing issues with participants. The longer treatment is in how to run usability testing.

How do you rank heuristic evaluation findings?

By a composite of severity (how disruptive the issue is when hit), frequency (how often users encounter it), and persistence (whether users adapt or keep hitting it). A 0 to 4 scale on each, summed, gives a composite score that sorts the list. The top of the ranking is where the planning slot goes; issues below a composite of four are flagged but typically not worth fixing in the next sprint. Score the cluster, not the strongest reviewer's opinion, by taking the median across reviewers who flagged the same underlying friction.

Should a heuristic evaluation replace usability testing?

No. Heuristic evaluation returns expert opinion calibrated against a principle set; it does not return user behavior. The two methods answer different questions, and replacing one with the other produces launches that ship polished fixes to issues real users never had. Use the evaluation to narrow the search to three to five load-bearing issues, then verify those issues with real participants on the surface where the friction was claimed, in voice, text, choice, or rating, with adaptive probing set to the depth the question deserves.

---

A heuristic evaluation that ships into a backlog without a severity ranking, run by reviewers who built the flow, with no participant verification on the load-bearing issues, is a method going through the motions. A heuristic evaluation that runs three to five independent reviewers against Nielsen's ten, scores each finding on severity and frequency and persistence, clusters the union of findings across reviewers, and verifies the top of the list with real users before the fix ships is the version that actually moves a product forward. Talkful runs the verification side: a standing study link on the surface where the issue was flagged, participants answer in voice, text, choice, or rating on their own time, configurable adaptive probes turn the polite first reaction into the second-turn reversal, and a real-time synthesis engine returns themes, citations, and sentiment to the team as the responses land, ready for the trio to ship from or for the agents you build with to act on. The wider voice user research guide covers where the method sits inside a continuous practice.