How to run user research on AI features

The first AI feature a product team ships almost always gets tested wrong. The team runs an eval, watches the model output on a test set, and calls the work done. Then a hundred users touch the feature, half of them use it once and never again, and nobody on the team can tell you whether the model was wrong or the feature was wrong. The eval scored the string. The users were reacting to something else.

This is a working guide on how to run user research on AI features: what to test that an eval cannot, how to write seed questions that surface adoption and trust instead of opinion of the model, how to set probe depth to catch the edge cases that decide the feature's fate, where to place the study so it collects signal continuously, and how to read the synthesis at the layer the team actually makes decisions.

What user research on AI features means

User research on AI features is a qualitative research method for evaluating the parts of an AI feature that an offline eval cannot see: whether a user recognizes what the feature does, whether they trust the output enough to act on it, whether the failure modes they encounter are recoverable, and whether the feature earns a place in a workflow they already have. The unit of analysis is not the model output. It is the participant's decision to keep using the feature, once, twice, and on the tenth Tuesday it matters.

The category is distinct from three adjacent practices. It is not a model eval (an eval measures the string the model returned against a rubric; user research measures what a person did with that string). It is not a usability test on the UI wrapper (a usability test catches whether the button is findable; user research catches whether the button is worth pressing when the model output is imperfect). And it is not a satisfaction survey (a survey collects a number; user research collects the reasoning behind the number, which is the part that tells the team what to change).

The related AI-powered async user research piece covers the broader category Talkful sits inside. This guide is narrower: how to research the feature you are building, not the method you are using to research it.

Why AI features break traditional research

Three ways AI features break the standard research playbook, and one that surprises teams every time.

The first answer is almost never the real answer. Ask a user "how do you feel about the summary the AI wrote?" and you will get "it was fine." That is the polite answer, and it is worthless. The real answer, which is that they rewrote three sentences before sending it and lost trust for two days after that, only surfaces when a probe follows the polite answer. A survey ends at "fine." A moderated interview probes past it. An asynchronous method with no follow-up loses the second turn, which is where every AI-feature decision actually lives.

The failure modes are long-tail and specific. A conventional feature has a small number of ways to break, and a usability test with five participants finds most of them. An AI feature has a distribution of failure modes that stretches into the hundreds: refusal on a legitimate request, hallucination on a niche topic, confident-but-wrong on a domain the model was undertrained on, tone that is off for the context, output that is technically correct but useless for the workflow. Five participants cannot cover that tail. The research design has to be continuous, not one-shot, or the team ships blind to the failures that only surface on the twentieth participant.

Trust is asymmetric and slow to rebuild. A single hallucination on a critical prompt costs more trust than a hundred correct answers earn. Once a user has been burned by an AI feature, their behavior changes in ways that are invisible to product analytics (they stop asking the questions they used to ask, they double-check the output every time, they route around the feature for the tasks that matter). User research is the only method that surfaces the trust asymmetry, because it asks about the near-miss and the workaround, not just the success case.

And the surprise: the model gets better while the feature gets worse. The team ships a new version, the eval scores climb, and the qualitative signal from users drops. The reason is almost always that the new version changed the failure surface in a way the eval did not measure. The feature is now correct more often but wrong in a way that is harder to recover from. Only user research surfaces this, because only user research asks what happened after the model was wrong.

How to run user research on AI features, step by step

Six steps, in order. Step one is the one most teams skip and the one that most often decides whether the study produces a decision or a document.

01 · Separate the model from the feature it wraps

Before writing any seed questions, write two sentences: what does the model do, and what does the feature do around it. The model summarizes a long email. The feature is a button on a message thread that generates a summary, shows it inline, lets the user edit it, and pastes the edited version into a reply. Those are two different things, and traditional evals only measure the first.

The team's research question almost always sits at the feature layer, not the model layer. Whether the summary is accurate is a model question (eval it). Whether the summary is worth pressing the button for on a Wednesday morning when the user has thirty messages is a feature question (research it). Write the research question at the layer where the team's decision lives. If the question is "is the model good enough," you do not need this method. If the question is "will the user press the button on Wednesday, and if the summary is wrong, will they press it again on Thursday," this method is the one.

02 · Write seed questions around observable behavior, not opinion of the AI

The single biggest craft mistake in AI-feature research is asking users what they think of the AI. That question invites a review, and reviews are polite. The stronger seeds anchor to a specific recent moment where the user encountered the feature, and ask them to walk through what they did.

• "Walk me through the last time you used the summary feature. What were you doing right before, and what did you do right after." This surfaces the workflow the feature sits inside.
• "Tell me about the last time the AI got something wrong. What did you notice, and what did you do next." This surfaces the failure recovery pattern, which is where trust is built or lost.
• "Describe a task you thought about using the AI feature for and decided not to." This surfaces the trust ceiling, which shows up as tasks the user does not delegate.

Notice what the seeds have in common. Each one anchors to a real moment, each one is open, and none of them ask the participant to evaluate the AI. The AI is present in the story but not the subject of the question. The companion post on writing user research questions covers the prompt-craft rules in more depth; for AI features the rule of thumb is: research the behavior the AI feature enables, not the AI itself.

03 · Set probe depth to the failure mode you are trying to catch

Adaptive probing on Talkful ships three depth settings (shallow, medium, expert), and the choice is per question rather than per study. For AI features, the depth-to-goal mapping is more specific than for general product research, because different questions in an AI-feature study are chasing different failure modes.

Shallow (at most one clarifying probe). Right for in-product feedback links attached to the AI feature itself ("was this summary useful? tell us why"). The user is a click away from disengaging; shallow keeps the response rate up while still recovering the reasoning behind the number. Placement in an activation moment or a first-use debrief also fits shallow.

Medium (a short chain of two or three probes). The default for standing product-discovery interviews on an AI feature. Enough depth to chase "the summary was fine" into "the summary was fine but I did not use it because I did not trust the tone for an executive audience," which is a different problem. Almost every AI-feature study has at least one medium-depth seed question.

Expert (probe until the answer contains the same context a senior researcher would dig out). Reserve for switch interviews with users who stopped using the feature, for internal stakeholder reviews with engineering and design before a launch, and for jobs-to-be-done research on the workflow the feature slots into. Expert is where the long-tail failure modes surface, because expert-depth probing keeps asking "and then what happened" until the participant has described the entire recovery arc.

The companion post on AI moderated user interviews covers depth calibration in general; for AI-feature research the specific pattern is: shallow for continuous in-product feedback, medium for weekly discovery, expert for switch interviews and internal review.

04 · Place the study where the AI feature actually gets used

The placement of the study link decides whether it collects the workflow the feature sits inside or a laboratory abstraction. For an AI feature specifically, the highest-signal placements sit at the moments the feature is either invoked, avoided, or abandoned.

• Inline with the AI feature output. A one-tap "tell us why" link attached to every model output. Shallow depth, one seed question, standing capture. Collects the reasoning behind adoption rate without a separate study.
• The moment the feature was not used. A prompt on the surface where the AI feature was available but the user did something else. This is the hardest placement to build and the highest-value one. It surfaces the trust ceiling directly.
• Immediately after a failure recovery. When the model returned something the user rejected (thumbs-down, regenerate, manual edit that erased most of the output), the same-session prompt catches the reasoning while the frustration is still specific. Medium depth.
• Post-onboarding or activation. After the user's first successful use, a prompt on the first-use debrief. Medium depth, medium participant investment.
• Switch or churn. For users who used the feature five or more times and then stopped, an outbound link with expert-depth probes. Expensive per response, priceless per insight.
• Internal stakeholder review. Before shipping a new version of the feature, a link inside the company for engineering, design, support, legal, and exec to answer in voice, text, choice, or rating. Expert depth. Produces a synthesized view of objections before the team ships.

The longer treatment of the standing-instrument shape is in the continuous discovery interviews piece; for AI features the operational version is shorter: at least one of the placements above should be capturing signal at all times, so the team is never guessing at the trust curve.

05 · Watch the first ten responses live, then tune

The temptation, once the study is published, is to open the dashboard a week later and read the synthesis. For an AI-feature study, resist it harder than usual. Watch the first ten responses as they land, with the audio on and the transcript open, and judge three things.

Are the probes the AI moderator is choosing the probes a human researcher would have chosen? For an AI-feature study the failure mode to watch for is over-probing on satisfaction seeds (the model does not need to be told the summary was fine six different ways) and under-probing on the "what did you do next" seed (which is the one that carries most of the signal).

Is the synthesis engine surfacing themes at the feature layer or the model layer? Themes that read like "the model was helpful" are a signal that the seed questions collapsed the participant back into an AI review. Themes that read like "users edit the summary before pasting it into their reply, and half of them stop trusting the tone after the second edit" are the shape you want.

And, specifically for AI features: is the study surfacing the failure modes the eval already caught, or the ones the eval missed? The first is confirmation; the second is the whole reason you are running the study.

"The first summary was great, so I used it on my Monday morning digest and it dropped a sentence I actually needed. Now I read the original anyway, and I only paste the summary if I have already read the thread. Which kind of defeats the point."

That is the workflow signal. The eval scored the sentence-drop as a minor error. The user reorganized their morning around it.

06 · Read the synthesis at the assumption layer, per decision

Once the depth is tuned and responses are flowing, real-time synthesis pulls themes, quotes, and citations across every participant as they land, with each quote linked back to the source transcript and audio. For an AI-feature study, read the synthesis at the assumption layer, not the response layer.

Before the study starts, write the assumptions the feature is built on: users trust the output enough to act on it without checking, users invoke the feature at least once a week after activation, users recover from failure without abandoning the feature, users prefer the AI-drafted output over their own writing enough of the time to justify the interruption. Each assumption becomes a lens on the synthesis. For each, ask: do the responses support, contradict, or fail to address it? Three buckets, one decision per assumption.

The synthesis playbook covers the general practice. For AI features the specific move is that failure-mode themes get their own read: which failure modes did the study surface, are they recoverable, and at which frequency does the failure cost more trust than a hundred successes earn back? That last question is the one that decides whether the feature ships as-is, ships with a smaller surface, or gets pulled back for a model change first.

When traditional methods still win

Three cases where user research on AI features is not the right method.

Pure model evaluation. If the question is whether the model returns the correct answer on a rubric, that is an eval question. Run the eval. User research is the wrong tool for measuring model accuracy in isolation. Anthropic's guidance on building evals covers the model-layer case at length.

Highly sensitive or regulated prompts. Legal, medical, financial, or safety-critical AI features benefit from moderated live interviews where the researcher can pause the probe if the participant hesitates. Default to shallow depth at most on sensitive seeds, or move to a moderated method.

A first-run pilot with three internal users. Sometimes the right first move is to hand the feature to three teammates for a week and read their reactions in Slack. If you have not done that yet, do it before you launch a research study. Async instruments amplify signal; they do not create it out of nothing.

What changes about the practice

Three second-month observations, worth naming because they only show up once a team has actually run this method on a shipping AI feature.

The unit of research moves from the model to the workflow. The team stops asking "is the model good enough" and starts asking "what did the user do next." The seed questions get shorter. The probe chains get longer. The synthesis themes stop reading like AI reviews and start reading like workflow adjustments.

Continuous feedback beats one-shot studies for AI features specifically. Because the failure surface is long-tail, the twentieth participant surfaces something the first five never touched. Standing feedback placements (inline, post-failure, post-activation) collect the long tail at zero marginal researcher cost. The full case for continuous placement is in continuous discovery interviews; for AI features it is not a stylistic preference, it is a requirement of the failure distribution.

Internal stakeholder review becomes a launch gate. Before the team ships a new version of an AI feature, a link goes out to engineering, design, support, legal, and exec, and every stakeholder answers three to five seed questions at expert depth. The synthesis produces a cross-functional view of objections before the team ships, and the objections that surface are almost always more specific than what a launch review meeting would have produced. Talkful teams treat this as the last step before publishing.

FAQ

What is user research on AI features?

User research on AI features is a qualitative method for evaluating whether users invoke an AI feature, trust its output enough to act on it, and recover from its failures. The unit of analysis is the participant's behavior around the feature, not the model output. Participants answer in voice, text, choice, or rating, an AI interviewer probes thin answers in real time, and a synthesis engine streams themes, failure modes, and quotes back as the responses land, with each theme linked to the source transcript.

How is this different from a model eval?

An eval measures whether the model returns the correct string on a rubric. User research on AI features measures what a person did with that string in a real workflow. A model can pass an eval and fail with users if the failure recovery is bad, the tone is wrong for the context, or the invocation cost is too high for the payoff. Evals are necessary and not sufficient. User research is what closes the gap between "the model is accurate" and "the feature is adopted."

How many participants do I need to study an AI feature?

Six to twelve responses on a homogeneous group reaches thematic saturation for most single-assumption studies, per the standard qualitative guidance. AI features are the exception where the long-tail failure surface justifies a continuous placement that accumulates responses over weeks. For a one-shot study on a specific assumption, ten to fifteen completed responses is enough. For a standing feedback placement inline with the feature, the count accumulates and the synthesis updates as responses land.

Should I run the study before or after launch?

Both, and the shape is different. Before launch, run an internal stakeholder review with engineering, design, support, and exec at expert depth to surface objections while the feature can still change. After launch, run a continuous placement inline with the AI feature at shallow depth to collect the adoption and trust signal, and reserve expert-depth switch interviews for users who used the feature and stopped. The pre-launch study catches the objections; the post-launch study catches the reality.

How do I write seed questions for an AI feature without leading the participant?

Anchor each seed to a specific recent moment ("walk me through the last time you used the feature"), keep the model out of the question ("what did you do next" rather than "what did you think of the AI"), and open the seed wide enough that the probe carries the depth ("tell me about the last time the feature got something wrong" surfaces the failure recovery pattern without prescribing it). The companion post on writing user research questions covers the general prompt-craft rules. For AI features the specific rule is: research the workflow the feature enables, not the AI itself.

Can I use the same study for external users and internal stakeholders?

Yes, and it is often the right move. The same study link, with the same seed questions and the same depth settings, can be sent to users externally and shared inside the company for engineering, design, support, legal, and exec review. The synthesis engine sorts by cohort so the team sees the internal objections and the external adoption signal side by side. This is how most Talkful teams run pre-launch and post-launch reviews on the same feature.

---

AI features are not the first product surface where the eval and the outcome disagree, but they are the surface where the disagreement is loudest and the cost of missing it is highest. Running user research on AI features closes the gap between "the model works" and "the feature is worth pressing on a Wednesday morning." Talkful ships configurable probe depth, multimodal participant answers, and real-time synthesis on the free plan, and the wider voice user research guide covers the habit once the first study is in flight.