Scope & Method
First-run experience of "Aria," a specimen AI support copilot: signup → first prompt → first escalation to a human. Three sessions as three user postures (curious, task-driven, hostile), every path walked including the ones the team would rather I didn't take.
Probabilistic-UX principles for AI surfaces — disclosure, trust calibration, error-as-fork design, handoff architecture — plus baseline usability heuristics. Every finding carries severity, evidence, the trust cost, a concrete fix, and an effort estimate.
Three Specimen Findings
Finding 4 · Critical — the empty first turn
Evidence: the post-signup screen presents an empty input labeled "Ask Aria anything." In the task-driven session it took 41 seconds and two abandoned drafts to compose a first prompt; the curious session opened with "what can you do?" — which Aria answered with a capability list the onboarding had already shown.
Why it leaks trust: "anything" transfers the burden of scoping the product from the product to the user, and the first exchange — the moment trust is most fragile — gets spent on meta-conversation instead of value.
Fix: replace the empty state with three concrete, clickable first tasks drawn from the user's stated role at signup ("Draft a refund reply" · "Summarize this ticket thread" · "What's waiting on me?"). Keep free input present but secondary.
Effort: S — copy + one component. Priority: this sprint.
Finding 7 · High — confidence without provenance
Evidence: Aria answers policy questions ("what's the refund window for annual plans?") in fluent declarative prose with no source, no timestamp, and no confidence signal. The specimen's answer was plausible and out of date.
Why it leaks trust: every confident wrong answer draws down a balance the interface never explicitly topped up. Users who catch one hallucinated policy stop trusting the correct answers too — the cost lands on the whole product, not the wrong turn.
Fix: answers grounded in a document show its name and last-updated date inline; answers below the grounding threshold say so and offer the escalation path before the user has to distrust something. Show the work early, spend the credit never.
Effort: M — needs retrieval metadata surfaced to the UI. Priority: next quarter, flagged to engineering now.
Finding 11 · Medium — the dead-end apology
Evidence: in the hostile session, three consecutive misunderstood prompts each produced "I'm sorry, I didn't understand that. Could you rephrase?" — identical copy, no state change, no exit ramp. The session ended there; a real user's would too.
Why it leaks trust: deterministic software fails loudly and recoverably; AI fails politely and terminally. Treating "I didn't understand" as a terminal state loses the user at the exact moment they were still willing to try.
Fix: failure is a fork, not a wall — second consecutive miss switches strategy (offer interpretations: "Did you mean A or B?"), third miss offers the human, with the transcript attached so the user never repeats themselves.
Effort: S–M. Priority: this sprint — it's the churn moment.
What the Other Pages Hold
A full Critical Flow report runs 12–18 findings in this format, ordered by a priority matrix (user impact × engineering effort), plus: a one-page executive summary your PM can forward, an annotated walkthrough of the flow with timestamps, a "stop doing" list (features actively costing trust), and a 45-minute recorded walkthrough call where you're encouraged to argue with me. Two rounds of async questions in the following two weeks are included.
Client reports are confidential — showing you a real one would tell you exactly how I'd treat yours. The format, depth, and honesty above are real; "Aria" is a composite assembled from patterns that appear across shipped AI products. When the report about your flow lands, it looks like this — about things only your product does.