Erik Caldwell
Connect on LinkedIn
Menu

The Topic Catalog · 18

AI-Native Product Design

Last updated · 8 min read

18.1 Beyond the Chatbot: What AI-Native Products Look Like

Priority: Must Understand

Executive Definition: Bolting a chat window onto an existing application does not make it AI-native: it's a conversational veneer over the same predetermined workflow, menus, and forms the product always had. An AI-native product is architected around users specifying a desired outcome and the system determining and executing the steps, tool calls, and data lookups needed to reach it: collapsing multi-screen, multi-click workflows into a single request-and-result loop, with the underlying UI adapting to what the task actually requires rather than to a fixed navigation tree.

Why It Matters: Most "AI transformation" initiatives at incumbent software companies amount to adding a chat sidebar that answers questions about a product whose core interaction model (click through screens, fill forms, follow a fixed sequence) is unchanged; this captures a fraction of the available value and is trivially copyable by any competitor. The organizations capturing real advantage are redesigning the product's control flow itself: the interface exists to let a user state an outcome and to make the system's actions and constraints legible, not to route them through a menu tree that was designed for a pre-AI world.

What I Need to Understand:

  • A chatbot layered onto an unchanged application is a distribution channel for the same workflow, not a redesign: the test is whether removing the chat window would leave the product's actual task-completion path unchanged (if yes, it isn't AI-native).
  • AI-native design shifts the unit of interaction from "screen and click" to "goal and result": the user states what outcome they want, and the system plans and executes the underlying steps (which may span multiple tools, records, or approvals) rather than requiring the user to navigate to each one.
  • This shift changes what UI is for: less about presenting every option up front, more about surfacing the system's plan, actions taken, and any decision points that need the human: which is why Topics 7 and 8 (approval UX, provenance/confidence UX) are inseparable from this one.
  • AI agents are increasingly consuming your product's interfaces directly, not just humans: interface clarity, semantic structure, and predictable behavior are now a functional requirement for machine users as well as human ones, not just an accessibility nicety.
  • Retrofitting an AI-native interaction model onto a workflow-first legacy architecture is usually harder than it looks, because the underlying system (permissions, data model, audit trail) was built assuming a human clicks through a known sequence, not that an agent takes multi-step action on the user's behalf.

Questions I Should Be Able to Ask My Team:

  1. For our flagship AI feature, if we removed the chat interface, would users still be forced through the exact same click-path they used before, and if so, what have we actually changed?
  2. Which of our products could plausibly let a user state an outcome ("close out this customer's account and confirm no open balances") instead of navigating a multi-screen workflow, and what's stopping us from building that today?
  3. Are we designing our APIs and interfaces to also be usable by AI agents acting on a user's behalf, or only by humans clicking through screens?

Technologies / Standards / Companies to Know: Agentic AI architectures, tool-calling/function-calling LLM patterns, model context protocols for agent-to-system integration

Recommended Learning:

Time Investment: 2-3 hours

18.2 Progressive Autonomy & Approval UX

Priority: Must Understand

Executive Definition: Progressive autonomy is the design principle that an AI agent's authority to act without human sign-off should scale gradually (with track record, reversibility of the action, and stakes) rather than being an all-or-nothing switch. Approval UX is the mechanism for that: it makes visible what the system is about to do, why, and what happens if the human does nothing, and it is what separates "AI that drafts a recommendation" from "AI that takes an irreversible action on your systems of record."

Why It Matters: The single most common failure mode in agentic AI deployments is granting broad autonomy before the organization has any track record with the narrower version: the cost of a bad autonomous action (a wrong email sent, an incorrect refund issued, a record silently modified) is rarely symmetric with the cost of asking for confirmation, and getting this wrong is both a trust-destroying incident risk and, per the EU AI Act, potentially a regulatory one for higher-risk use cases.

What I Need to Understand:

  • Autonomy should be modeled as a spectrum with defined levels (e.g., recommend-only, act-with-confirmation, act-with-notification, fully autonomous within bounds), not a binary "AI can act" toggle: several published frameworks (analogous to vehicle autonomy levels) formalize this and are useful as a shared internal vocabulary.
  • The right autonomy level for a given action depends on reversibility and stakes, not on model capability: a highly capable model doing something irreversible and high-stakes still warrants a checkpoint; a low-stakes reversible action doesn't need one even from a less capable model.
  • Approval UX has to show the plan, not just ask for a yes/no: a confirmation dialog that doesn't let the user see what specifically will happen (which records, which recipients, which amount) is a rubber stamp, not oversight.
  • Default friction should be intentional and preserved in some contexts (compliance-sensitive actions, financial transactions) even as autonomy elsewhere increases: removing friction indiscriminately in the name of "frictionless AI" recreates the exact failure mode this principle exists to prevent.
  • Autonomy level should be an explicit, changeable configuration tied to demonstrated reliability in your own environment, not a fixed vendor default accepted at install time.

Questions I Should Be Able to Ask My Team:

  1. For each AI agent with write access to production systems, what autonomy level does it operate at, and what evidence justified that level rather than a more conservative one?
  2. When our agents ask for human approval, does the interface show the actual plan and its consequences, or just a generic confirm/cancel button?
  3. What is our process for increasing an agent's autonomy level over time, and what would trigger us to reduce it after an incident?

Technologies / Standards / Companies to Know: Human-in-the-loop / human-on-the-loop patterns, agent action logging and audit trails, "Levels of Autonomy for AI Agents" taxonomy

Recommended Learning:

Time Investment: 2-3 hours

18.3 Citations, Provenance, Confidence Indicators & AI Failure UX

Priority: Monitor

Executive Definition: This is the design discipline of making an AI system's uncertainty and sourcing visible and actionable: showing where an answer came from (provenance), how confident the system actually is (calibrated confidence, not decorative), and what to do when it's wrong (failure UX): rather than presenting every output with the same uniform, polished confidence regardless of accuracy. Nielsen Norman Group research documents that current chatbot interface conventions actively work against this, making outputs look more trustworthy than they are.

Why It Matters: Published research finds generative AI hallucination rates in the range of 13.5%–33% depending on task, yet interface conventions (confident tone, polished formatting, warnings that scroll off screen) create a "halo effect" that discourages users from verifying outputs: meaning the UX itself, not just model accuracy, is a source of real business risk when unverified AI output reaches customers, financial records, or compliance-sensitive decisions.

What I Need to Understand:

  • NN/g research shows chatbot interfaces are currently designed in ways that discourage error-checking (authoritative tone, formatting polish, disclaimers that disappear from view): this is a design failure, not solely a model-accuracy failure, and is fixable independent of model improvements.
  • The rule of thumb from NN/g's research on trust: users should only rely on AI output they can verify or already know to be true, and should stay within their own domain of expertise when accepting AI claims at face value: a principle that should show up in your internal AI usage guidance, not just in UX design.
  • Verification should be designed to be as low-effort as generation: inline source links, clickable claims, and prompts that invite scrutiny ("what's your confidence here," "what would change this answer") outperform generic disclaimers.
  • Confidence indicators are only useful if calibrated to actual accuracy; a system that displays uniform confidence regardless of correctness is worse than no indicator, because it actively misleads.
  • Failure UX (what happens when the AI is wrong, unavailable, or refuses) deserves as much design investment as the success path: most AI product reviews focus entirely on the happy path.

Questions I Should Be Able to Ask My Team:

  1. In our customer- or employee-facing AI features, is there any confidence signal at all, and if so, has anyone validated that it's calibrated to actual accuracy rather than decorative?
  2. Do our interfaces make it easy to trace an AI claim back to its source, or does the user have to take the output on faith?
  3. What does our product do when the AI is wrong, uncertain, or can't complete a request: has that path been designed deliberately, or does it default to a generic error?

Technologies / Standards / Companies to Know: RAG (retrieval-augmented generation) citation patterns, confidence calibration techniques, NN/g chatbot design guidelines

Recommended Learning:

Time Investment: 1 hour