A Practical Taxonomy of AI Product Failures

“The AI was wrong” is a useful alarm and a poor diagnosis. The same customer-visible symptom can begin in retrieval, prompting, model routing, tool use, application state, or the interface around the model. Labelling the first plausible cause too early sends the investigation down a narrow—and often expensive—path.

This taxonomy is for triage. It helps a team turn a Source Report into a testable Failure Hypothesis, decide which Evidence to collect, and record the result in a Repro Pack. A category is not a Confirmed Root Cause. That requires decisive Evidence or engineering verification.

How to classify an incident

Start with the customer’s symptom and the execution that produced it. Then record:

• one primary category and any credible alternatives;
• the failure stage, such as retrieval, tool use, generation, or display;
• the mechanism, such as missing, stale, incorrect, unauthorised, truncated, or timed out;
• confidence and the Evidence supporting it; and
• the next check that would separate the leading explanations.

Use “insufficient Evidence” when the execution cannot be found or the required telemetry was not retained. An absent trace does not prove that an operation did not happen.

The regression checks below describe behaviours worth protecting once the expected outcome is agreed. For generative output, assert stable criteria rather than one exact sentence.

1. Retrieval failure

The system retrieved context that was missing, irrelevant, stale, incorrectly scoped, or too low-quality to support the task.

Observable symptoms: The answer ignores a known document, cites an old policy, uses the wrong customer record, or says information is unavailable even though it exists in the authorised source.

Evidence that separates it: Inspect the effective retrieval query, filters, ranked results, document IDs and versions, index or sync timestamps, and the source-of-truth record. A strong retrieval Failure Hypothesis needs a visible break before generation—for example, the correct policy is current in the source system but absent from the retrieved results. If the correct evidence reached the model and the final answer contradicted it, look instead at tool-result handling or hallucination.

Common confounders: A permission filter may have correctly hidden the document. Query rewriting may have changed the user’s intent. The right passage may have been retrieved and then truncated, or the product may have promised coverage the knowledge base never provided.

Regression check: Given the representative question, tenant scope, and a fixed document set, the expected source appears within the accepted retrieval range and the answer is grounded in the current version.

2. Prompt regression

A change to a system prompt, instruction, template, or prompt-building step degraded previously accepted behaviour.

Observable symptoms: A constraint, response format, tool-use pattern, or workflow stops working near a prompt deployment. Similar inputs behaved acceptably before the change.

Evidence that separates it: Compare a failing run with a healthy control using the same input, stable context, model configuration, and product state. Record the stored prompt version and the effective prompt actually sent to the model; middleware may rewrite or omit instructions. A before-and-after difference tied to the prompt change supports the hypothesis. A recent deployment timestamp alone does not.

Common confounders: Model routing, retrieval contents, feature flags, account settings, or tool schemas may have changed at the same time. A one-off stochastic output is not enough to establish a regression.

Regression check: Run the known example against the revised prompt and assert the required behaviour or rubric criteria. Keep the model, context, and configuration controlled so the test isolates the prompt change.

3. Model or provider regression

A change in the requested or actual model, provider, inference settings, router, or fallback path degraded quality or reliability.

Observable symptoms: Behaviour shifts after a model migration, outputs differ across regions or routes, or failures cluster around fallback traffic or a provider incident.

Evidence that separates it: Capture both requested and actual model identifiers, provider, routing decision, fallback path, inference parameters, deployment time, and comparison runs against a known-good configuration. Repeated evaluation failures under the new configuration—while the input, effective prompt, tools, and context remain stable—support a regression hypothesis.

Common confounders: Prompt or retrieval changes can coincide with a model release. Sampling makes individual outputs vary, even with the same settings. A single unsupported answer is evidence of a generation failure in that execution, not proof of a broad model regression.

Regression check: Evaluate the same representative cases against the old and new configurations using behaviour-based criteria. Record the actual route for every run and compare distributions, not just one favourable output.

4. Tool-call failure

An agent omitted a required tool call, chose the wrong tool, supplied invalid arguments, or failed to recover from a rejected call.

Observable symptoms: The assistant guesses information it should have looked up, performs the wrong action, asks for data a tool already accepts, or stalls after a validation error.

Evidence that separates it: Reconstruct the tool decision from the tools made available to that execution, their schemas, the selected call, arguments, validation response, retry sequence, and routing instructions. If the required tool was available and the trace shows no call—or shows an invalid call that was not recovered—the failure occurred before a useful result existed.

Common confounders: The tool may have been unavailable because of permissions, feature flags, or account state. A successful call followed by a wrong answer belongs under tool-result handling. A timeout after dispatch belongs under latency or timeout, even if the customer symptom looks similar.

Regression check: Given the representative request and authorised tool set, the agent selects the required tool, produces schema-valid arguments, and handles a recoverable validation error according to the workflow contract.

5. Tool-result handling failure

A tool returned useful data, but a later transformation, model step, or application mapping ignored, reversed, truncated, or misrepresented it.

Observable symptoms: The source system and raw tool response say one thing while the assistant reports another. For example, a billing tool returns paid and the final answer says overdue.

Evidence that separates it: Follow the value from the raw tool response through parsing, transformation, model context, generated output, and displayed result. This category becomes especially strong when the first incorrect value is visible at a specific boundary. An explicit application mapping from paid to unpaid, for example, can establish a Confirmed Root Cause after engineering verifies the trace and code path.

Common confounders: The raw response itself may be stale or scoped to the wrong tenant. The result may never have reached the model because of context limits. The model may have received the correct value and generated an unsupported contradiction, which is better classified as hallucination or generation failure.

Regression check: Feed a fixed, representative tool result through the same transformation path and assert that its material facts survive into the user-visible answer or action.

6. Hallucination

The generated output contains a material claim that is unsupported by the Evidence available to that execution.

Observable symptoms: The assistant invents a policy, citation, date, calculation, product capability, or account fact—or confidently contradicts every supplied source.

Evidence that separates it: Compare each disputed claim with the complete context the model received: retrieved passages, tool results, system instructions, conversation state, and authoritative references. If the correct inputs reached generation, no later mutation changed the response, and the claim has no support, “hallucination in this execution” is a defensible classification.

Common confounders: Missing or stale retrieval can starve the model of the right facts. A post-processing or display layer can alter an otherwise supported output. Ambiguous expected behaviour may turn an apparent hallucination into an expectation mismatch.

Regression check: For a stable evidence set, require the assistant to ground material claims in that evidence, abstain when support is absent, and avoid the known unsupported assertion. Do not require the exact original wording.

7. Refusal or policy failure

The system refuses an allowed task, complies with a task the product should block, or applies the wrong policy for the account, locale, or context.

Observable symptoms: A routine request receives a safety refusal, a restricted operation proceeds, or equivalent users receive inconsistent policy outcomes.

Evidence that separates it: Collect the applicable policy and moderation versions, classifier output, refusal reason, effective system instructions, user intent, locale, account context, and final response. The key comparison is between the policy that should have applied and the decision actually made—not merely whether the output contained refusal language.

Common confounders: The user may lack product permission rather than trigger a content policy. A tool validation error can be phrased as a refusal. The policy may be working as designed while product copy created a broader expectation.

Regression check: Maintain paired allowed and disallowed examples for the same policy boundary. Assert the expected disposition and essential explanation while allowing non-material wording variation.

8. Memory or state failure

Required conversation, workflow, or account state was missing, stale, associated with the wrong turn, or applied out of order.

Observable symptoms: The assistant forgets a confirmed preference, repeats a completed step, answers from an earlier turn, or applies state belonging to another session.

Evidence that separates it: Trace the relevant state writes and reads using conversation, session, and workflow identifiers. Inspect summarisation, expiry, ordering, retries, cache behaviour, and the exact state supplied to the failing execution. A state hypothesis is supported when the authoritative write exists but the subsequent execution reads an older, missing, or incorrectly associated value.

Common confounders: The state may never have been persisted by design. Retrieval may have returned stale information that looks like memory. The user’s expectation of long-term memory may exceed the product’s documented behaviour.

Regression check: Execute the representative multi-turn or multi-step sequence and assert that the required state is written, read under the correct identity, and applied in order at the point where it matters.

9. Data-permission failure

The system exposes data outside an authorised scope or withholds data the person should be able to access.

Observable symptoms: A user sees another tenant’s record, receives an answer based on the wrong account, or cannot retrieve information available to peers with the same intended role.

Evidence that separates it: Compare authenticated tenant and user identity with permission checks, retrieval filters, cache keys, tool scopes, returned record ownership, and audit logs. A mismatch between the request’s authorised scope and the owner of the returned record is decisive Evidence of a boundary failure.

Common confounders: A legitimate role change may not have propagated yet. The source record may be missing rather than denied. User-expectation mismatch can look like under-permission when product roles are poorly explained.

Regression check: Test both sides of the boundary: authorised identities can access the intended synthetic record, and unauthorised or cross-tenant identities cannot retrieve, infer, or receive it through caches or generated output.

Treat any possible cross-tenant disclosure as a security incident under the organisation’s response process, not as an ordinary AI-quality ticket. Avoid reproducing it with real customer data.

10. Latency or timeout failure

A slow dependency, queue, retry, timeout, or fallback changes the answer, prevents completion, or returns a superficially successful but stale result.

Observable symptoms: The response never arrives, arrives after the product deadline, loses part of the requested work, or uses stale fallback data without making that limitation clear.

Evidence that separates it: Build an end-to-end timing breakdown covering queues, retrieval, tools, provider calls, retries, timeout thresholds, cancellation, and fallbacks. A timeout span followed by a documented stale-cache fallback explains both the operational event and the resulting quality failure; “the provider was slow” without timings does not.

Common confounders: Client rendering or network delivery may be slow after generation completes. A malformed tool request can fail quickly and then retry. A valid fallback may be working as designed, leaving an expectation or disclosure problem rather than a timeout defect.

Regression check: Introduce a controlled slow or failing dependency and assert the product’s time budget, retry limit, fallback choice, and user-visible disclosure. Test the response content as well as whether the request returned successfully.

11. User-expectation mismatch

The product behaves as designed, but the interface, documentation, examples, or prior product behaviour led the user to expect something else.

Observable symptoms: The requested outcome falls outside the product’s stated capability, an ambiguous label suggests stronger guarantees than the system offers, or support and engineering disagree on what “correct” means.

Evidence that separates it: Reconstruct the user’s intended job and compare the observed behaviour with current product requirements, interface copy, documentation, examples, and any established acceptance criteria. This category is appropriate only after checking that the system actually followed its design; it must not become a convenient label for closing difficult investigations.

Common confounders: Documentation may describe the intended behaviour while the deployed product violates it. A real retrieval, policy, or permission failure can masquerade as misunderstanding. Previous product behaviour may have reasonably established the expectation even if it was undocumented.

Regression check: Protect the clarified contract at the right layer. That may be an acceptance test for the intended behaviour, a content test for revised copy, or a usability check that representative users understand the limitation before acting.

Write the classification as a hypothesis

A useful Repro Pack does not stop at a label. It shows why the label currently leads and what would overturn it:

Primary: retrieval failure—stale context (medium confidence). The answer cites an older policy date, but the retrieval trace and document version are not yet available. Alternative: user-expectation mismatch if the requested policy was outside the assistant’s documented scope. Next check: locate the execution by account and time window, then compare the retrieved document version with the source system.

This gives engineering a concrete starting point without laundering uncertainty into a Confirmed Root Cause. If the next check cannot be performed, the honest outcome is an Evidence gap—and a precise request for the telemetry or identifier needed next.

Put the taxonomy to work

Use the AI bug report template to record the classification without overstating it, and follow the fictional stale-invoice Repro Pack for a worked example.

FaultTrail is building this support-to-evidence workflow with small AI SaaS teams. If it resembles a recurring problem in your product, explore the early pilot. Start by describing your workflow and systems; do not send tickets, traces, credentials, personal information, or confidential customer content.