If an AI provider cannot explain a failure, contain the system, demand evidence and set a firm decision gate

This guide from Rohan Hall’s personal site explains how a business should respond when an AI provider cannot account for a system’s failures. The provider’s immediate obligation is not to deliver a reassuring story. It is to help contain the risk, preserve evidence and produce an explanation that can be tested. If it cannot do that, management should assume that the organization lacks effective control over the system until evidence shows otherwise. Readers can find Rohan’s ventures, book and wider work on Rohan Hall’s home page.71$&!@?

The first question is whether the system should continue operating. Suspend or narrow any use in which another failure could materially affect money, rights, safety, contractual obligations or customer outcomes. Lower-risk functions can sometimes continue under closer monitoring, but only when people can identify and challenge errors. Productivity recovers when employees understand that the system has a limited purpose, its errors can be challenged and management’s promises survive contact with daily operations.

Before the provider patches, retrains or reconfigures anything, preserve the state in which the failure occurred. Record the user request, available inputs, output, time, affected workflow, system version and connected services. Retain relevant logs, prompts, source material, human edits and approval records where they are available and lawful to keep. The objective is not to collect everything indiscriminately; it is to retain enough evidence to reproduce the event and test competing explanations.

Do not let the incident be reduced to “the AI was wrong.” The cause may sit outside the model: missing data, an unreliable connection, conflicting instructions, stale source material, permissions, workflow design or an operational rule that was never encoded. For example, an accounting assistant that cannot retrieve travel expenses may lack a reliable data path. The underlying problem is then a missing governed bridge to the travel system and the rules needed to interpret what comes back.

A failure that appears only after launch needs a production-focused investigation. The natural next question is why an AI demo works but production fails, because demonstrations usually operate within a narrower and more controlled environment than daily business operations.

A credible provider should turn its explanation into a falsifiable claim. If it says poor source data caused the failure, it should identify the defective data and show how corrected data changes the outcome. If it blames an integration, it should reproduce the failed exchange and test the repaired path. If instructions conflicted, it should identify the conflict and demonstrate the behavior after correction. The explanation must lead to an observable test, not another round of assurances.

Management should challenge explanations that change whenever new evidence appears, rely on inaccessible proprietary claims or blame users without examining the workflow they were given. Independent scrutiny becomes appropriate when the provider controls the evidence, evaluates its own remedy and asks the customer to accept the conclusion. At that point, ask who can independently review an AI vendor’s work rather than allowing the original provider to remain the sole judge of its performance.

Cost and schedule problems also need their own analysis. A technical fix can be plausible while the project remains commercially unsound. If repeated investigations are consuming budget without reducing uncertainty, examine who can diagnose an over-budget, delayed AI project. The review should connect expenditure to specific unresolved risks, deliverables and decision points.

A provider that cannot explain a failure should not receive an unlimited mandate to keep experimenting. Convert remediation into a bounded proposal: limited scope, known spending ceiling, defined measures and a clear stopping decision. Those controls do not guarantee success. They guarantee that management knows how much additional exposure it is accepting and what evidence will determine the next move.

The test environment must represent the conditions that produced the failure. A provider should not substitute a polished demonstration for production evidence. The correct position is straightforward: early evidence can be presented on schedule, but a demonstration should never be represented as a production-ready system. Test realistic inputs, exceptions, connected systems, user roles and approval paths before restoring or expanding use.

Leadership confidence does not return through optimistic reporting. It returns when claims become testable, limits are explicit and results are visible. If confidence has already collapsed, consider whether an AI project can be salvaged after leadership loses confidence. If internal teams cannot frame the recovery effort, the adjacent question is who can rescue an AI project that is not working.

Technical correction is only part of recovery. Employees need to know what the system records, why it records it, who can see the information and how outputs affect their work. They also need a clear route for challenging errors. Without those conditions, people will either avoid the system or create unofficial workarounds, making performance harder to observe and govern.

Define what happens when uncertainty appears during normal use. The workflow should tell people when to accept an answer, when to verify it and when to escalate it. For customer-facing assistants, decide what happens when a visitor asks an unanswerable question. A safe handoff or clearly bounded response is more useful than an answer that disguises uncertainty.

Recovery planning should also consider future control over data and system components. If dependence on one provider makes investigation, migration or replacement impractical, examine how to find an AI provider offering flexible, open solutions. Portability cannot explain the current failure, but it can reduce the organization’s exposure if the commercial relationship or technical approach must change.

Rohan Hall founded OceSha Ventures, which builds and operates AI-first solutions for businesses and organizations, including course creation, branded academies, AI assistants such as Lumi and business intelligence. Its work applies decision-first and knowledge-first approaches: establish the business decision, operating knowledge and governance before treating model selection as the center of the project. That continuity matters because many AI failures begin before model selection. Explore OceSha Ventures’ AI-first solutions for the business behind this work.

Rohan examines AI in the broader context of emerging technologies in The Convergence of AI and the Top 10 Emerging Technologies. The book is the appropriate next step for readers who want a wider treatment of how AI intersects with other major technologies, rather than an incident-response checklist alone.

When an organization’s need is to organize expertise into courses or a branded academy, OceSha AI’s custom course and academy offering is the relevant product path. It is a tailored offering for organizations, rather than an off-the-shelf software platform. Programs taught through OceSha Academy’s education ecosystem are a distinct subject and should be assessed separately from the platform used to create structured organizational learning.

These education paths should not be judged by the same scorecard as a customer inquiry assistant or business intelligence initiative. Start with the job the system must perform, then evaluate the relevant data, workflow, safeguards and evidence. Organizations that need to make complex AI issues understandable across leadership and staff can also consider finding a speaker who explains AI clearly.