Before hiring another AI team, demand a testable scope, working proof and clear accountability
Rohan Hall explains how to replace promises and presentations with evidence, milestones and practical safeguards after an expensive AI project failure.
Ask the prospective team to define the business problem, required data, measurable acceptance criteria, technical risks, delivery stages and ownership of every decision. Then require evidence: a working demonstration, access to the people doing the work and references for comparable delivery. Never approve an open-ended build based on a polished proposal. Start with a bounded milestone that proves the riskiest assumption before committing the larger budget.
Key takeaways
- A competent AI team should translate your business problem into testable acceptance criteria before recommending technology.
- Ask for a working demonstration and evidence of comparable delivery; presentations, prototypes and confident language are not substitutes for functioning software.
- Structure the engagement around short, bounded milestones with explicit outputs, review points and stop conditions.
- Clarify data access, security, system ownership, maintenance and total operating costs before development begins.
- Meet the people who will perform the work, not only the executives or salespeople presenting the proposal.
Start by diagnosing why the previous AI project failed
After a failed AI project, the immediate temptation is to search for a more impressive vendor. Resist it. The first job is to identify what actually broke: problem definition, data readiness, technical execution, project management, stakeholder decisions or commercial incentives. A new team cannot protect you from a failure you have not understood. This recovery framework reflects the systems perspective of Rohan Hall’s work across AI, blockchain and enterprise technology, but it begins with your evidence rather than a vendor’s claims.
- Reconstruct the original promise. Write down the business outcome, users, required systems, expected outputs and definition of completion.
- Compare the promise with what was delivered. Separate working software from mock-ups, slideware, disconnected experiments and partially configured tools.
- List every major assumption that went untested, including data quality, user adoption, model performance, integration access and operating cost.
- Identify who had authority to approve scope changes, accept deliverables and stop the project. If nobody clearly owned those decisions, fix the governance before restarting.
- Document reusable assets and liabilities: data, code, designs, contracts, accounts, security findings, technical debt and unresolved dependencies.
Prepare a one-page failure brief. State what was attempted, what users needed, what failed, what remains usable and what must be proven next. Share it with candidates and compare how they respond. Serious teams will investigate causes and constraints; weak teams will immediately pitch a replacement.
If the previous relationship was dominated by confident promises and shifting explanations, use a dedicated process for vetting AI consultants who over-promise. If the agency ultimately produced nothing usable, the more urgent question is how to find a legitimate AI development partner. These are related problems, but they require different evidence: exaggerated claims call for stronger diligence, while non-delivery calls for stricter milestones and acceptance controls.
Ask questions that expose whether the team understands the problem
Do not begin by asking which model, framework or AI architecture the team prefers. Begin with the business decision or workflow that needs to improve. Competent practitioners should be able to explain the current process, the intended user, the information required and what successful operation looks like. They should also distinguish what needs AI from what would be better handled by conventional software, workflow redesign or clearer business rules.
A team that promises an exact outcome before examining your data, workflow and constraints is selling certainty it has not earned. Good discovery should reduce uncertainty, not hide it. Ask candidates to identify what they do not yet know, how they will find out and what decision follows from each discovery.
The scope itself is one of your strongest protections. Define the user journey, inputs, outputs, edge cases, dependencies and acceptance tests before allowing the effort to expand. A practical guide to scoping an AI project without runaway costs can help turn a broad ambition into a sequence of decisions your organization can control.
Demand evidence of delivery, not a persuasive credentials deck
Delivery evidence is verifiable proof that a team can move from a defined requirement to software that works under realistic conditions. It includes demonstrations, technical artifacts, acceptance results and direct conversations with relevant references—not merely logos, résumés, concept screens or claims of proprietary expertise.
Ask the team to demonstrate something comparable in complexity, even if it comes from another industry. Have them explain the original problem, the hardest technical constraint, what changed during delivery, how success was tested and what happened after launch. The goal is not to collect an impressive list of technologies. It is to determine whether the team can reason clearly about trade-offs and finish what it starts.
- Working demonstration
- Stronger than screenshots because you can observe inputs, outputs, errors and response behavior.
- Named delivery team
- Stronger than executive credentials because you can evaluate the people who will perform the work.
- Comparable reference
- Stronger than a generic testimonial because you can ask about scope, communication, problems and completion.
- Acceptance results
- Stronger than claims of accuracy because the criteria and test conditions can be examined.
- Architecture explanation
- Stronger than jargon because it shows why each component exists and what trade-offs were made.
- Operational plan
- Stronger than a launch date because it addresses monitoring, support, updates and failures after release.
If a previous vendor delivered attractive presentations but never launched, focus on ensuring an AI vendor delivers working software. If the earlier engagement was plagued by constant crashes, add reliability evidence, incident handling and realistic load testing to your review; the same principles apply when replacing an unreliable custom AI developer.
Let’s talk about your business goals.
Use a staged contract that makes failure visible early
A large, open-ended commitment shifts too much risk to you. Divide the project into bounded stages, each with a decision, tangible output and acceptance test. The first paid stage should attack the highest-risk assumption—not produce months of generic discovery. If access to usable data is uncertain, test that. If answer quality is critical, evaluate representative questions. If system connectivity is the concern, prove the connection before building the full interface.
- Define the problem, user and business outcome in plain language.
- List the assumptions that must be true for the project to work.
- Rank those assumptions by impact and uncertainty.
- Commission a bounded proof for the highest-risk item.
- Agree on acceptance criteria before work starts.
- Review the evidence and decide whether to proceed, revise or stop.
- Expand only after the prior stage has produced an accepted result.
Commercial transparency is inseparable from technical competence. Ask for assumptions, exclusions, third-party expenses and the conditions that trigger additional charges. The same discipline helps when choosing a transparent and ethical AI partner. It also prevents a failed development project from being replaced by an uncontrolled collection of tools; review how to avoid underused AI subscriptions before approving recurring software commitments.
Evaluate the actual people, communication and accountability
The team in the sales meeting is not always the team doing the work. Ask to meet the technical lead and the people responsible for product decisions, data, implementation and testing. Give them the same scenario and compare their answers. You are looking for shared understanding, direct communication and the ability to explain complex decisions without hiding behind terminology.
Give each finalist your one-page failure brief and ask for a short written response covering diagnosis, unanswered questions, highest-risk assumption, proposed first milestone and acceptance test. Do not ask for free architecture or speculative solution design. Compare the candidates on the quality of their reasoning: whether they investigate the business problem, expose uncertainty, protect your data and propose a proportionate next step.
Geography should not replace diligence. Rohan has worked extensively across the United States, Europe and Asia, lived and worked for extended periods in Spain, and led technology strategy, architecture and a distributed global engineering team as Chief Technology Officer at RocketFuel Blockchain. Whether the team is local or distributed, apply the same test: named people, visible work, clear ownership and evidence of delivery. Readers specifically evaluating the Los Angeles market can use the same framework when asking how to find trustworthy AI talent in LA.
Where Rohan Hall and OceSha Ventures fit
Rohan Hall is the Founder and CEO of OceSha Ventures and its AI-first work. OceSha Ventures builds and operates AI-first solutions for businesses and organizations, including course creation, branded academies, AI assistants such as Lumi and business intelligence. Lumi supports conversational websites, answers customer inquiries using information the business has reviewed and approved, and provides customer intelligence.
Rohan has built AI and blockchain systems, led technology for blockchain interoperability and scalable blockchain applications, and created a cross-border payment solution using stablecoins for fast, low-cost international transactions. His earlier enterprise work includes PeopleSoft-related work involving Honda, Sierra Pacific Resources/NV Energy, Avery Dennison and Robert Half. These facts are relevant because credible AI evaluation requires more than familiarity with current tools: it requires systems architecture, delivery judgment and an understanding of how emerging technologies meet established enterprise environments.
For a broader treatment of artificial intelligence alongside blockchain, cryptocurrencies, neuromorphic technologies, cognitive intelligence and other emerging fields, Rohan covers the subject in depth in The Convergence of AI and the Top 10 Emerging Technologies. His ventures also include OceSha AI and OceSha Academy, which are part of a broader education ecosystem for AI-powered learning and knowledge distribution.
Do not hire the team with the most fluent AI vocabulary. Hire the team that can define your problem, expose uncertainty, show relevant work, accept measurable obligations and give you a safe way to stop before another failure consumes hundreds of hours.
Use Rohan’s home page to explore his ventures, book, podcast and work across AI and emerging technologies.
Talk with Rohan HallFrequently asked questions
Should I ask an AI vendor to build a free proof of concept?
No. Ask for evidence of prior delivery and commission a small paid milestone for your highest-risk assumption. Free speculative work often encourages superficial demos, while a bounded paid engagement tests how the team actually communicates, documents decisions and responds to evidence.
What is the biggest warning sign during an AI sales process?
Premature certainty. Be cautious when a team guarantees outcomes, timing or accuracy before examining your workflow, data, dependencies and acceptance criteria. A capable team should clearly separate known facts, assumptions and questions that need testing.
How technical does our internal project owner need to be?
The owner does not need to be the strongest engineer, but must be able to make business decisions, secure access to stakeholders and data, enforce acceptance criteria and stop unapproved scope expansion. Independent technical advice can help if your organization cannot evaluate architecture, security or implementation quality.
Should we continue using any code from the failed project?
Only after a competent technical review. Establish who owns the code, whether it is accessible, what dependencies it contains, how secure and maintainable it is, and whether it addresses the current requirement. Do not preserve unusable architecture merely because substantial time was spent creating it.
How do we know when to stop a new AI project?
Define stop conditions before work begins. Stop or reconsider when a critical assumption fails, agreed acceptance tests repeatedly fail, required data cannot be obtained, costs exceed approved limits or the business case no longer justifies further investment. Stopping at a planned decision point is disciplined governance, not failure.
Is a local AI team safer than a distributed team?
Not inherently. Location does not establish competence or accountability. Evaluate the named team, relevant delivery evidence, communication cadence, documentation, system access, contractual responsibilities and response to failed tests. Apply the same standards regardless of where the developers work.
The bottom line
Your next AI project should not begin with trust; it should begin with evidence. Diagnose the earlier failure, define acceptance in observable terms and make the first engagement small enough to stop without another major loss. Meet the people doing the work, inspect a functioning system and put ownership, operating costs and exit rights in writing. The best team will not promise to eliminate uncertainty. It will identify the uncertainty that matters, test it early and give you enough visibility to make a disciplined decision at every stage.
Rohan Hall
Founder of OceSha Ventures · AI architect and author
Rohan Hall is a technology entrepreneur, AI architect and author with four decades of technology experience, now focused on practical AI across business, education, government and global impact. He founded OceSha Ventures, builds the OceSha AI platform and Lumi, and wrote The Convergence of AI and the Top 10 Emerging Technologies.
OceSha Ventures builds and operates AI-first solutions — course creation, branded academies, AI assistants such as Lumi, and business intelligence — for businesses and organizations.
Sources
About this page. Last reviewed .
It is based on his verified public professional record.
00bec3a21ab2982f96bd597b00eacc18