Back to Academy
Intermediate
14 min
Bitcoin Operations

Reading an AI vendor proposal (what to ask, what to skip)

The eight questions that surface real capability vs. polished slideware. Print it. Bring it to your next vendor call.

AI Operations
AI Operations · Lesson 3 of 5

The vendor pitch landscape in 2026

If you're seriously considering AI for your business, you're probably hearing from several vendors. AI as a Service providers. Off-the-shelf SaaS companies with "AI" added to their feature list. Boutique consulting firms. Custom-build agencies.

Most of them sound good in the meeting. Most have polished slides. Most know the right buzzwords. The hard part is separating the ones who can actually deliver from the ones who can't.

This lesson is the eight-question framework that does that separation. Print it. Use it. Bring it to your next vendor call literally — the act of working through these questions in real time tells you almost everything you need to know.

The questions are arranged least to most diagnostic. The first few separate competent vendors from incompetent ones. The last few separate competent vendors from genuinely excellent ones.

Question 1: "Can you show me three current customers doing exactly this?"

Why it matters: The fastest separator between real capability and demo-grade pitches.

Good answers: A real vendor names companies (or describes them anonymized), summarizes what was built, and shares specific outcomes. Bonus: offers to connect you with a customer as a reference.

Bad answers:

  • "We've worked with hundreds of clients across many verticals."
  • "Our case studies are on our website."

If a vendor can't name three current customers in similar situations, they probably don't have them.

Question 2: "What does the deployment process actually look like, week by week?"

Why it matters: Reveals whether they've shipped AI deployments or only proposed them.

Good answers: A specific timeline with week-by-week milestones, named deliverables, and dependencies on your input.

Bad answers: "Implementation typically takes 4-8 weeks." (No detail.) "Our team will work closely with you throughout." (No structure.)

A vendor that can't describe their own deployment process is going to invent it as they go — at your expense.

Question 3: "What's the operational model after deployment? Who runs it day to day?"

Why it matters: Many AI vendors are great at building but terrible at operating. They hand you a finished deployment and walk away.

Good answers: Clear roles. "Our team monitors 24/7, handles model drift, manages updates." OR "We hand off operational ownership to your team with a 30-day shadowing period."

Both are valid. What's not valid is no answer.

Question 4: "What happens when the AI gets it wrong?"

Why it matters: Real AI deployments have failure modes. Vendors who've actually shipped have handled them.

Good answers: Specifics about error monitoring, escalation paths, fallback procedures, retraining cycles, accountability, response SLAs.

Bad answers: "The AI is very accurate." "We handle that during deployment." "It rarely gets things wrong."

The error-handling answer reveals more about a vendor's operational maturity than almost any other question.

Question 5: "How do you handle our data — both during deployment and ongoing?"

Why it matters: Security and compliance. The answer determines your legal and reputational risk.

Good answers:

  • Specific certifications (SOC 2, ISO 27001, HIPAA if relevant)
  • Clear policy: "We don't train on customer data" / "Data isolated per tenant"
  • Data residency commitments if relevant
  • A written DPA they'll sign

Bad answers: "Security is very important to us." Resistance to signing a DPA.

Security claims should be verifiable in writing, not just spoken.

Question 6: "What's the total cost of ownership over 12 and 24 months?"

Why it matters: Real costs are rarely visible in the initial proposal.

Good answers: A complete model — monthly fees, integration, ongoing operational cost, change management, training, your team's time.

Bad answers: Just a software fee. Significant "implementation costs" that weren't in the initial proposal. Hand-waving about "training your team is your responsibility."

A vendor who can't estimate full TCO is either inexperienced or hoping to surprise you with costs later.

Question 7: "What are the cases where we shouldn't work with you?"

Why it matters: Vendors who can name their own poor-fit cases are credible. Vendors who claim to serve everyone aren't.

Good answers: "We're not great for businesses under $X in revenue. We don't serve highly regulated industries that need certifications we don't have. We're not the right fit if you need fully custom with no shared components."

Bad answers: "We can work with any business." Awkward deflection.

A vendor who can't articulate where they don't fit will try to force-fit you. Run.

Question 8: "What's something you've learned the hard way that we should know going in?"

Why it matters: Separates good vendors from genuinely excellent ones. Vendors who've shipped at scale have learned things — sometimes painful — that change how they approach the work.

Good answers: Real war stories with takeaways. "We learned that data quality matters more than we used to acknowledge; we now refuse to start builds until the data is clean."

Bad answers: "Everything's been smooth, actually." Generic platitudes. Pivoting to a sales pitch.

A vendor with no hard-earned lessons hasn't done enough deployments to learn them — meaning yours will be their learning experience.

The proposal pitfalls to ignore

Things vendor proposals emphasize that don't actually matter:

1. Which underlying models they use. GPT vs. Claude vs. Gemini matters far less than configuration and operation. A great deployment on a "lesser" model outperforms a poor deployment on a "better" one.

2. Their team's credentials. "From Google/OpenAI/Meta" is impressive but not predictive. Shipped work matters more than résumés.

3. The size of the vendor. Bigger isn't better. Small teams of 3-5 with direct accountability often outperform large vendors.

4. Length and polish of the proposal. A 60-page proposal isn't more credible than a 6-page one.

5. Pricing — to a point. Don't pick cheapest. Don't pick most expensive on the assumption they're best.

How to use this framework

Print the eight questions. Bring them to the next vendor call. Ask them in order. Take notes.

If answers are vague on Questions 1-4, they're probably not real practitioners. Strong on 1-4 but weak on 5-7, they may be capable but immature operationally. Strong on all eight? Seriously consider them.

The act of asking these questions changes vendor behavior. Vendors who realize you're evaluating seriously become more honest. Vendors who can't answer move on without wasting your time.

What's next

You now know the eight diagnostic questions, what good vs. bad answers look like for each, the pitfalls that don't predict success, and how to use the framework in practice.

Next up: Lesson 22 — Rolling out AI to your team without the revolt.

Frequently asked

Questions that come up after this lesson.