When to ship an AI feature: the readiness checklist

An AI feature is ready to ship when it solves a real problem faster or better than the current way, when a real person has tested it on actual work, and when you have a plan for what happens when it gets something wrong.

Perfection is not the bar. Accuracy of 85 to 95 percent is often good enough if the work saved is real and the failures don't break the business. A tool that reads 90 percent of invoices correctly and flags the rest for a human eye is worth shipping. A tool that catches 80 percent of duplicate customer records and shows you the matches so you pick the final answer is useful on day one.

Here's what actually matters:

Does it save real time on real work.

Before you ship, someone from your team should use the feature on actual customer data or live work for at least a week. Not test data. Not a demo. Real invoices, real customer emails, real property listings, real patient notes.

Time the work before and after. If a dental clinic in Kelowna spends two hours a day typing patient summaries into the chart, and an AI feature cuts that to 30 minutes, that's a ship-ready feature. If a plumber in Richmond spends 40 minutes a day texting job confirmations and the tool does it in five minutes, you have something worth releasing.

The number doesn't have to be huge. Even one hour a week saved across a five-person team adds up. But you need to measure it. "It feels faster" is not a data point.

One real person has tested it and said yes.

The person who does the work every day needs to try it. Not the founder. Not the product lead. The person who will actually use it.

Take them through a half-day or a full day of their normal work using the new feature. Watch what they do. Listen to what they say. Ask them directly: would you use this if we shipped it tomorrow.

If the answer is yes and they mean it, you're ready. If they say yes but list three things that would make it better, write those down for version two. If they say they'd use it but only if you change something fundamental, that's a signal to iterate before shipping.

This step is not optional. A property manager in Surrey using your tool on five of their own properties will find problems a demo never will.

It fails in a way you can live with.

Your AI feature will make mistakes. The question is whether the mistakes are tolerable.

A tool that sometimes misses a field in a form is fine if the person submitting the form can see what the AI filled in and correct it before it saves. A tool that occasionally suggests the wrong reply to a customer email is okay if a human reads it before it sends. A tool that sometimes misclassifies an expense is acceptable if an accountant in Burnaby reviews the flagged ones before closing the books.

Before you ship, write down the three most likely ways your feature will fail. For each one, ask: what happens next. Is there a person in the loop. Can the mistake be caught and fixed. Can it be undone.

If the answer to all three is yes, you can ship. If any answer is no, you need to redesign the feature or the process around it.

You can support it without burning out.

When customers use your feature, some will report that it didn't work the way they expected. Some will send you examples of things it got wrong. Some will ask if it can do something slightly different.

Before you ship, decide how you will handle those requests. Will one person answer support emails about this feature. Will you batch them and review them weekly. Will you use early feedback to prioritize what to build next.

If you don't have a plan, you will either ignore customers or spend all your time on support. Neither is sustainable.

A 12-person software company in Gastown might assign one person to spend two hours a week on feature feedback for the first month, then one hour a week after that. A distributor in Calgary might review feedback monthly and share it with the product team. Whatever you choose, decide it before the feature goes live.

The shipping threshold, plainly.

Ship when:

• A real user has tested it on real work for at least a few days and said they would use it.
• You can measure that it saves time or reduces errors compared to the current way.
• You have written down what happens when it fails and who catches those failures.
• You have a plan for supporting it that doesn't require you to work nights.

Do not ship when:

• You have only tested it on demo data or made-up examples.
• You are still waiting for accuracy above 95 percent when 80 percent would solve the problem.
• The feature has no graceful failure mode and a mistake could cause real harm.
• You have no idea how you will support it once customers start using it.

Where to start

• Pick one person on your team who does the work this feature will help. Schedule a two-hour working session with them this week using the feature on their actual work. Write down how long it takes them to do the task with and without the AI.

• Write down the three most likely ways your feature could fail or give a wrong answer. For each one, describe what would happen next and whether a person would catch it. If any failure has no safety net, you have work to do before shipping.

• Decide right now who will handle customer questions and feedback about this feature once it is live, how much time they will spend, and when you will review feedback together. Write it down and share it with the team.