Before You Buy AI, Decide How You'll Know It Worked

Most small-business owners have heard the pitch: AI will answer your phones, book appointments, handle paperwork, and save you hours. Some of that can be true. But a few months after adding a new tool, plenty of owners can't answer a simple question: Did it actually help?

Usually that isn't because the tool failed. It's because nobody wrote down what things looked like before it started. Without a "before," the "after" is just a feeling.

This article is the first in a short series on how to tell whether AI and automation is really paying off in a business like yours. We'll start with the most important step, and it happens before you buy anything.

Start with one workflow, not "AI for the business"

"We want to use AI" is too big to measure. "We want fewer missed calls after 5 p.m." is something you can measure.

Pick one workflow that bugs you and your team every week. Some common ones:

• Answering and logging calls that come in after hours or during the lunch rush
• Confirming, reminding, and rescheduling appointments
• Pulling details off intake forms and typing them into your software
• Following up on unpaid invoices, missing documents, or unsigned forms

Now you're not measuring "AI." You're measuring how one job gets done today.

The five numbers that matter most

You don't need a dashboard with forty charts. For most service businesses, five measures cover the bulk of what matters. Pick the two or three that fit your workflow.

1. Response time. How long does it take for someone who reaches out to get a real answer? Measure from the moment they call, text, email, or submit a form to the moment they get a useful reply, not just an automatic "we got your message."

2. Missed calls. How many calls go unanswered, roll to voicemail, or hang up before anyone picks up? Look at business hours and after hours separately, since they usually tell different stories.

3. Hours spent on repetitive tasks. How many staff hours a week go into the workflow you picked? Rough is fine. "About 20 minutes per new client, and we get around 15 a week" is a solid starting number.

4. Error or rework rate. Out of every batch of items (appointments booked, forms entered, invoices sent), how many need fixing? Wrong date, misspelled name, missing insurance details, a reminder that went to the wrong number. Count the fixes, and roughly how long they take.

5. How often a human has to step in. Once AI is involved, some conversations and tasks will get handed back to a person. That's expected, and often it's exactly right. Some calls should go to a human. But you want to know how often it happens and why, so you can tell good handoffs (an upset customer, a complex legal question) from avoidable ones (the system didn't understand a common request).

That last number doesn't exist until something is running, but it helps to decide now what you'll track. A handoff is not a failure. A handoff nobody planned for, happening over and over for the same reason, is something to fix.

How to set a baseline with tools you already have

A baseline is just a snapshot of how things work today. Two weeks of normal operation is usually enough to get a fair picture. Avoid holiday weeks if you can, or note that's when you measured.

Most of what you need is already sitting in systems you use:

• Your phone system. Most business phone services (and many cell carriers' business plans) have a call log or report showing total calls, missed calls, and time of day. Export it or screenshot it.
• Your scheduling or practice-management software. Look for no-shows, cancellations, reschedules, and how far in advance appointments get booked.
• Your email inbox and web forms. Pick a sample of messages and note when each came in and when someone replied.
• Your CRM, if you have one. Look at how long leads sit before first contact, and how many never get a reply.
• A simple spreadsheet or tally sheet. For hours and errors, the most reliable method is the low-tech one: ask the people doing the work to jot down time spent and anything they had to fix. A sheet by the front-desk computer works fine.

Hypothetical example: Say a three-person front desk at a dental office wants to measure appointment reminders and rescheduling. For two weeks, they pull the daily call log, note the no-shows from their scheduling software, and keep a tally of how many minutes each person spends on reminder calls. At the end, they have three numbers: missed calls per day, no-shows per week, and staff hours on reminders. That's a baseline, and it took discipline, not new software.

A note on privacy. When you pull call logs, emails, or schedules, you're handling customer, client, or patient information. Keep your baseline to counts and times (how many, how long, what time of day), not names, phone numbers, or details of anyone's situation. Store the spreadsheet where only the people who need it can open it. If you're in healthcare, therapy, law, or accounting, treat the baseline file with the same care as any other record covered by your privacy obligations.

Decide what "worked" means before you start

With a baseline in hand, write down what success would look like in plain language. For example:

• "We want after-hours calls to get a response the same evening instead of the next morning."
• "We want to cut the time spent typing intake forms roughly in half, without adding errors."
• "We're fine with a human stepping in on billing questions, but routine scheduling should be handled start to finish most of the time."

Also write down what would make you stop. Maybe it's more errors than before, customers complaining that they can't reach a person, or staff spending more time checking the system than they saved. Deciding this ahead of time keeps the decision honest later.

Finally, pick a date to check back. Thirty days gives you an early read, and 90 days gives you a real one.

Where we fit in

When we do an assessment with a business, writing down the baseline and agreeing on what "worked" means is part of the job, before anything gets built. We'd rather you be able to see the value (or the lack of it) for yourself than take our word for it. And if the numbers say a workflow isn't worth automating, that's useful to know too.

Your before-you-buy checklist

• Pick one workflow to focus on first.
• Choose two or three of the five measures: response time, missed calls, hours on repetitive tasks, error/rework rate, human handoffs.
• Collect two weeks of baseline data from your phone logs, scheduling software, inbox, CRM, or a simple tally sheet.
• Keep the baseline to counts and times, not personal details, and store it securely.
• Write down, in plain language, what "worked" looks like.
• Write down what would make you stop or change course.
• Put a 30-day and a 90-day check-in on the calendar.

Next in the series: how to run a two-week baseline using only the tools you already have.

If you'd like a second set of eyes on your baseline, or want to talk through which workflow to start with, email us at hello@rsoftware.net. We're happy to start with one workflow.

Written with help from our AI assistant.