A reference implementation: AI voice and chat for a clinic contact center

A note before this one: the clinic itself is a composite, not a named client — the same recurring pattern we see (a contact center only reachable when staff are free), written up as a technical walkthrough. What’s changed since we first published this: we’ve now built the underlying system for real, end to end — a real phone number, a real scheduling backend, real test calls — rather than describing what we’d design. The business scenario below is illustrative; the engineering isn’t. Costs and containment-rate figures are still illustrative, since there’s no real clinic’s call volume behind them yet.

The problem

A clinic contact center, staffed during business hours, handling appointment bookings, rescheduling, and a long tail of basic questions — opening hours, whether a referral is needed, what to bring to a first appointment. The problem wasn’t call volume, it was availability: staff were busy with in-person patients or simply not there outside a narrow window, so a meaningful share of calls and chats went unanswered or abandoned. The brief was an AI voice and chat assistant, available around the clock, natural enough to feel like talking to a person — with a human always reachable for anything the assistant shouldn’t handle alone.

Start with measurement, not a build

Before recommending anything, we measured the thing we were trying to fix. One week of the existing contact center: calls received, calls answered, average time to answer, chat messages received and response times, and a rough tag of what each contact was actually about (booking, rescheduling, a basic question, something clinical). That gave us two things a demo never would: a real baseline to measure against later, and a breakdown of which contact types were actually routine enough to automate — which turned out to be the majority, but not all, of the volume.

The recommendation that came out of it wasn’t “automate the contact center.” It was narrower: automate booking, rescheduling, and the basic-question long tail; escalate everything clinical, ambiguous, or emotionally charged straight to a person; and keep a human able to see and step into any AI conversation while trust in the system was still being built.

The architecture, as built

One structural decision worth calling out: we didn’t build this clinic-specific. The business, its services and hours, what the assistant may handle and which capabilities are switched on are all configuration, not code. The same assistant can run for a clinic, a bank branch or a restaurant, and we’ve proven that with a second, unrelated scenario. If your business doesn’t look exactly like a clinic, that’s the point: the architecture doesn’t assume it does.

The main building blocks:

Telnyx for the phone line, chosen for cost and a real-time media interface built with voice AI in mind.
A small, fast language model for the conversation. This task is structured: understand the request, fill in the details, call the right tool, confirm back to the caller. A small model handles that reliably when the scope is tight, at a fraction of the latency and cost of a frontier model. It is weaker at anything ambiguous or outside the flow, which is exactly why narrow scope and a clean handover to a person matter more than model size here.
Pipecat for orchestration: the turn-taking, interruptions and timing that make a voice assistant feel like a conversation rather than a walkie-talkie. Our companion piece explains what that involves.
Streaming speech recognition and speech synthesis, chosen for how they perform on real phone audio, not on clean sample recordings. A noisy phone line is much harder than a studio clip, and the difference only shows up once you test with a real phone.

What broke, and what we learned fixing it

This is the section a hypothetical write-up can’t have. Every one of these was found on a real test call, not caught in review beforehand — which is itself the point: a system that looks right in isolation and a system that behaves right on a live call are different claims, and only one of them is testable without picking up a phone.

The assistant didn’t know what day it was. It had to guess when resolving “tomorrow” or “next Monday”, and routinely guessed wrong, correctly reporting no availability for the wrong day. The scheduling integration was right. Nothing about the flow looked broken from the outside, and only a real call found it.
Silence during a lookup sounds like a dead phone, not “still working”. The first time the assistant checked real availability, the caller heard nothing at all while it waited. People don’t wait through silence on a phone the way they watch a spinner on a screen.
A tool doing exactly what it should still needs to be talked about well. Asked for availability, the assistant read every open slot of the day aloud. No receptionist does that.
A model will happily narrate its own machinery. It read internal identifiers out loud, and after that was fixed, it started describing its own tool calls to the caller. A small model treats anything it can see as fair game to say unless it learns otherwise.
A setting in the code isn’t the same as a setting that’s deployed. A new capability simply didn’t take effect, because the live configuration was never updated to match. Worth a deployment checklist, not just a code review.
Ending a call needs the same care as starting one. Early on, the assistant would finish helping and then wait, indefinitely, until the caller hung up. That sounds small until you’re the one on a call that never ends.
A vendor outage will happen, and silence is the worst way to fail. Mid-project, our speech provider’s account hit a billing problem and rejected every connection. The caller heard the call connect, then nothing, because the only way the system could speak was the part that had broken. Now an outage of any AI service still ends with the caller hearing an apology. The first version of that fallback had its own bug, found and fixed the same day, on a real call.

None of these are exotic failures. They’re the specific, boring kind that only shows up once a real caller (or a caller-shaped test) hits the system, and they’re exactly why “we tested it internally” and “we ran it against a real phone line” are different levels of confidence to be offering a client.

Other implementation challenges

“Indistinguishable from a person” runs straight into disclosure obligations. The EU AI Act requires informing users when they’re interacting with an AI system unless it’s obvious from context — and on a phone call, it usually isn’t obvious. We didn’t fight this; we designed a brief, natural disclosure into the greeting (“Hi, you’re speaking with [Clinic]’s assistant — I can help with bookings and quick questions, and I’ll bring in a team member for anything else”) and found it didn’t meaningfully change how people used it. Fighting the requirement would have been the wrong hill.
A small model is fast and cheap, and it will confidently misunderstand things outside its scope. Relative dates (“the Tuesday after next”), compound requests, and anything ambiguous need either tight slot-filling prompts and confirmation steps, or a fast handoff — not a bigger model as the first fix.
The clinic’s scheduling system had no clean API — a common story with practice-management software. A meaningful share of the integration work was making that system reliably scriptable at all, not the AI layer on top of it. We validated the integration against a live scheduling account before trusting it, and found places where the system behaved differently from its documentation. Those only show up by calling it, not by reading about it.
Staff were, reasonably, worried about being replaced. The framing that worked was positioning the assistant as absorbing overflow and after-hours volume — the calls that were previously going unanswered — rather than replacing the people handling the ones that came through during business hours. Getting reception staff involved in defining the escalation rules made them co-owners of where the line sits, not just spectators.
Scope creep pressure, post-launch. Once it worked well for booking, the natural next ask was “can it also answer questions about symptoms.” That’s exactly the line we’d drawn on purpose — logistics and administration, not clinical judgment — and held.

Cost structure

Illustrative, not a quote — the real shape of it is what matters:

Telephony: per-minute PSTN cost through Telnyx, scales directly with call volume.
Inference: a small model is small enough to self-host cost-effectively on modest GPU infrastructure at real call volumes, rather than paying per-token to a hosted frontier model for every turn of every call — a deliberate cost decision we now have real per-call instrumentation to make with actual data, not a guess.
STT/TTS: usage-based, typically per-minute or per-character — and, as above, worth budgeting for a fallback path that has to work even when the primary vendor doesn’t.
One-time: integration work (dialogue design, tool/API integration with the scheduling system, testing against real call audio, and — now proven necessary — testing against real failure modes) — the majority of total project cost, as it usually is.
Ongoing: monitoring, evaluation, and iterating on the escalation rules as real usage reveals edge cases the initial design didn’t anticipate.

At meaningful call volume, the ongoing cost per contact handled by the assistant runs well below the cost of a human-handled contact — but the comparison that actually matters isn’t cost-per-contact, it’s the contacts that were previously going unanswered entirely.

Current state, in this scenario

Voice and chat both live, handling the bulk of routine booking and FAQ volume, with a real end-to-end test call confirmed working — webhook through to the caller hearing a natural reply and the interaction logged with an inferred satisfaction rating. Human oversight — a live view into AI conversations with the ability to step in — is still on, tapering as confidence in the escalation rules builds rather than being switched off on a fixed schedule. Clinical questions and anything ambiguous still escalate directly, by design.

Illustrative KPIs

The kind of metrics that actually matter here, with illustrative ranges for a deployment like this:

Availability: business hours only → 24/7.
Containment rate: roughly 60–70% of routine contacts (booking, rescheduling, basic questions) resolved without a human, in line with the baseline week’s breakdown of contact types.
Abandoned/missed contact rate: meaningfully down, since the assistant answers immediately rather than a caller hitting a busy line and giving up.
Time to first response: from minutes-to-hours (voicemail, callback queues) down to immediate.
Escalation resolution: full conversation context passed to the human agent, rather than the caller re-explaining from scratch.

Benefits and cost savings

Beyond round-the-clock availability:

Fewer no-shows, from automated appointment confirmations and reminders the assistant can send without anyone having to remember to.
Freed-up staff time — routine bookings and repeat questions stop competing with in-person patients and genuinely complex calls for the same attention.
Consistent answers. The assistant doesn’t have an off day or give five different staff members’ five different phrasings of the same policy.
Captured demand that was previously lost — every answered call that would otherwise have been abandoned is a booking that didn’t have to be won back later.
No need to staff for peak — Monday-morning call spikes get handled without overstaffing the rest of the week to cover them.

What’s next on the roadmap

Widening the assistant’s scope from booking into pre-visit intake questionnaires, once the current scope has enough real usage data to justify it.
Proactive outreach — reminders and post-visit check-ins — not just inbound handling.
A proper observability dashboard for staff, replacing ad hoc spot-checking with a real view into what the assistant is handling and where it’s escalating.
Revisiting the model choice once there’s enough evaluation data to know where a small model’s limits are actually being hit, rather than upgrading speculatively.
Confirming the vendor-outage fallback against more failure types than the one that’s actually happened so far.

This is the same discipline as the delivery model itself — measure first, scope deliberately, keep a human in the loop where it matters, and let real usage data decide what happens next rather than guessing upfront.

Get in touch if something like this is on your roadmap.