AI ENGINEERINGAGENTIC SYSTEMSLLM INTEGRATIONPRODUCTION AI

SMS Lead Qualification Agent

An autonomous conversational agent built on Anthropic's Claude that qualifies inbound leads, books sales meetings, and operates 24/7 with sub-5-second first-touch response time.

Role: AI Systems Architect & Developer

Client: Confidential — Australian Vocational Training Group
Period: 2026
Type: Independent contract via OD-LAN Enterprises
1,278
UNIQUE LEADS
344
BOOKINGS
26.8%
CONVERSION RATE
4.6s
P50 RESPONSE TIME
$270
TOTAL OPERATING COST

The Challenge

The client's training business had a sales coverage gap. Inbound enquiries arriving outside business hours went unanswered, and by morning the leads had gone cold. Hiring more sales staff was the obvious answer but not the smart one.

The brief was to close the coverage gap with software: engage leads instantly, qualify them against business rules, push them toward booked sales meetings, and operate at a cost that beat the human alternative — 24 hours a day, seven days a week.

The Solution

A conversational SMS agent operating under a dedicated persona

A conversational SMS agent built on Anthropic's Claude, operating under a dedicated persona. The agent qualifies inbound leads against business rules sourced from the client CRM, books sales meetings via a scheduling link, holds back leads with incomplete CRM data and auto-re-evaluates them as enrichment catches up, and gracefully hands off out-of-hours conversations to humans the next business morning.

Prompt Versioning in Source Control

Ten prompt versions to date, each driven by transcript analysis of the prior period. Per-version booking rates, message counts, and conversation outcomes are queryable from production data. Every iteration was motivated by evidence, not intuition.

Asymmetric-Error Reasoning in Product Behaviour

Misclassifying a soft “not interested” as a hard STOP costs a warm lead permanently; missing a STOP costs one polite reply. The active prompt version is anchored in that tradeoff — a deliberate product decision encoded in the agent's behaviour.

Holdback Funnel

Beyond the conversations the agent handled, it triaged and held back leads arriving without a course code or assigned rep, then auto-re-evaluated them as CRM data caught up. Many were eventually promoted into live conversations once enrichment completed.

Per-Conversation Observability

Every conversation, every prompt version, every outcome is tracked and re-queryable — with a monitoring dashboard for conversation lifecycle and a script suite for re-running production metrics on demand. Observability was first-class from day one, not bolted on after launch.

Technology Stack

LANGUAGETypeScript end to end, Turborepo monorepo
AIAnthropic Claude Sonnet (multiple model versions across prompt iterations)
BACKENDHono on Fly.io, Redis + BullMQ for async workflow
FRONTENDNext.js monitoring dashboard on Vercel
DATASupabase Postgres with Prisma ORM
TELEPHONYTwilio for SMS delivery
CRMHubSpot integration (lead data, course codes, rep assignment)

Outcome

Lifetime metrics — first seven weeks of production

344 booked sales meetings at a 26.8% conversion rate, a 73% attendance rate on audited bookings, and a total operating cost of $270 USD — roughly $0.79 per booking acquired.

Performance

  • 1,278 unique leads contacted across 1,409 conversations
  • 344 bookings at 26.8% conversion (agent-attributed, excluding self-bookings)
  • 73% attendance on audited bookings
  • 4.6s p50 / 5.2s p90 first-touch response, 24/7 — beats the industry “5-minute rule” by ~60x

Cost & Quality

  • $270 USD total operating cost ($26 Claude API, $244 Twilio SMS)
  • 1 opt-out across 1,409 conversations (0.07%) vs. 1–3% industry baseline
  • 10 prompt versions under source control, each driven by transcript analysis
  • That single opt-out was a false positive — it drove the next prompt iteration

A Natural Experiment

In week three, a new human sales rep's queue was held off the agent for 60 hours as a calibration test. Across that window, the agent matched the human's 20% booking rate while delivering 100% coverage versus the human's 40%, with sub-5-second first contact across cold weekend traffic.

Need an AI Agent Built for Production?

From prompt architecture and eval strategy to deployment and observability — let's build something that works in the real world.

GET IN TOUCH

Client and specific metrics anonymised under NDA. Full case study with named client available on request under mutual confidentiality.