AYUSH MAHAJAN
REAL-TIME VOICE PRODUCT2026
perMail

A phone agent that knows when the call needs a human.

I built perMail for moments when a small business cannot answer: it works out who called and why, then turns the conversation into a clear owner handoff instead of leaving another transcript to process.

For
Small teams that cannot answer every call
Ayush owned
Sole builder across the live voice path, FastAPI backend, notification handoff, and reliability debugging
Scope
Incoming call → live conversation → urgency → owner handoff
CONCEPTUAL PRODUCT FLOWBuilt from verified system behavior; not a product screenshot
LIVE CALL PATHREAL-TIME
  1. 01INCOMING CALL
  2. 02LISTEN
  3. 03UNDERSTAND
  4. 04RESPOND
WHAT SHOULD HAPPEN NEXT?
ROUTINECOMPLETE + SUMMARIZE
URGENTBRING IN A PERSON
AFTER THE CALLCALLER · REASON · URGENCY · NEXT ACTIONOWNER HANDOFF →
01 / THE PRODUCT STORY

Before architecture, the workflow.

SITUATION

The call arrives while the owner is unavailable.

For a small team, answering every call is unrealistic. But a missed call can contain a new lead, an urgent customer problem, or noise. The owner needs enough context to act without replaying the whole interaction.

FRICTION

Answering is only the first half of the job.

A useful phone product has to feel responsive while speech is still unfolding, handle imperfect turns and multilingual use, recognize urgency, and finish the operational handoff after the caller hangs up.

WHAT I BUILT

A live receptionist connected to the work after the call.

I built the streaming voice path, intent and urgency handling, spoken response, structured summary, business-owner notification, and the escalation path that can bring a person back into the conversation.

02 / WHAT USING IT FEELS LIKE

The product in human steps.

The default story stays at the level of the person and the work they are trying to finish.

  1. 01
    The call is answered

    The caller reaches a live voice flow instead of voicemail.

  2. 02
    The need is understood

    The system captures why they called and whether it is urgent or unwanted.

  3. 03
    The right path is chosen

    Routine calls continue; high-urgency calls can move toward a human bridge.

  4. 04
    The owner gets context

    A structured follow-up contains the caller, reason, urgency, and next action.

03 / THE DIFFICULT PRODUCT DECISION

Should the agent try to complete every call by itself?

No. Escalation is a product capability, not an automation failure.

The cost of being wrong changes with the call. The agent may handle routine context, but urgent or consequential situations need a clear path back to a person.

04 / WHAT CHANGED DURING DEVELOPMENT

The live call and the follow-up needed different clocks.

BEFORE

Reliability work exposed problems across audio-frame parsing, turn gating, cold starts, and stream state. Slow downstream work also competed with the part of the system the caller could feel.

OBSERVATION

Only a narrow set of operations had to happen before the next spoken response. Summaries, formatting, alerts, and owner workflow could continue after the live turn.

CHANGE

I kept turn handling, transcription, interpretation, and response on the conversational path, then treated structured follow-up and notification as a separate post-call path.

RESULTING PRODUCT PRINCIPLE

The product boundary became clearer: keep the caller moving now, then give the business enough evidence to take over well.

05 / VERIFIED EVIDENCE

What the record supports—and where it stops.

FULL PATHowned by Ayush

Voice loop, backend, handoff, alerts, and reliability debugging.

Source
Canonical résumé and project record
Boundary
Customer identities, numbers, recordings, and caller data remain private.
HUMAN PATHescalation built in

Urgent or consequential calls can move toward a person instead of forcing automation to continue.

Source
Canonical résumé and project record
Boundary
Implemented product behavior, not an adoption or outcome claim.
AFTER CALLstructured owner handoff

The follow-up carries the caller, reason, urgency, and next action into the owner's workflow.

Source
Canonical résumé and project record
Boundary
Workflow scope only; private caller and business data are not published.
06 / OPTIONAL TECHNICAL ROUTE

Architecture is available, not compulsory.

The technical route explains how the live call stays separate from downstream work, which failures shaped the design, and what the public evidence does—and does not—prove.

FOR TECHNICAL READERSArchitecture, boundaries, failure modes, and verification
T01

Two paths, one product

The caller-facing loop stays narrow; slower owner-facing work continues after the turn or call.

  • Twilio Media Streams carries live call audio into the application.
  • Pipecat coordinates turn detection, transcription, model response, and speech output.
  • Intent, urgency, and spam classification shape the next product action.
  • Structured summaries, alerts, and WhatsApp-ready owner handoff run outside the caller's critical path.
T02

The interface is silence

In a phone product, a stalled turn feels broken before an error message can explain it.

  • Incomplete speech and background noise make turn boundaries uncertain.
  • Indian business calls can move between Hindi, English, and Hinglish.
  • Cold starts and stream-state errors are user-experience failures, not only infrastructure failures.
  • Escalation has to preserve enough context for a person to take over without starting again.
T03

Failure paths designed into the product

The agent needs bounded behavior when the audio, interpretation, or downstream handoff is uncertain.

  • Malformed audio frames or disconnected streams fail the live conversation honestly.
  • Turn gating avoids replying over incomplete caller speech.
  • Urgency creates an explicit human-escalation branch rather than a stronger model prompt.
  • Post-call handoff is structured so the owner can inspect what the system understood.
T04

Evidence and current limits

Implementation behavior and ownership scope remain explicit without implying adoption.

  • Full-path ownership covers the voice loop, backend, handoff, alerts, and reliability debugging.
  • Urgent calls have an explicit route toward a person instead of an unbounded automation path.
  • The post-call workflow structures the caller, reason, urgency, and next action for the owner.
  • Recordings and customer data remain private and are not converted into decorative screenshots.
  • The public case study makes no retention, conversion, or broad-production claim.
TECHNOLOGY CONTEXT

The implementation followed the product boundary.

  • Twilio
  • Pipecat
  • FastAPI
  • Deepgram
  • Groq / Llama
  • WhatsApp handoff
07 / WHAT THIS PROVES FOR A CLIENT

This work shows I can own a real-time AI product across conversation design, telephony, backend state, operational handoff, and the reliability problems that appear only when the system meets live input.

Have a useful AI idea trapped in an unclear brief or brittle prototype?

Send me the rough version →