Bridgeside AI
← All posts
freeswitchintegrationguideSep 10, 20265 min read

How to Run an AI Phone Agent on FreeSWITCH with ESL

Add an AI phone agent to FreeSWITCH with mod_audio_stream for media and the Event Socket Layer for transfers and hangups, plus notes on running many calls at once.

By Walter Brennan

How to Run an AI Phone Agent on FreeSWITCH with ESL

FreeSWITCH is the platform people reach for when they need scale and control: high concurrency, clean media handling, and an event system that lets you script call behavior precisely. Those same strengths make it an excellent host for an AI phone agent. You do not replace FreeSWITCH with an AI platform; you stream call audio to the agent and let it drive the call through the Event Socket Layer.

This guide explains the integration in FreeSWITCH terms: media over mod_audio_stream, control over ESL, and how to keep concurrency healthy as you grow.

The shape of an AI phone agent integration

At a high level there are two channels of communication between FreeSWITCH and the AI:

  1. Media: the live call audio, streamed to the agent and back.
  2. Control: the instructions to transfer or hang up, driven by ESL.

Keeping media and control separate is idiomatic FreeSWITCH, and it maps neatly onto how an AI agent works. The agent listens to the media stream, decides what to do, and expresses those decisions as ESL events and commands.

Media: mod_audio_stream over WebSocket

The cleanest way to get call audio out of FreeSWITCH is mod_audio_stream, which opens a WebSocket and pushes call audio to a remote endpoint in real time. In the dialplan, you start the stream and include the campaign's dial code in the URL so the platform knows which configuration to use. The portal generates the full extension with your token filled in; a simplified version looks like this:

<extension name="ai_agent">
  <condition field="destination_number" expression="^85(\d{3})$">
    <action application="answer"/>
    <action application="set" data="api_result=${uuid_audio_stream ${uuid} start wss://app.bridgeside.ai/fs?token=…&amp;agent=$1&amp;uuid=${uuid} mono 8k}"/>
    <action application="park"/>
  </condition>
</extension>

The uuid_audio_stream command begins streaming; park holds the call open while the conversation happens. The platform receives the audio, runs its speech-to-text, language model and text-to-speech pipeline, and streams synthesized speech back over the same socket. Because the media is handled as a stream rather than a re-dial, turn-taking stays responsive.

Bridgeside AI uses mod_audio_stream for FreeSWITCH because it is the least-friction path and works well with a cloud endpoint.

Control: the Event Socket Layer

ESL is where FreeSWITCH shines for this use case. The platform holds one ESL connection to each FreeSWITCH server you register (it connects to your event socket on TCP 8021) and uses it to observe call state and issue commands.

  • Transfer to a human. When the agent qualifies a lead, it issues uuid_transfer to move the parked call to a human destination in the dialplan context you set: an agent extension, a gateway or a queue. Because the call is parked and live, the human joins a conversation that is already connected.
  • Hangup. When the conversation ends, the agent issues uuid_kill over ESL, and your normal CDR and post-call logic run.
  • State awareness. ESL events tell the platform when the channel answers, when media is flowing, and when the far end hangs up, so the agent never talks into a dead line.

Because the platform connects to your event socket, there is nothing extra to run on your side beyond mod_event_socket. Each call is identified by its channel uuid, which the dialplan passes in the stream URL, so commands always reach the right call.

Concurrency at scale

FreeSWITCH is chosen for concurrency, so it is worth being explicit about how AI calls scale.

Each AI call consumes one media stream and one control channel. There is no shared serialization point in the integration, so concurrent capacity is bounded by three things:

  1. CPU and bandwidth on the FreeSWITCH box, mostly for media handling.
  2. Your network egress to the platform for the audio streams.
  3. The channels on your Bridgeside AI plan (see pricing).

To keep things healthy at high concurrency:

  • Pin mod_audio_stream to 8 kHz mono unless you have a specific reason for wideband; it halves bandwidth and matches telephone-quality audio.
  • Watch RTP and socket file-descriptor limits on the host and raise ulimit accordingly.
  • Spread very large deployments across multiple FreeSWITCH instances behind your existing routing, exactly as you would for human traffic.

Because the agent is per-call, adding capacity is horizontal: more FreeSWITCH, more streams, no central bottleneck to re-tune.

Recording and transcripts

FreeSWITCH's record_session captures the full call, AI side included, since the AI is simply the media peer. You keep recordings where you already keep them, in the format you already use. Bridgeside AI does not store call audio. It keeps a per-call transcript for the retention period you choose (1 to 90 days), which is invaluable for QA, compliance review and finding patterns across large call volumes.

Dispositions and downstream systems

For outbound work, each call should produce a structured outcome. FreeSWITCH has no lead database, so the platform posts the result (outcome, disposition, a short summary, the answers captured to your questions and any do-not-call request) to the campaign's signed webhook, which you can feed into your CRM. Treat the DNC signal as first-class: if a lead asks to be removed, that must flow into your suppression list immediately, before the next campaign runs.

Security notes

  • Terminate the media WebSocket over wss:// so audio is encrypted in transit.
  • Restrict ESL to the platform's addresses and use a strong ESL password; ESL is powerful and should never be open.
  • The platform only needs uuid_transfer, uuid_kill and event subscription.

When FreeSWITCH is the right host

If you are already on FreeSWITCH, keep it. Its media flexibility and ESL control make it arguably the nicest place to run an AI voice agent, and the integration is entirely additive. If you are choosing between platforms specifically to run AI, FreeSWITCH is a strong pick when you expect high concurrency or want fine-grained, event-driven control of call behavior.

Rollout checklist

  • Load and configure mod_audio_stream.
  • Add the dialplan extension that starts the stream and parks the call.
  • Make the event socket reachable from the platform's address only, with a strong password and an ACL.
  • Confirm a test call streams audio both ways and transfers cleanly to a human.
  • Verify recording captures the full call and transcripts appear.
  • Wire dispositions and DNC suppression.
  • Load-test at your expected peak concurrency and tune host limits.

Add the media stream, add the ESL control, and FreeSWITCH does what it already does best while the agent handles the conversation. Nothing about your routing, trunking or scale strategy has to change.

For how transfers should feel to your closers, see warm vs blind transfer, and the FreeSWITCH integration page has the short version of this setup. If you run FreeSWITCH and want to trial an agent, contact us and we will help you wire up mod_audio_stream and ESL on a test extension.

Put an AI agent on your dialer

60 free minutes, no card, one dialplan entry.