FreeSWITCH mod_audio_stream, a Practical Guide for Voice AI
What mod_audio_stream does, how to load it, the uuid_audio_stream commands and options, how audio gets back to the caller, and the errors you will meet first.
FreeSWITCH mod_audio_stream is the shortest path from a live call to a WebSocket server. The module takes a channel's audio, sends it as raw 16-bit PCM over a WebSocket to any URL you give it, and can play audio that the server sends back. That makes it a natural fit for speech recognition, call analytics and AI voice agents, without writing a FreeSWITCH module of your own. This guide covers what the module does, loading it, the uuid_audio_stream API, getting audio back to the caller, the errors you are most likely to hit, and how Bridgeside uses it.
The module is open source and lives at github.com/amigniter/mod_audio_stream. Options change between releases, so treat its README as the reference for your version and this post as the practical companion.
What FreeSWITCH mod_audio_stream does
Once a stream starts on a channel, the module:
- Opens a WebSocket (
ws://orwss://) to the URL you passed. - Optionally sends a text frame of metadata you supplied.
- Sends the channel's audio as binary frames of L16, meaning 16-bit linear PCM, at 8 or 16 kHz.
- Listens for text frames from the server. Some are commands it understands, such as audio to play.
- Closes the socket when you stop the stream or the channel hangs up.
The call itself stays in FreeSWITCH. The module only taps audio and plays responses, so the dialplan, your gateways, recording and ESL all keep working as before.
Building and loading FreeSWITCH mod_audio_stream
mod_audio_stream is not part of the FreeSWITCH source tree, so you build it separately against the FreeSWITCH version you run. On a package install that means the FreeSWITCH development headers; on a source install, the source tree you built from. The build also needs the usual SSL and zlib development libraries. Follow the build steps in the README for the release you choose, and build on a machine with the same FreeSWITCH version as production.
Once the shared object is in your modules directory, load it and check it:
fs_cli -x "load mod_audio_stream"
fs_cli -x "module_exists mod_audio_stream"
To load it at every start, add one line to autoload_configs/modules.conf.xml:
<load module="mod_audio_stream"/>
If load fails, the FreeSWITCH log usually says why: a missing shared library or a module built against different headers.
The uuid_audio_stream API
Everything is driven by one API command, uuid_audio_stream, with a channel UUID and a sub-command:
uuid_audio_stream <uuid> start <ws-url> <mix-type> <sample-rate> [metadata]
uuid_audio_stream <uuid> send_text <metadata>
uuid_audio_stream <uuid> pause
uuid_audio_stream <uuid> resume
uuid_audio_stream <uuid> stop [metadata]
The options on start:
- mix-type:
monosends only the caller's audio,mixedsends both sides mixed into one channel, andstereosends both sides as two channels. A voice agent wantsmono, because it should hear the caller and not its own voice coming back. - sample-rate:
8kor16k. Phone calls on G.711 carry 8 kHz audio, so8kloses nothing and halves the bandwidth. - metadata: an optional string, often JSON, sent as a text frame when the socket opens. Useful for passing a call ID or campaign name if you do not want them in the URL.
pause and resume stop and restart sending audio without closing the socket, for example while a caller reads a card number to a human agent. stop closes the stream; the channel carries on.
You can run these from fs_cli, over ESL, or inside the dialplan.
Starting a stream from the dialplan
Most deployments start the stream in the dialplan with API expansion, then park the call so it stays up while the server talks to the caller. This is the extension Bridgeside's portal generates (your copy has your token):
<extension name="bridgeside_ai">
<condition field="destination_number" expression="^85(\d{3,6})$">
<action application="answer"/>
<action application="sleep" data="200"/>
<action application="set" data="STREAM_HEART_BEAT=15"/>
<action application="set" data="api_result=${uuid_audio_stream ${uuid} start wss://app.bridgeside.ai/fs?token=YOUR_TOKEN&agent=$1&uuid=${uuid}&phone=${caller_id_number}&lead=${caller_id_name} mono 8k}"/>
<action application="park"/>
</condition>
</extension>
Three details matter:
- Answer first. There is no audio to stream on an unanswered channel. The short sleep lets media settle.
- Escape ampersands. The dialplan is XML, so every
&in the URL must be written&. A bare&makesreloadxmlfail. - Park, do not hang up.
parkkeeps the channel alive with no application running, which is what you want while the WebSocket server holds the conversation. Control comes from outside, over ESL.
api_result captures the command's reply, so you can log it or branch on it.
Channel variables worth knowing
The module reads a few channel variables when a stream starts. Two that Bridgeside relies on:
STREAM_HEART_BEAT: seconds between keep-alive pings on the WebSocket. Fifteen keeps idle connections from being dropped by proxies and load balancers.STREAM_MESSAGE_DEFLATE: controls permessage-deflate compression on the socket. Your server must accept whatever the module negotiates; Bridgeside's endpoint accepts compression.
The README lists the others, including buffer size and TLS options. Check them against your version before you rely on them.
Sending audio back to the caller
Streaming in is half the job. For a voice agent the server must also talk back. mod_audio_stream accepts a JSON text message of type streamAudio whose data carries base64 encoded audio, its format (raw PCM or WAV, among others) and its sample rate. The module writes the audio to a temporary file and plays it on the channel. A killAudio message stops playback at once.
Bridgeside sends agent speech as streamAudio messages carrying WAV at the stream's sample rate, and sends killAudio when the caller starts talking over the agent, so barge-in works.
One detail is worth knowing if you build your own server. The module does not tell you when a file has finished playing. If you send a second file while the first is still playing, they can overlap or cut each other off. Bridgeside's approach is to send one WAV at a time and send the next when the previous one's duration has elapsed, with a short pre-buffer at the start of each reply. Each later chunk contains everything synthesized while the previous one played, so chunks grow and the gaps between them stay tiny.
Common errors and fixes
| Symptom | Likely cause |
|---|---|
uuid_audio_stream reported as an unknown command | Module not loaded. Run load mod_audio_stream and check the log |
reloadxml fails after adding the extension | An unescaped & in the URL |
| Stream never connects | DNS, TLS or a firewall blocking outbound 443 from the switch. Test the URL from the switch itself |
| Socket opens, then the server closes it | The server refused the request. Bridgeside answers 401 for a bad token and 429 when the plan's channel limit or call rate is reached |
| Caller hears nothing back | The server is not sending streamAudio, or the module version you built does not support playback |
| Agent hears itself | mixed or stereo used where mono was meant |
| Choppy or fast audio | Sample rate in the message does not match the audio |
Turn up logging with fs_cli -x "console loglevel debug" while you test, and set it back afterwards.
How Bridgeside uses FreeSWITCH mod_audio_stream
Bridgeside uses mod_audio_stream for media and the FreeSWITCH event socket for control. The stream URL carries your organization token, the campaign's dial code (the digits after 85) and the channel UUID. The caller's number and name ride along for results. Bridgeside holds one ESL connection per registered switch, on port 8021, and uses uuid_transfer to hand a caller to a person and uuid_kill to end the call. It listens for the hangup event to close the session.
If the UUID is missing from the URL, or the switch is not registered on the Dialers page, the conversation still runs, but the agent cannot transfer or hang up. The architecture view of all this is in the FreeSWITCH AI agent guide, and the ESL side is covered in running an AI phone agent on FreeSWITCH with ESL.
FAQ
What is mod_audio_stream in FreeSWITCH?
It is an open source FreeSWITCH module that streams a channel's audio as raw 16-bit PCM to a WebSocket server and can play audio the server sends back. It is controlled with the uuid_audio_stream API command.
Is mod_audio_stream part of FreeSWITCH?
No. You build it separately against your FreeSWITCH version and load it like any other module.
How do I stop a mod_audio_stream stream?
Run uuid_audio_stream <uuid> stop. The stream also ends when the channel hangs up, or when the server closes the socket.
Should I use mono, mixed or stereo?
Use mono for a voice agent or speech recognition on the caller. Use mixed or stereo when you need both sides, for example for analytics on a call between two people.
Does mod_audio_stream FreeSWITCH support 16 kHz?
Yes, with 16k. On calls that use a narrowband codec such as G.711 it only doubles the bandwidth.
Try it on a test extension
Add the extension above with your own token, register the switch on the Dialers page, and dial 85 plus your campaign's code. The FreeSWITCH integration page has the short checklist, and pricing explains per-minute credit. You get 60 free minutes, no card, at app.bridgeside.ai.