Every VoIP call, every SIP trunk, every AI voice agent depends on a protocol most people never hear about. SIP gets the attention — but SIP doesn’t carry a single millisecond of your voice. RTP does. This is how it works, why it’s separate from SIP, and why it quietly determines whether your calls sound clear or garbled.
What is RTP?
RTP (Real-time Transport Protocol) is the protocol that carries the actual audio and video data in a VoIP call. While SIP sets up, manages, and ends the call, RTP is what transports the voice itself — the digitised sound of the conversation — as a stream of small packets across the internet in real time.
Put simply: SIP is the phone ringing and connecting; RTP is the voice travelling down the line once you’ve picked up.
The core idea: streaming voice as packets
When you speak into a VoIP phone or an AI voice agent answers a call, your voice is converted from analogue sound into digital data. That data can’t travel across the internet as one continuous stream the way it did on old copper phone lines. Instead, it’s broken into hundreds of small packets per second, each one carrying a tiny slice of audio.
RTP is the structure that wraps each of those packets. It adds the information needed to turn a chaotic stream of internet packets back into smooth, intelligible speech at the other end.
Every RTP packet carries three things that make real-time audio possible:
- A sequence number — so the receiving device knows the correct order, even if packets arrive scrambled.
- A timestamp — so the audio can be played back at the right speed and stay in sync (critical when video is involved too).
- The payload — the actual compressed audio, encoded by whichever codec the call negotiated (G.711, Opus, and so on).
Internet traffic behaves like city traffic: congestion, bottlenecks, and different routes mean packets rarely arrive in the exact order they were sent. Without RTP’s sequence numbers and timestamps, the result would be unintelligible noise. With them, the receiving end can reassemble the stream correctly — or make an intelligent decision when a packet is missing.
Why real-time audio breaks the normal rules of the internet
Most internet data transfer prioritises accuracy over speed. When you download a file, it doesn’t matter if it arrives in 2 seconds or 5 — but it matters enormously that not a single byte is lost or corrupted. That’s why file transfers use TCP, which constantly checks for errors and re-requests any missing data.
Voice is the opposite. In a live conversation, a packet that arrives late is worse than useless — it’s already too late to play. Re-requesting a lost packet would introduce a delay far more damaging to the conversation than simply skipping that sliver of audio.
This is why RTP runs over UDP, not TCP. UDP is the “fire and forget” transport — it sends packets without waiting for confirmation or re-requesting lost ones. RTP accepts that the occasional packet will vanish, because for real-time voice, a tiny dropout is always better than a delay. Your brain fills in a 20-millisecond gap without noticing; it cannot tolerate half a second of lag.
This trade-off — favouring speed and timeliness over perfect delivery — is the single most important thing to understand about RTP. It’s a protocol designed around the reality that in conversation, timing is everything.
RTP never travels alone: the protocol stack
RTP is always wrapped inside other protocols, each handling a different job. In a typical VoIP call, the layers look like this:
- IP (Internet Protocol) — handles addressing and routing across the network.
- UDP (User Datagram Protocol) — the fast, no-handshake transport layer RTP rides on.
- RTP — structures the audio packets with sequence numbers and timestamps.
- RTCP (RTP Control Protocol) — RTP’s companion. It doesn’t carry audio; it monitors call quality in real time, reporting packet loss, jitter, and delay back to both endpoints so they can adapt.
RTCP is the unsung hero here. It’s constantly measuring the health of the stream and feeding that data back, which is what allows a well-built VoIP system to adapt to changing network conditions mid-call — adjusting jitter buffers, flagging quality problems, and giving carriers the telemetry they need to route around trouble.
SIP vs RTP: the difference that confuses everyone
This is the single most common point of confusion in VoIP, so it’s worth stating plainly.
SIP (Session Initiation Protocol) is signalling. It finds the other party, negotiates what codec to use, rings the phone, and tears the call down at the end. SIP is the call control.
RTP is media. Once SIP has set up the call, RTP carries the actual voice. SIP could be thought of as the waiter who takes your order and brings the bill; RTP is the meal itself.
Crucially, SIP and RTP travel separately — often over different ports, and sometimes even different network paths. A call can have perfect SIP signalling (it connects fine) but terrible RTP (the audio is choppy or one-way). This is why “the call connected but I couldn’t hear them” is such a common VoIP problem — it’s almost always an RTP issue, not a SIP one. One-way audio, in particular, is the classic symptom of RTP packets being blocked in one direction, usually by a firewall or NAT misconfiguration.
What RTP has to do with call quality
Because RTP carries the actual audio, it’s where call quality lives or dies. Three RTP-level problems account for the vast majority of VoIP quality complaints:
Jitter — packets arriving at uneven intervals. RTP’s timestamps let the receiver run a “jitter buffer” that smooths out the variation, but too much jitter overwhelms the buffer and the audio breaks up.
Packet loss — when too many packets vanish, the small gaps RTP normally papers over become audible dropouts. A little loss is invisible; sustained loss destroys the call.
Latency — the total delay from speaker to listener. RTP itself is low-overhead, but every hop, every transcode, and every buffer adds milliseconds. Beyond about 150ms one-way, conversation starts to feel unnatural — people talk over each other.
A good carrier engineers its network to keep all three under control: minimal hops, no unnecessary transcoding, and media routing that keeps RTP paths short.
Why RTP matters more than ever for AI voice agents
Here’s the part almost no one writes about — and it’s where RTP has become newly critical.
Every AI voice agent — built on Pipecat, Vapi, Retell, or a custom stack — depends on RTP to carry audio between the caller and the AI pipeline. When someone talks to an AI agent on the phone, their voice travels as RTP packets to the platform, gets transcribed (STT), processed by an LLM, synthesised back into speech (TTS), and returned as RTP packets. RTP is the transport layer on both ends of that loop.
And AI voice is far less forgiving of RTP problems than human conversation is. Two reasons:
Latency budget is already spent. A natural AI conversation needs the total round-trip — RTP transport plus STT plus LLM plus TTS — to stay under about 800 milliseconds. The AI processing alone eats 300-500ms of that. So the RTP media path has almost no slack. Every extra hop or transcode at the RTP layer directly erodes the conversational feel of the agent.
Speech recognition degrades with packet loss. A human listener mentally reconstructs a word with a small dropout in the middle. A speech-to-text model often can’t — a lost RTP packet can turn a correctly recognised word into a transcription error, which then cascades into a wrong LLM response. Clean RTP delivery matters more for a machine listener than a human one.
This is why the codec choice matters so much for AI voice. Using G.711 — an uncompressed codec — means the carrier passes RTP straight through with no transcoding, avoiding the 5-20ms delay that compressed codecs add at each conversion. For AI agents, keeping the RTP path clean and transcode-free is one of the few latency levers you fully control.
If you’re building voice AI, the RTP layer isn’t plumbing you can ignore — it’s part of your latency and accuracy budget. (For the full picture of how this fits together, see our guide on SIP trunking for AI agents and the breakdown of WebRTC vs SIP for voice bots.)
How RTP fits into a carrier network
When a voice termination provider like IDT Express connects your call, RTP is what actually crosses their network carrying the audio. A carrier’s job, at the RTP level, is to move those packets with minimal loss, minimal jitter, and minimal added latency — routing them along the shortest clean path and avoiding unnecessary transcoding that would add delay.
This is the difference between a wholesale carrier and a retail reseller layered on top of one: fewer hops, direct interconnects, and G.711 passthrough all keep the RTP stream cleaner, which is what your callers actually hear. Good carrier engineering is, in large part, good RTP handling.
The short version
SIP connects the call. RTP carries the voice. They travel separately, which is why a call can connect perfectly yet sound terrible — audio problems are almost always RTP problems.
RTP works by breaking your voice into timestamped, sequenced packets sent over UDP, accepting the occasional lost packet in exchange for the low latency real-time conversation demands. Its companion RTCP monitors quality so the system can adapt.
And in 2026, RTP matters more than ever: it’s the media layer under every AI voice agent, where the latency and packet-loss tolerances are tighter than any human conversation ever required. Whether you’re running a traditional VoIP system or building an AI calling platform, the quality of your RTP path is the quality of your calls.
Frequently asked questions
What is RTP in VoIP?
RTP (Real-time Transport Protocol) is the protocol that carries the actual audio and video data in a VoIP call. It breaks the voice stream into small packets, each stamped with a sequence number and timestamp, and sends them over UDP so they can be reassembled into smooth speech at the receiving end. RTP handles the media (the voice itself), while SIP handles the signalling (setting up and ending the call).
What is the difference between SIP and RTP?
SIP (Session Initiation Protocol) sets up, manages, and terminates a call — it’s the signalling layer. RTP (Real-time Transport Protocol) carries the actual audio once the call is connected — it’s the media layer. They operate separately and often over different network ports, which is why a VoIP call can connect successfully (SIP working) but have no audio or one-way audio (RTP failing).
Does RTP use TCP or UDP?
RTP runs over UDP, not TCP. UDP sends packets without waiting for delivery confirmation or re-requesting lost ones. This suits real-time voice because a late packet is useless in a live conversation — it’s better to drop an occasional packet than to introduce the delay that re-transmission would cause. TCP’s reliability guarantees would add latency that damages call quality more than minor packet loss does.
Why is there no audio on my VoIP call even though it connected?
This is almost always an RTP problem, not a SIP problem. The call connecting means SIP signalling worked. No audio, or audio in only one direction, means RTP packets are being blocked — usually by a firewall or NAT device that permitted the SIP signalling but is dropping the RTP media stream. Checking firewall rules for the RTP port range and confirming NAT traversal settings resolves most of these cases.
What causes poor RTP call quality?
Three RTP-level issues cause most VoIP quality problems: jitter (packets arriving at uneven intervals), packet loss (too many packets dropped, causing audible gaps), and latency (excessive delay from speaker to listener). A jitter buffer smooths minor variation, and RTP tolerates minor loss, but sustained problems degrade the call. Minimising network hops, avoiding unnecessary transcoding, and using a well-engineered carrier network keep all three under control.
Why does RTP matter for AI voice agents?
RTP carries the audio between the caller and the AI pipeline in every AI voice agent. AI voice is less tolerant of RTP problems than human conversation because the total latency budget (RTP transport plus speech-to-text, LLM, and text-to-speech) must stay under about 800ms for natural conversation, and because speech recognition models degrade more from packet loss than human listeners do. Using G.711 codec passthrough to avoid transcoding keeps the RTP path fast and clean, which directly improves AI agent responsiveness and transcription accuracy.
IDT Express engineers its carrier network for clean RTP delivery — minimal hops, G.711 passthrough, and direct interconnects that keep your call audio clear. Explore SIP trunking with IDT Express →
