If you arrived here from the business version, you're in the right place. That piece covered why the demo takes an afternoon and production takes months. This is the companion "what" — the actual chain between two tabs, stage by stage, and what fails at each link.
Two browser tabs, one video call. Between them sit eight distinct stages, and exactly two of them handle video.
The other six are negotiation and transport: agreeing how to talk, finding a path, securing it, and compensating for a network that will not behave. That ratio explains the single most useful fact about WebRTC architecture — almost nothing that breaks in browser video is the video. It is the route to it, and the handshake around it.
Walk the chain once in order and both the failure modes and the latency budget fall out of the same walk.
The WebRTC architecture in eight stages
1. Signalling — and it is not part of WebRTC
Before any media moves, the two peers must exchange session descriptions: what codecs they support, what media they intend to send, and how to reach them. This exchange is SDP offer/answer, and the specification deliberately does not say how to transport it.
That is not an oversight. It means you build signalling yourself, usually over WebSocket, along with the identity, room membership and permission model around it. Most SDK integrations are mostly this — rooms, tokens and tracks is the shape it takes in practice.
The consequence for debugging: signalling bugs are always yours. There is no standard implementation to blame, no reference behaviour to compare against, and no browser doing it for you. It is also where a surprising share of "WebRTC problems" actually live.
2. ICE — finding a path that works
With descriptions exchanged, each side gathers candidate addresses: its local host address, its public address as seen from outside (discovered via STUN), and a relayed address on a TURN server if one is configured. The two sides then probe every plausible pairing until something connects, and pick the best one.
Most of the time a direct path works and costs you nothing. When it does not — symmetric NAT, corporate firewalls, restrictive carriers — the only option is a relay, and roughly one session in five ends up relayed on the best public measurement available.
This is the stage that produces "works on my machine" in its purest form. Your office network and your customer's office network are not the same problem, and only one of them was tested.
3. DTLS handshake — encryption is mandatory
Once a path exists, the peers run a DTLS handshake over it and derive the keys used to encrypt media as SRTP. This is not optional and cannot be turned off; unencrypted WebRTC media does not exist.
For security reviews that is a straightforward answer. For latency it is a cost: the handshake is a round trip before the first frame, which is part of why connection setup is measured in hundreds of milliseconds rather than tens. It also introduces a failure mode that reads as mysterious when you meet it — a certificate fingerprint mismatch between what was advertised in SDP and what the handshake presents fails the connection after ICE has already succeeded, which looks like the network breaking for no reason.
4. Codec negotiation — the intersection, not the preference
Both sides listed supported codecs in their session descriptions. What gets used is the intersection, ordered by preference, and the intersection is frequently smaller than either side expects.
H.264 and VP8 are effectively universal. VP9 and AV1 are widely but not uniformly supported. H.265 over WebRTC ships in Chrome 136 and Safari 18, both requiring hardware encode and decode, and is absent from Firefox and from Edge — which is notable precisely because Edge runs the same Chromium engine as Chrome and still has not enabled it.
So codec support is not a property of the standard or even of the engine. It is a shipping decision per browser, dependent on the user's hardware, and negotiated per call. The practical consequence is that H.264 stays the floor and anything better is an opportunistic upgrade you cannot rely on.
5. RTP and RTCP — the media, and the conversation about the media
Media flows as RTP packets. Alongside them runs RTCP, a feedback channel carrying reception statistics, retransmission requests for lost packets, and keyframe requests.
RTCP is where the system's self-correction lives, and it has a cost worth knowing. When a receiver misses something it cannot reconstruct, it asks for a fresh keyframe. Keyframes are large — often an order of magnitude bigger than the frames between them — so a burst of keyframe requests from multiple receivers can push a publisher's outbound bitrate up substantially at exactly the moment the network is already struggling. mediasoup's documentation notes broadcaster bitrate rising two to three times under repeated keyframe requests.
This is also why a newly joined viewer sees black for a moment: they cannot decode anything until the next keyframe arrives, and asking for one early costs the publisher bandwidth.
6. The jitter buffer — where real-time quietly goes
Packets arrive out of order and unevenly spaced. The jitter buffer holds them briefly, reorders them, and releases frames at a steady cadence so playback is smooth rather than stuttering.
It is doing exactly what it should, and it is deliberately trading latency for smoothness. The buffer is adaptive: a network with more variance gets a deeper buffer and therefore more delay. Nobody chooses this per call, and almost nobody counts it when quoting a latency number.
Which matters, because this is a real part of the delay a user experiences rather than an implementation detail. On a video wall, what that delay costs an operator watching a wall is the operational version of the same arithmetic.
7. Bandwidth estimation — reacting before it breaks
Bandwidth estimation runs continuously, inferring available capacity from the timing of arriving packets. Modern implementations watch the delay gradient — whether packets are arriving progressively later than they were sent — which signals a filling queue before any packet is actually lost.
When it detects congestion it reduces the sending rate, which means dropping resolution, frame rate, or a simulcast layer. Users read this as the call getting worse. It is the safety mechanism working: the alternative to a downgrade is not better quality, it is a stalled call.
8. Decode and render
Frames leave the jitter buffer, get decoded, and are painted to the screen. Straightforward, and the reason it belongs in the list is that it is not free — and it is where the honest latency accounting gets uncomfortable.
The chain, and who owns each link
The same eight stages, with ownership and failure mode side by side. The pattern in the middle column is the point.
Two stages touch video. Six are negotiation, transport and compensation — and every one of the six has a failure mode that presents to a user as "the video is broken."
Where the milliseconds actually go
Worth putting numbers on the chain, because the intuition is usually wrong about which links are expensive.
Transitive Robotics measured a WebRTC pipeline end to end using an infinite-mirror setup — a camera pointed at a screen showing its own feed, with high-precision timestamps overlaid. On commodity USB3 cameras at 30 fps with a 60 Hz display, capture, USB transfer and display rendering together accounted for roughly 100 ms. H.264 encode added about 10 ms, decode about 10 ms, and a remote hop through a cloud relay about 30 ms. Total: around 110 ms with no encoding at all, 130 ms locally, 170 ms remote.
Read that again in the context of the chain. The camera and the screen — the two ends nobody thinks about — cost more than the encoder, the network and the decoder combined.
This is the honest counterweight to every latency claim in this category, ours included. Sub-100ms glass-to-glass is not available on a commodity webcam, not because the protocol is slow but because the hardware at each end is. Any number below that is measuring network transport plus codec, which is a legitimate and useful figure — as long as the measurement point is stated.
What WebRTC architecture hands you, and what it doesn't
The browser gives you ICE, DTLS-SRTP, RTP handling, jitter buffering, bandwidth estimation and codec implementations. That is a substantial amount of hard engineering, standardised and free.
It does not give you signalling, identity, room membership, permissions, reconnection logic, TURN infrastructure, or any behaviour beyond two peers. Each of those is yours, and together they are most of what a video integration actually consists of.
Two peers is also the architectural limit of what the browser alone will do. A third participant means every peer sends to every other peer, which client uplink and CPU cannot sustain past four or five — at which point what an SFU is and why every platform uses one stops being background reading and becomes a component you have to run.
Build, Buy, or Deploy
Three routes, with the honest boundary on each.
Build on open source
Working directly against the browser APIs with mediasoup, Janus, Pion or LiveKit on the server gives you every stage of this chain to inspect and tune. That is genuinely valuable when you have unusual requirements, and it means owning signalling, TURN, reconnection and the browser-quirk long tail — the six stages that are not video.
Buy a cloud video API
Per-minute providers wrap the whole chain, which is the fastest route to a working feature and removes stages 1 through 7 from your problem list. The limits are visibility and control: when something fails at stage 2 or stage 7, you generally cannot see which stage it was, and you inherit the provider's policy where you might have wanted your own.
Deploy a commercial platform
The middle path runs a commercial platform on infrastructure you choose. Samvyo is one such option: based on SFU architecture, shipping embeddable SDKs, with the media path, TURN and recording kept on infrastructure you control. Stage 2 is the relevant one — TURN placement is yours, which is both the most common failure point in the chain and the one most affected by where the relay physically sits. It is resilient by design rather than something you assemble.
Where it does not fit: if you need to modify behaviour at the protocol level — custom congestion control, an unusual codec pipeline — building against the libraries directly is the honest answer. And for a two-party feature that will never need a media server, the browser alone plus a little signalling is genuinely sufficient.
The Bottom Line
Eight stages sit between two video tabs and only two of them are video. Signalling, ICE, DTLS, codec negotiation, RTCP feedback, jitter buffering and bandwidth estimation are the rest — and they are where the failures, and most of the engineering, actually live.
When browser video breaks, the odds strongly favour the route or the handshake over the media. Debug the chain in order rather than starting at the picture, and measure latency with the measurement point stated — because the camera and the screen cost more than the network does.
What's Next
The business companion, why the demo takes an afternoon and production takes months, turns this chain into a scope table you can plan against. For the stage most likely to bite first, roughly one session in five ends up relayed covers the relay layer and what it costs.
Frequently Asked Questions
What is WebRTC architecture, in simple terms?
A chain of eight stages between two endpoints: signalling to exchange session descriptions, ICE to find a network path, a DTLS handshake to derive encryption keys, codec negotiation, RTP and RTCP to carry media and feedback, a jitter buffer to smooth arrival, bandwidth estimation to adapt to congestion, and decode and render. Only the last two and the codec stage touch video.
Is signalling part of WebRTC?
No, and this surprises most people. The specification defines the session description format but deliberately not how to transport it. You build signalling yourself, typically over WebSocket, along with identity, room membership and permissions. It is one of the largest pieces of work in any integration and every bug in it is yours.
Why do WebRTC calls fail on corporate networks?
Stage two. ICE tries to find a direct path between peers, and corporate networks often prevent one — symmetric NAT, blocked UDP, firewalls allowing only outbound TCP on 443. Without a TURN relay configured for those conditions the connection simply never establishes, and it fails silently on exactly the networks enterprise customers use.
Is WebRTC media encrypted?
Always, and it cannot be disabled. After ICE finds a path, the peers run a DTLS handshake and use the derived keys for SRTP. There is no unencrypted mode. The cost is a round trip during setup, and one unusual failure mode: a certificate fingerprint mismatch fails the connection after ICE has already succeeded.
What is a jitter buffer and does it add latency?
It holds arriving packets briefly to reorder them and release frames at a steady cadence, so playback is smooth instead of stuttering. Yes, it adds latency by design, and it is adaptive — a more variable network gets a deeper buffer and more delay. It is a real component of what a user experiences, and it is routinely excluded from quoted latency figures.
Which codec should I use for WebRTC?
H.264 as the floor, since it is effectively universal. Treat anything better as an opportunistic upgrade: VP9 and AV1 are widely but unevenly supported, and H.265 ships in Chrome 136 and Safari 18 with hardware encode and decode required, while Firefox and Edge do not support it at all. Codec availability is a per-browser shipping decision, not a property of the standard.
At what point do I need a media server instead of peer-to-peer?
At the third participant. Two peers connect directly with no server in the media path. From three onward each participant must send to every other participant, which client uplink and CPU cannot sustain past four or five. A media server such as an SFU then becomes a requirement, and it is a step change in architecture rather than an incremental addition.
\
