Real-time video on the web has transitioned from peer-to-peer (Mesh) architectures to centralized servers. In this article, we dive into how Selective Forwarding Units (SFUs) work, the orchestration challenges they present, and how CoveKit handles this complexity so you don’t have to.

Mesh vs. SFU architectures

In a standard Mesh (P2P) network, every participant in a call sends their audio and video tracks directly to every other participant.

  • Mesh Bandwidth Equation: If there are $N$ participants, each user must upload $N-1$ streams and download $N-1$ streams.
  • The Limit: For a call with 5 participants, this requires uploading 4 video streams simultaneously. For typical residential upload connections, this immediately leads to packet loss, frame drops, and frozen feeds.

An SFU (Selective Forwarding Unit) solves this by acting as a routing hub. Each participant uploads exactly one video stream and one audio stream to the server. The SFU then selectively forwards those streams to the other $N-1$ participants.

This drops the upload requirement to $1$ stream regardless of the meeting size, drastically improving service quality.

The Orchestration Challenge

While SFUs solve the client bandwidth problem, they introduce severe backend orchestration overhead:

  1. RTP Parameters Exchange: The client and server must negotiate codecs, packet payloads, and security keys (DTLS/SRTP).
  2. Dynamic Transports: Separating upload (send) and download (recv) transports to optimize connection handshakes.
  3. Consumer Lifecycle Management: Ensuring that when a participant turns their camera off or leaves, all downstream consumers are cleaned up, avoiding orphaned transport allocations.

How CoveKit Simplifies SFU Orchestration

CoveKit handles the low-level SDP/RTP negotiations and transport management behind a simple, unified interface.

When a client joins a CoveKit room:

const room = new CoveKitRoom(client, {
  onLocalStream: (stream) => {
    // Render your local preview
  },
  onRemoteTrack: (track, peerId, kind) => {
    // Attach the track to a remote <video> or <audio> player
  }
});

Under the hood:

  • CoveKit instantiates your send and receive transports.
  • It dynamically manages producer registration.
  • It automatically subscribes you to new participants’ tracks as they join.

This allows developers to build high-performance video products without having to write a single line of low-level WebRTC transport code.

Stay tuned for our next article, where we will build a complete React video room in under 50 lines of code!