← Realtime / realtime / sfu / media-transport-adapters

WebSocket adapter

Stream audio and video between WebRTC tracks and WebSocket endpoints. Supports ingesting audio from WebSocket sources and sending WebRTC audio and video to WebSocket consumers. Video egress is supported as JPEG at approximately 1 FPS.

• AI services with WebSocket APIs for audio processing
• Custom audio processing pipelines
• Legacy system bridges
• Server-side audio generation and consumption
• Video snapshotting and thumbnails
• Computer vision ingestion (low FPS)

graph LR
    A[External System] -->|Audio Data| B[WebSocket Endpoint]
    B -->|Adapter| C[Realtime SFU]
    C -->|New Session| D[WebRTC Track]
    D -->|WebRTC| E[WebRTC Clients]

graph LR
    A[WebRTC Source] -->|WebRTC| B[Realtime SFU Session]
    B -->|Adapter| C[WebSocket Endpoint]
    C -->|Media Data| D[External System]

POST /v1/apps/{appId}/adapters/websocket/new

{
  "tracks": [
    {
      "location": "local",
      "trackName": "string",
      "endpoint": "wss://...",
      "inputCodec": "pcm",
      "mode": "buffer"
    }
  ]
}

{
  "tracks": [
    {
      "trackName": "string",
      "adapterId": "string",
      "sessionId": "string",    // New session ID generated
      "endpoint": "string"      // Echo of the requested endpoint
    }
  ]
}

{
  "tracks": [
    {
      "location": "remote",
      "sessionId": "string",
      "trackName": "string",
      "endpoint": "wss://...",
      "outputCodec": "pcm"
    }
  ]
}

{
  "tracks": [
    {
      "trackName": "string",
      "adapterId": "string",
      "sessionId": "string",    // Same as request sessionId
      "endpoint": "string"      // Echo of the requested endpoint
    }
  ]
}

POST /v1/apps/{appId}/adapters/websocket/close

{
	"tracks": [
		{
			"adapterId": "string"
		}
	]
}

• Codec: Opus
• Sample rate: 48 kHz
• Channels: Stereo

Media uses Protocol Buffers. Audio uses PCM payloads; video uses JPEG payloads:

• 16-bit signed little-endian PCM
• 48 kHz sample rate
• Stereo (left/right interleaved)
• Video: JPEG image payload (one frame per message)

message Packet {
    uint32 sequenceNumber = 1;  // Used in Stream mode only
    uint32 timestamp = 2;       // Used in Stream mode only
    bytes payload = 5;          // Media data
}

Ingest mode (buffer): Only the payload field is used, containing chunks of audio data.

Stream mode (egress):

• For audio frames:
  • sequenceNumber: Incremental packet counter
  • timestamp: Timestamp for synchronization
  • payload: Individual PCM audio frame data
• For video frames (JPEG):
  • timestamp: Timestamp for synchronization
  • payload: JPEG image data (one frame per message)
  • Note: sequenceNumber may be unset for video frames

• Supported WebRTC input codecs: H264, H265, VP8, VP9
• Output over WebSocket: JPEG images at approximately 1 FPS

Connects to your WebSocket endpoint:

1. WebSocket upgrade handshake
2. Secure connection for wss:// URLs
3. Media streaming begins

• Binary messages: PCM audio data in chunks
• Maximum message size: 32 KB per WebSocket message
• Important: Account for serialization overhead when chunking audio buffers
• Send audio in small, frequent chunks rather than large batches

• Binary messages: Individual frames with metadata (audio or video)
• Audio frames include:
  • Timestamp information
  • Sequence number
  • PCM audio frame data
• Video frames include:
  • Timestamp information
  • JPEG image data
  • Note: Sequence number may be unset for video frames
• Frames are sent individually as they arrive from the WebRTC track
• Video frames are emitted at approximately 1 FPS

1. Connects to the WebSocket endpoint
2. Audio streaming begins
3. Video streaming begins (if configured)
4. For WebRTC to WebSocket streaming, briefly retries the same endpoint after disconnects
5. Connection closes when closed, on error, or after the automatic reconnect window is exhausted

When you use the WebSocket adapter in Stream mode (egress) to send live audio or video from the SFU to your own WebSocket endpoint (WebRTC → WebSocket), the SFU automatically reconnects after brief endpoint disconnects or restarts.

The SFU retries the same WebSocket endpoint for up to 5 seconds. No API changes are required. If the endpoint remains unavailable after the reconnect window, the adapter closes and your application must create a new adapter to resume streaming.

Automatic reconnection uses live-first buffering while the WebSocket endpoint is temporarily unavailable:

• Audio buffering: The SFU keeps a short, bounded backlog of audio frames. If the interruption lasts longer than the backlog can cover, older audio may be dropped so reconnect recovery stays bounded.
• Video buffering: The SFU keeps only the latest available JPEG frame. Newer frames replace older frames while reconnecting, so video resumes near-live instead of replaying stale frames.
• Delivery behavior: Buffering reduces media loss during brief interruptions, but it is not a replay mechanism and does not guarantee gapless or exactly-once delivery.

Automatic reconnection applies only when using Stream mode (egress). It retries the same endpoint only and does not provide multi-endpoint failover.

Currently in beta and free to use.

Once generally available, billing will follow standard Cloudflare Realtime pricing at $0.05 per GB egress. Only traffic originating from Cloudflare towards WebSocket endpoints incurs charges. Traffic ingested from WebSocket endpoints into Cloudflare incurs no charge.

Usage counts towards your Cloudflare Realtime free tier of 1,000 GB.

• Closing an already-closed instance returns success
• Close when sessions end
• When using Stream mode (egress), handle adapter closure after the 5-second automatic reconnect window is exhausted.
• When ingesting from WebSocket to WebRTC, implement reconnection logic in your WebSocket client if the connection drops.
• Make your WebSocket endpoint restart-safe so it can accept reconnects to the same URL during brief restarts.

• Deploy WebSocket endpoints close to Cloudflare edge
• Use appropriate buffer sizes
• Monitor connection quality

• Secure WebSocket endpoints with authentication
• Use wss:// for production
• Implement rate limiting

• WebSocket payloads: PCM (audio) for ingest and stream; JPEG (video) for stream
• Beta status: API may change in future releases
• Video support: Egress only (JPEG)
• Video frame rate: Approximately 1 FPS (beta; not configurable)
• Streaming reconnects: When using Stream mode (egress), the SFU automatically retries the same WebSocket endpoint for short disconnects only. It does not fail over to alternate endpoints.
• Best-effort recovery: Brief reconnects reduce media loss, but do not guarantee gapless or exactly-once delivery.
• Video reconnect behavior: Video resumes from the latest available JPEG frame rather than replaying older frames.
• Unidirectional flow: Each instance handles one direction

• Audio (PCM over WebSocket): Cloudflare Realtime Examples – ai-tts-stt ↗
• Video (JPEG egress): Cloudflare Realtime Examples – video-to-jpeg ↗

1. Replace custom signaling with adapter API calls
2. Update WebSocket endpoints to handle PCM format
3. Implement adapter lifecycle management
4. Remove custom STUN/TURN configuration

Q: Can I use the same adapter for bidirectional audio? A: No, each instance is unidirectional. Create separate adapters for send and receive.

Q: What happens if the WebSocket connection drops?

A: When using Stream mode (egress), the SFU automatically retries the same WebSocket endpoint for up to 5 seconds. If the endpoint comes back within that window, streaming resumes automatically.

Audio uses a short bounded backlog to reduce audible loss during brief interruptions. Video resumes from the latest available JPEG frame instead of replaying older frames.

If the endpoint remains unavailable after the 5-second automatic reconnect window, the adapter closes and must be recreated.

When ingesting from WebSocket to WebRTC, your WebSocket client should reconnect and recreate the adapter as needed.

Q: Is there a limit on concurrent adapters? A: Limits follow standard Cloudflare Realtime quotas. Contact support for specific requirements.

Q: Can I change the audio format after creating an adapter? A: No, audio format is fixed at creation time. Create a new adapter for different formats.