demo · v141

Metadata Mux

Inject app-level metadata into the encoded media stream — caption track sync ticks, ML-derived face landmarks, watermark IDs — by appending bytes to each frame. The peer's transform peels the metadata back off before handing the frame to the decoder. One bidirectional pipeline, no separate data channel.

Heads up Requires Chrome 141+. Mind the per-codec framing — appended bytes that change frame size affect bitrate accounting and packet boundaries. The webrtc-encoded-transform explainer covers safe patterns.
checking support…

sender worker

Each encoded frame gets a 12-byte trailer with a timestamp, watermark ID, and CRC.

frame.data = withTrailer(frame.data, meta);

receiver worker

Same worker shape on the peer side. Trailer is stripped, metadata is dispatched to the UI layer; payload goes to the decoder unchanged.

const { payload, meta } = stripTrailer(frame.data);

the worker

// sender worker
self.onrtctransform = (e) => {
  const { readable, writable } = e.transformer;
  readable.pipeThrough(new TransformStream({
    transform(frame, controller) {
      const meta = currentMeta();         // e.g. captionTick: 1234
      const buf = new Uint8Array(frame.data.byteLength + 12);
      buf.set(new Uint8Array(frame.data), 0);
      writeMeta(buf, frame.data.byteLength, meta);
      frame.data = buf.buffer;
      controller.enqueue(frame);
    },
  })).pipeTo(writable);
};

why this angle

The V2 API isn't only about encryption — the same primitive (read, transform, write encoded frames) lets you mux any side-band data into the bytestream. Caption sync, watermarking for replay traceability, ML landmarks attached frame-by-frame for low-latency avatars: all things conferencing teams ship today through separate datachannels at the cost of skew. Doing it through the encoded-transform pipeline keeps the metadata locked to its source frame.

see also

scenario focus

Select a scenario to focus its rendered example and summary.