MediaStreamTrackProcessor frame counters

concepts

1. Pipeline Meter

   Live readout of totalFrames and discardedFrames from a webcam track processor. Slow down the consumer with a slider and watch dropped-frame count climb.

2. Diagnostic Overlay

   The Google Meet motivating use case. A HUD over the camera feed showing declared vs. delivered fps in real time — exactly the gap that surfaces when a USB hub halves the camera rate or the system is under load.

3. Pipeline Health Monitor

   A canvas-based video source feeds a real MediaStreamTrackProcessor. Drag the consumer-rate slider below the 30 fps production rate and watch totalFrames and discardedFrames diverge on a live chart — with a health indicator that turns red when the discard rate exceeds 20%.

4. Frame Budget Calculator

   Enter your source frame rate, per-frame transform time, and stream buffer size. The calculator computes your frame budget, budget utilization, and predicted discardedFrames count over a simulation window — with a visual timeline showing how much of each frame interval is consumed by the transform. Use this to size transforms before shipping.

   Calculator Performance planning

5. Counters API Explorer

   Feature-detects totalFrames and discardedFrames, then creates a canvas-based MediaStream source and attaches a processor. Reads both counters every second via a live property panel. Raise the consumer delay slider to induce drops and watch discardedFrames climb. Falls back to a simulation if Chrome 150 is not available.

   Live API Feature detection

6. Worker Processor

   The recommended architecture: move the MediaStreamTrackProcessor into a dedicated Worker thread and post totalFrames / discardedFrames back to main via postMessage. The main thread receives only health stats — never raw frames. A canvas-based synthetic stream produces frames at 30 fps; a consumer-delay slider induces frame drops in the worker. Full architecture diagram and postMessage log included.

   Live API Workers Architecture

why it shipped

Real-time video pipelines built on Insertable Streams of Media (frame-by-frame processing via WHATWG streams) are notoriously hard to diagnose. If the consumer can't keep up with the producer — for example, because a frame transform is too slow on the device — frames are silently dropped on the floor. Until now there was no API to detect this. The two counters give pipelines a simple health probe: a non-zero growth rate on discardedFrames means you're losing data and should back off the per-frame work.

references

• ChromeStatus entry
• Insertable Streams of Media spec