demo · v137

Streaming meter

Pick a prompt, watch the on-device model stream tokens, and the meter graphs first-token latency, inter-token gap, and total throughput in real time. Adjust temperature and max tokens between runs to see the cost shift.

Origin trialRequires Chrome 137+ with on-device models enabled (chrome://flags/#prompt-api-for-gemini-nano). If unavailable the meter falls back to a synthetic stream so the visualization still works — clearly marked.
probing…
[awaiting stream]

first token (ms)

—

tokens / second

—

tokens total

—

last gap (ms)

—

the code

const session = await window.ai.languageModel.create({
  temperature: 0.7,
  topK: 40,
});

const stream = session.promptStreaming("Write a haiku about Chromium.");
const reader = stream.getReader();
let prev = "";
while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  // Each chunk is the *full* response so far. Diff against prev.
  process.stdout.write(value.slice(prev.length));
  prev = value;
}

see also