Stream a model response with JavaScript

Keep the API key on your server

Use this example in a Node.js server process, not in a public browser bundle. A browser application should call your authenticated backend, which applies its own access and spending controls before calling Jevrouter.

The example uses the OpenAI JavaScript SDK and server-sent events. Streaming improves the time until a user can start reading; it does not imply lower total cost or a faster completed answer.

npm install openai
export JEVROUTER_API_KEY='YOUR_PROJECT_API_KEY'

Read text and usage from the stream

Set stream_options.include_usage to request the final usage information. The usage chunk can have an empty choices array, so read usage independently from text. max_retries is disabled to keep retry decisions in your application.

import OpenAI from "openai";
import { randomUUID } from "node:crypto";

const client = new OpenAI({
  baseURL: "https://api.jevrouter.io/v1",
  apiKey: process.env.JEVROUTER_API_KEY,
  maxRetries: 0,
  timeout: 125_000,
});

const stream = await client.chat.completions.create({
  model: "openai/gpt-4.1-mini",
  messages: [{ role: "user", content: "Explain an idempotency key in three sentences." }],
  max_tokens: 512,
  stream: true,
  stream_options: { include_usage: true },
}, { headers: { "Idempotency-Key": randomUUID() } });

for await (const chunk of stream) {
  const choice = chunk.choices[0];
  if (choice?.delta?.content) process.stdout.write(choice.delta.content);
  if (choice?.finish_reason) console.error("Finish:", choice.finish_reason);
  if (chunk.usage) console.error("Usage:", chunk.usage);
}
process.stdout.write("\n");

Treat a partial answer as a distinct result

When streaming fails after content has arrived, keep the partial result and request identifier. Do not append the output of a second model to the first model's answer. Jevrouter does not transparently switch models after committing the stream.

User cancellation should abort your upstream request too. The request can still have billable usage if the supplier already produced content. Check the settled request record rather than assuming that closing the connection makes it free.

Separate tool calls from display text

This short example displays text only. Tool calls arrive as incremental arguments; applications must assemble them, validate the complete JSON and apply their own authorization before executing a tool. Model-generated arguments are untrusted input.

Measure both time to first token and time to a complete accepted result. A fast first chunk can hide a slow or truncated completion, and the source catalog does not supply a Jevrouter latency guarantee.

Models mentioned in this guide