> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trylimitless.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming

> Read text incrementally and handle usage, completion, and interruption.

Set `stream: true` to receive server-sent events. The OpenAI SDK parses events for you.

## Chat completion stream

```javascript theme={null}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.LIMITLESS_API_KEY,
  baseURL: process.env.LIMITLESS_BASE_URL,
  maxRetries: 0,
});

const stream = await client.chat.completions.create({
  model: process.env.LIMITLESS_MODEL,
  messages: [{ role: "user", content: "Write a short welcome message." }],
  max_tokens: 512,
  stream: true,
  stream_options: { include_usage: true },
});

try {
  for await (const chunk of stream) {
    process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
    if (chunk.usage) console.log("\nUsage:", chunk.usage);
  }
} catch (error) {
  console.error("The stream did not finish:", error.message);
}
```

The gateway requests `include_usage: true` for all chat streams, even when the caller omits it or sets it to `false`. A provider usage chunk can have an empty `choices` array. Always check that a choice exists before reading its delta.

At the wire level, events are `data: {JSON}` records separated by a blank line. Chat streams normally terminate with `data: [DONE]`. Network chunks do not necessarily align with event boundaries; use the SDK or a proper SSE parser.

## Responses stream

The Responses endpoint emits typed events. See [stateless Responses](/guides/responses) for a complete example. Do not parse Responses events as chat `choices` chunks.

## Handle interruption

When the gateway observes your client disconnect, it cancels the upstream request. The backend may already have processed part of it. Test cancellation through your deployed client and proxy path: closing the local stream alone does not prove that the disconnect reached the gateway.

* A final usage event may be absent after interruption. Missing token counts remain **unknown** in Activity.
* HTTP `200` indicates that a stream started; it does not prove the full generation completed.
* A failed stream can retain partial text. Decide whether your application should display, discard, or label it.
* Do not replay a partial request automatically without considering duplicate generation and charges.

For an observed customer Stop/disconnect, confirmed final usage is charged normally. If final usage is unavailable, the gateway releases the [credit reservation](/billing), makes no customer charge and leaves the token counts unknown; Limitless absorbs any unmeasured supplier cost. Unknown usage from timeouts, provider failures or platform shutdown can still retain a hold for reconciliation.

The streaming examples use the official [OpenAI streaming interface](https://developers.openai.com/api/docs/guides/streaming-responses). Limitless endpoint coverage is described in [compatibility](/guides/compatibility).

## Messages and the console Playground

The [Messages endpoint](/guides/messages) streams native Anthropic events. Its event names and content-block structure differ from Chat and Responses; use the matching SDK.

The console Playground displays text as it arrives. **Stop** cancels the browser request and asks the backend to stop generation. Text already received stays visible and is marked as stopped or incomplete. Once the gateway observes the cancellation, it charges final reported usage or releases the reservation if final usage is unavailable. This does not guarantee that the supplier used no further tokens. Review Activity for the recorded outcome and Billing for the settled charge or released reservation.

Response [policies](/guides/policies) can delay visible output while evaluation runs. This may make a policy-protected request look less incremental even though the client requested streaming.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.