Streaming

Handle Chat Completions and Responses SSE events separately.

Set stream: true. Responses normally use text/event-stream. Prefer SDK parsing: a network chunk is not an SSE event, and one event can span multiple chunks.

Endpoint differences

Endpoint	Text deltas	Completion and errors
Chat Completions	choices[].delta.content	Inspect finish_reason; usually ends with data: [DONE]
Responses	delta on response.output_text.delta events	Require response.completed; handle response.failed, response.incomplete, and error

Role, tool-call, and usage chunks may have no text; choices can be empty. Chat’s length means a generation limit was reached. tool_calls requires tool handling and is not a complete natural-language answer.

An HTTP 200 can still be followed by an event-level error or a dropped connection. Mark retained partial text as incomplete. Do not treat EOF as success or automatically replay a generation that has already started.

Python: Chat Completions

Follow the Python SDK guide to install dependencies and set environment variables. Save as stream-chat.py and run python stream-chat.py. This text-only example treats only finish_reason=stop as complete.

import os
from openai import OpenAI, APIConnectionError, APIStatusError, APIError

with OpenAI(
    base_url=os.environ["API_BASE_URL"].rstrip("/"),
    api_key=os.environ["API_KEY"],
    timeout=60.0,
    max_retries=0,
) as client:
    finished = False
    try:
        with client.chat.completions.create(
            model=os.environ["API_MODEL"],
            messages=[{"role": "user", "content": "Say hello in one short sentence."}],
            stream=True,
        ) as stream:
            for chunk in stream:
                for choice in chunk.choices:
                    if choice.index != 0:
                        continue
                    print(choice.delta.content or "", end="", flush=True)
                    if choice.finish_reason is not None:
                        if choice.finish_reason != "stop":
                            raise SystemExit(f"\nText did not complete: {choice.finish_reason}")
                        finished = True
        if not finished:
            raise SystemExit("\nStream ended without a completion marker.")
        print()
    except APIStatusError as error:
        raise SystemExit(f"\nRequest failed (HTTP {error.status_code}).")
    except APIConnectionError:
        raise SystemExit("\nConnection failed; partial output may be incomplete.")
    except APIError:
        raise SystemExit("\nStream failed; partial output may be incomplete.")

TypeScript: Responses

Follow the TypeScript SDK guide. Save as stream-responses.mts and run node stream-responses.mts. Select a model that supports streaming Responses.

import OpenAI from 'openai';

const { API_BASE_URL, API_KEY, API_MODEL } = process.env;
if (!API_BASE_URL || !API_KEY || !API_MODEL) {
  throw new Error('Set API_BASE_URL, API_KEY, and API_MODEL first.');
}

const client = new OpenAI({
  baseURL: API_BASE_URL.replace(/\/+$/, ''),
  apiKey: API_KEY,
  timeout: 60_000,
  maxRetries: 0,
});

let completed = false;
try {
  const stream = await client.responses.create({
    model: API_MODEL,
    input: 'Say hello in one short sentence.',
    stream: true,
  });
  for await (const event of stream) {
    if (event.type === 'response.output_text.delta') {
      process.stdout.write(event.delta);
    } else if (event.type === 'response.completed') {
      completed = true;
    } else if (['error', 'response.failed', 'response.incomplete'].includes(event.type)) {
      throw new Error('Response did not complete.');
    }
  }
  if (!completed) throw new Error('Stream ended without a completion event.');
  process.stdout.write('\n');
} catch (error) {
  console.error(error instanceof OpenAI.APIError && error.status
    ? `\nRequest failed (HTTP ${error.status}).`
    : '\nStream failed or did not complete; partial output may be incomplete.');
  process.exitCode = 1;
}

Deployment and usage

If all text arrives at once, check response buffering and timeouts in the gateway or reverse proxy. Cancelling a client reader does not guarantee that upstream execution stops.

Chat’s stream_options.include_usage requires channel support; usage events may have no choices. Responses usage usually appears in the final response. A dropped connection may prevent final usage from arriving. Consult console logs; missing usage does not mean zero usage.

Reference: OpenAI streaming events.

Native protocols and streaming tools

Claude Messages and Gemini generateContent provide dedicated native protocol examples. For argument assembly and function execution, see Chat / Responses streaming tools.

Runnable native SSE and tool loops for Claude and Gemini include completion validation and signature preservation.

简体中文