Skip to content

Streaming

Turn the request into a token-by-token stream. Same auth surface as Chat; the SDK just returns an iterable instead of a completed response.

Official docs verified 2026-08-08

Native flagship

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    base_url=os.environ["AZURE_OPENAI_BASE_URL"],
)

stream = client.chat.completions.create(
    model=os.environ["AZURE_OPENAI_DEPLOYMENT"],
    messages=[{"role": "user", "content": "Stream a haiku"}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)
print()

Identical to the Kubernetes snippet — Azure Functions + Container Apps propagate the same env vars. If you're exposing the stream over HTTP from the serverless side, wrap the loop in a generator + text/event-stream response.

import os
from google import genai

client = genai.Client(
    vertexai=True,
    project=os.environ["GOOGLE_CLOUD_PROJECT"],
    location=os.environ.get("GOOGLE_CLOUD_LOCATION", "us-central1"),
)

for chunk in client.models.generate_content_stream(
    model=os.environ.get("GEMINI_MODEL", "gemini-3.6-flash"),
    contents="Stream a haiku",
):
    print(chunk.text, end="", flush=True)
print()

Identical to Kubernetes on Cloud Run — the SDK's streaming works the same regardless of runtime.

Bedrock's Converse API has a companion converse_stream that yields incremental events. Handle contentBlockDelta for text chunks; other event types (messageStart, contentBlockStart, messageStop) mark the shape.

import os, boto3

client = boto3.client(
    "bedrock-runtime",
    region_name=os.environ.get("AWS_REGION", "us-east-1"),
)
resp = client.converse_stream(
    modelId=os.environ.get("BEDROCK_MODEL_ID", "amazon.nova-pro-v1:0"),
    messages=[{"role": "user", "content": [{"text": "Stream a haiku"}]}],
)
for event in resp["stream"]:
    if "contentBlockDelta" in event:
        print(event["contentBlockDelta"]["delta"].get("text", ""), end="", flush=True)
print()

Same as K8s. If exposing over HTTP from Lambda / App Runner, wire the iterator to a text/event-stream response generator.

Claude through-line

Anthropic's SDK exposes streaming as a context manager on all three backends (.messages.stream(...)) that yields deltas via text_stream.

import os
from anthropic import AnthropicFoundry

client = AnthropicFoundry(
    api_key=os.environ["ANTHROPIC_FOUNDRY_API_KEY"],
    resource=os.environ["ANTHROPIC_FOUNDRY_RESOURCE"],
)
with client.messages.stream(
    model=os.environ.get("FOUNDRY_CLAUDE_DEPLOYMENT", "claude-opus-5"),
    max_tokens=1024,
    messages=[{"role": "user", "content": "Stream a haiku"}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
print()

import os
from anthropic import AnthropicVertex

client = AnthropicVertex(
    project_id=os.environ["GOOGLE_CLOUD_PROJECT"],
    region=os.environ.get("VERTEX_REGION", "global"),
)
with client.messages.stream(
    model=os.environ.get("VERTEX_CLAUDE_MODEL", "claude-opus-5"),
    max_tokens=1024,
    messages=[{"role": "user", "content": "Stream a haiku"}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
print()

import os
from anthropic import AnthropicBedrock

client = AnthropicBedrock(
    aws_region=os.environ.get("AWS_REGION", "us-west-2"),
)
with client.messages.stream(
    model=os.environ.get(
        "BEDROCK_CLAUDE_MODEL", "global.anthropic.claude-opus-4-6-v1"
    ),
    max_tokens=1024,
    messages=[{"role": "user", "content": "Stream a haiku"}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
print()

Notes

  • The Anthropic through-line surface is identical on all three clouds — same messages.stream call, same text_stream iterator, same event lifecycle. Only the client class + auth env vars differ.
  • The native surfaces differ: OpenAI returns delta chunks with .choices[0].delta.content; Gemini yields chunk.text on generate_content_stream; Bedrock emits typed events (contentBlockDelta, messageStop, …) via converse_stream.
  • Neither the native path nor the through-line requires any change in the deploy shape — Terraform + Helm are the same as chat.