Streaming | Council developer console

Set stream to true and the response arrives as server sent events (text/event-stream). The data: lines carry chat.completion.chunk objects in the OpenAI shape, and the stream ends with data: [DONE].

Python

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://developer.council.health/v1",
    api_key=os.environ["COUNCIL_API_KEY"],
)

response = client.chat.completions.create(
    model="YOUR_MODEL_ID",  # GET /v1/models lists yours
    messages=[{"role": "user", "content": "Summarize first line treatment for hypertension."}],
    stream=True,
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

Node.js

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://developer.council.health/v1",
  apiKey: process.env.COUNCIL_API_KEY,
});

const response = await client.chat.completions.create({
  model: "YOUR_MODEL_ID", // GET /v1/models lists yours
  messages: [{ role: "user", content: "Summarize first line treatment for hypertension." }],
  stream: true,
});

for await (const chunk of response) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

curl

curl -N https://developer.council.health/v1/chat/completions \
  -H "Authorization: Bearer $COUNCIL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "YOUR_MODEL_ID",
    "messages": [{ "role": "user", "content": "Summarize first line treatment for hypertension." }],
    "stream": true
  }'

What the stream contains

When citations are enabled and retrieval found usable passages, one extra event with "object": "council.citations" arrives just before [DONE]. It carries the same citation objects a non streaming response returns. When no passages were found, or optional retrieval could not run, the stream has no citations event and no warning.

Server sent events

data: {"object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "Guidelines"}}]}

data: {"object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " recommend"}}]}

data: {"object": "council.citations", "citations": [{"id": 1, "scope": "public_kb", "title": "..."}]}

data: [DONE]

If your SDK drops events it does not recognize, read the stream directly to receive citations, as in the curl example.

How streaming is billed

Token counts are not known while a stream is open, so streaming requests are billed on an estimate: the max_tokens you set (or 4,096 if you leave it out), priced as output tokens. Set max_tokens close to what you need. Usage for streamed requests is marked as estimated on the Usage page. A streamed request with citations enabled is billed at grounded chat rates even if optional retrieval could not run.

Errors during a stream

Authentication, balance, and validation errors return a normal JSON error before the stream starts. If the model fails after the stream has started, the connection closes early; retry the request.