> ## Documentation Index > Fetch the complete documentation index at: https://docs.breezeblue.ai/llms.txt > Use this file to discover all available pages before exploring further. # Realtime text to speech > Use the text-to-speech WebSocket for realtime conversation audio. Realtime text to speech is a WebSocket channel for conversational products. Reuse one physical connection across nearby turns to avoid repeated setup, while treating a long conversation as a logical session made of bounded WebSocket connections. Use `POST /v1/text-to-speech/{voice_id}/stream` for one-shot streaming generation. Use `wss://api.breeze.blue/v1/text-to-speech/{voice_id}/stream-input` when your product sends text incrementally during a live conversation. The realtime WebSocket always streams raw `pcm_s16le`, 24000 Hz, mono, 16-bit audio as binary frames. It does not accept `output_format`; use the HTTP text-to-speech endpoints for `mp3` or `wav`. ## Choose an integration Create a browser-safe session, exchange JSON turn events, and save binary PCM audio. Use typed connection helpers while consuming audio concurrently with incremental text input. Mint a short-lived token on your backend and play PCM chunks with the Web Audio API. Validate credentials, voice routing, TTFA, and multi-turn behavior before writing application code. ## Create a browser session Create short-lived browser tokens from your backend. Do not expose long-lived API keys in browser code. ```bash theme={null} curl -X POST "https://api.breeze.blue/v1/text-to-speech/voc_xeh3w54cqvnp/realtime-sessions" \ -H "xi-api-key: $BREEZE_API_KEY" \ -H "content-type: application/json" \ -d '{ "model_id": "breeze-tts-2", "language_code": "en", "instructions": "Keep turns responsive and conversational.", "voice_settings": { "guidance_scale": 4.0 }, "inactivity_timeout_seconds": 30 }' ``` This HTTP request only creates the short-lived session. It does not synthesize audio. Official SDKs and the CLI use `direct_websocket_url` when the response provides it, so the credential stays out of the URL. Older clients and custom integrations can continue to open `websocket_url`. Wait for `session.ready`, send a turn, and keep consuming frames until `turn.done`. The response includes `client_secret`, `websocket_url`, `expires_at`, and the fixed `audio_format`. It also includes `direct_websocket_url` when the direct transport is available. `model_id`, `language_code`, `voice_settings`, inactivity timeout, and logging behavior are fixed when the session is created. `instructions` supplies the initial value and is the one session setting that can later be replaced between turns through the WebSocket. When `voice_settings.guidance_scale` is omitted, the voice's saved setting is used. The `client_secret` expires 60 seconds after the session is created; open the WebSocket within that window. The token is validated only at connect time — once the connection is established, the session can stay open for up to 30 minutes regardless of the token expiry. ## Connect ```text theme={null} GET wss://api.breeze.blue/v1/text-to-speech/{voice_id}/stream-input?client_secret=... ``` When `direct_websocket_url` is present, connect to that query-free URL and offer these two WebSocket subprotocols in order: ```text theme={null} breeze-realtime-v1 breeze-realtime-token.{client_secret} ``` The server selects only `breeze-realtime-v1`; it never echoes the token-bearing protocol. A direct client must fail the connection if the selected protocol is different or absent. The `client_secret` remains a short-lived credential: do not log the offered protocols or put the token into URLs. Browsers can pass the protocol array to the native `WebSocket` constructor, and server-side clients can use their WebSocket library's subprotocol option. Server-side clients may also connect with `Authorization: Bearer ...` or `xi-api-key`. Browser clients should use `client_secret`. API-key connections accept the `model_id`, `language_code`, `inactivity_timeout_seconds`, and `enable_logging` query parameters. `client_secret` connections accept no other query parameters: initial session settings come from the session-creation request, and any additional query parameter is rejected with `VALIDATION_ERROR`. After the connection is ready, use `session.update` to replace `instructions`; query parameters cannot update it. `output_format` is always rejected. Wait for the `session.ready` event before starting the first turn; it carries the session limits your client should honor. ## Raw WebSocket quickstart This Node.js 22+ example uses only `fetch` and the native `WebSocket`. It creates a browser-safe session, waits for `session.ready`, sends one turn, and writes the binary audio frames to `out.pcm`. ```javascript theme={null} import { writeFile } from "node:fs/promises"; const apiKey = process.env.BREEZE_API_KEY; if (!apiKey) throw new Error("Set BREEZE_API_KEY before running this script"); const voiceId = "voc_xeh3w54cqvnp"; const response = await fetch( `https://api.breeze.blue/v1/text-to-speech/${voiceId}/realtime-sessions`, { method: "POST", headers: { "content-type": "application/json", "xi-api-key": apiKey, }, body: JSON.stringify({ model_id: "breeze-tts-2", inactivity_timeout_seconds: 30, }), }, ); if (!response.ok) { throw new Error(`Session creation failed: ${response.status} ${await response.text()}`); } const session = await response.json(); const websocketUrl = session.direct_websocket_url ?? session.websocket_url; const protocols = session.direct_websocket_url ? ["breeze-realtime-v1", `breeze-realtime-token.${session.client_secret}`] : undefined; const ws = new WebSocket(websocketUrl, protocols); ws.binaryType = "arraybuffer"; const pcmChunks = []; await new Promise((resolve, reject) => { const timeout = setTimeout(() => { ws.close(); reject(new Error("Timed out waiting for turn.done")); }, 30_000); ws.onerror = () => { clearTimeout(timeout); ws.close(); reject(new Error("WebSocket connection failed")); }; ws.onopen = () => { if ( session.direct_websocket_url && ws.protocol !== "breeze-realtime-v1" ) { clearTimeout(timeout); ws.close(); reject(new Error("Direct WebSocket did not negotiate breeze-realtime-v1")); } }; ws.onmessage = (event) => { if (typeof event.data !== "string") { pcmChunks.push(Buffer.from(event.data)); return; } const message = JSON.parse(event.data); if (message.type === "session.ready") { ws.send(JSON.stringify({ type: "turn.start", turn_id: "turn_1" })); ws.send(JSON.stringify({ type: "text.append", text: "Hello from Breeze." })); ws.send(JSON.stringify({ type: "text.flush" })); ws.send(JSON.stringify({ type: "turn.end" })); } else if (message.type === "error") { clearTimeout(timeout); ws.close(); reject(new Error(`${message.code}: ${message.message}`)); } else if (message.type === "turn.done") { clearTimeout(timeout); ws.send(JSON.stringify({ type: "session.close" })); resolve(); } }; }); await writeFile("out.pcm", Buffer.concat(pcmChunks)); ``` The output is headerless `pcm_s16le`, 24000 Hz, mono, 16-bit audio. Use the [browser example](#browser-playback-with-the-native-websocket) for chunk-by-chunk playback or the [CLI](#cli) to save a WAV container without writing audio plumbing. ## Client events Send text frames as JSON. Binary client frames are rejected with a `VALIDATION_ERROR` error frame; the session stays open. | Event | Payload | Description | | ---------------- | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | `turn.start` | `turn_id` (optional string) | Start a turn. Omit `turn_id` to let the server assign `turn_1`, `turn_2`, ... Only one turn can be active at a time. | | `text.append` | `text` (required, non-empty string) | Append text to the active turn. Each payload is at most 2 KB (2048 UTF-8 bytes). | | `text.flush` | — | Ask the model to synthesize the text buffered so far instead of waiting for more context. | | `turn.end` | — | Mark the turn's text as complete. Audio keeps streaming until `turn.done`. | | `turn.cancel` | — | Cancel the active turn (barge-in). The server replies with `turn.cancelled`. | | `session.update` | `instructions` (required, non-empty string, maximum 1,000 characters) | Replace the synthesis instructions for subsequent turns. Send only between turns and wait for `session.updated` before `turn.start`. | | `ping` | — | Keepalive. The server replies with `pong` and the session's idle timer resets. | | `session.close` | — | Graceful close. The server cancels any active turn, sends `session.closed`, and closes with code 1000. | ```json theme={null} {"type":"turn.start","turn_id":"turn_1"} {"type":"text.append","text":"Hello "} {"type":"text.flush"} {"type":"turn.end"} ``` ### Update instructions between turns Keep the same WebSocket open when only the speaking direction needs to change. Send one update while the session is idle: ```json theme={null} {"type":"session.update","instructions":"Speak faster and with more energy."} ``` The server forwards the update through the active synthesis connection and acknowledges it only after the new value has been applied: ```json theme={null} {"type":"session.updated","instructions":"Speak faster and with more energy."} ``` The acknowledged value applies to the next `turn.start` and later turns. Wait for `session.updated` before starting that next turn. Only one update may be pending at a time. An update sent while a turn is active receives a recoverable `BAD_REQUEST` error; it does not cancel or otherwise interrupt the active turn, and the previous instructions remain in effect. `instructions` must be a string with non-whitespace content and at most 1,000 characters. Other session settings cannot be changed over the WebSocket. To change the model, language, voice settings, inactivity timeout, or logging behavior, create a new session. ### Limits * Each `text.append` payload is at most 2 KB (2048 UTF-8 bytes). Read the exact value from `session.ready.max_append_bytes`. * Total turn text is capped at the text-to-speech generation limit, currently 1,000 characters. Read the exact value from `session.ready.max_turn_characters`. * One session has one active turn at a time. Wait for `turn.done` or `turn.cancelled` before sending the next `turn.start` — a `turn.start` while a turn is active is a protocol error that cancels the active turn. * Send `session.update` only between turns, with at most one update awaiting `session.updated`. The acknowledged instructions apply from the next turn. * `client_secret` tokens expire 60 seconds after session creation. * Idle timeout defaults to 30 seconds and can be set up to 180 seconds. * A session can stay open for up to 30 minutes. * The server sends `session.expiring` about two minutes before the maximum lifetime so the client can prepare a replacement connection. * Each API key can hold up to 20 concurrent realtime sessions. See [Rate limits](/reference/rate-limits). ## Server events Audio is sent only as binary PCM frames. Decode it as signed 16-bit little-endian PCM at 24000 Hz, mono. All other server frames are JSON text frames with a `type` field. ### `session.ready` Sent once after the session is set up. Client events are processed after this frame. | Field | Type | Description | | ---------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `session_id` | string | Server-assigned session identifier. | | `audio_format` | object | Fixed stream format: `codec` (`pcm_s16le`), `sample_rate` (24000), `channels` (1), `sample_width_bits` (16). | | `max_append_bytes` | integer | Maximum UTF-8 bytes per `text.append` payload (2048). | | `max_turn_characters` | integer | Maximum total characters per turn (1000). | | `inactivity_timeout_seconds` | integer | Effective idle timeout for this session. | | `max_session_seconds` | integer | Maximum session lifetime (1800). | | `expires_at` | string | Absolute UTC deadline for this physical WebSocket session in RFC 3339 format. Use a monotonic timer based on `max_session_seconds` for local scheduling. | ### `session.updated` Acknowledges an idle `session.update` after the synthesis connection has applied the new instructions. | Field | Type | Description | | -------------- | ------ | --------------------------------------------------------------------------------------------------------------------------- | | `instructions` | string | Exact confirmed value. Use it for the next turn and retain it when your application creates a replacement physical session. | ### `session.expiring` Sent once, about two minutes before the physical WebSocket reaches its maximum lifetime. It is advisory: the current connection remains usable until `expires_at`, but a long-running client should open a fresh session and move the next turn to it. Never move or replay a turn that is already active. | Field | Type | Description | | --------------------- | ------- | --------------------------------------------------------------- | | `session_id` | string | Session that is approaching its deadline. | | `expires_in_seconds` | integer | Approximate whole seconds remaining when the event was emitted. | | `max_session_seconds` | integer | Maximum lifetime of this physical WebSocket. | | `expires_at` | string | Same absolute UTC deadline sent in `session.ready`. | ### `turn.started` Acknowledges `turn.start`. | Field | Type | Description | | ----------------- | ------ | ----------------------------------- | | `turn_id` | string | The active turn id. | | `history_item_id` | string | History item created for this turn. | ### `audio.started` Sent once per turn, immediately before the turn's first binary audio frame. | Field | Type | Description | | -------------- | ------- | -------------------------------------------------------- | | `turn_id` | string | The active turn id. | | `ttfa_ms` | integer | Time to first audio, measured from `turn.start` receipt. | | `audio_format` | object | Same fixed format as `session.ready.audio_format`. | ### `turn.done` Sent when all audio for a completed turn has been delivered. | Field | Type | Description | | ----------------- | --------------- | ----------------------------------------------------------------------------------------- | | `turn_id` | string | The completed turn id. | | `history_item_id` | string | History item for this turn. | | `status` | string | `done`. | | `text_characters` | integer | Characters of appended text after Unicode normalization; this is the billable unit count. | | `audio_bytes` | integer | Total PCM bytes delivered for the turn. | | `duration_ms` | integer or null | Duration of the delivered audio. | | `ttfa_ms` | integer or null | Time to first audio for the turn. | ### `turn.cancelled` Same fields as `turn.done` with `status` set to `cancelled`. Sent after `turn.cancel`, when a turn ends without producing any audio (for example `turn.end` with no appended text), when a protocol error cancels the active turn, or when the session closes with a turn still active. ### `usage.committed` Sent after `turn.done`, once billing for the turn has been booked. It is emitted only for completed turns and can arrive while a later turn is already running. `usage.committed` is a best-effort notification on the physical WebSocket that produced the turn. If the client closes or rotates that WebSocket after `turn.done` but before background settlement completes, the charge still commits even though the event can no longer be delivered. Use history and usage APIs as the durable billing record; do not interpret a missing `usage.committed` frame as an unbilled turn. | Field | Type | Description | | ----------------- | --------------- | --------------------------------- | | `turn_id` | string | The billed turn id. | | `history_item_id` | string | History item for the turn. | | `text_characters` | integer | Billed character count. | | `output_format` | string | `pcm`. | | `ttfa_ms` | integer or null | Time to first audio for the turn. | ### `pong` Reply to a client `ping`. No additional fields. ### `session.closed` Reply to a client `session.close`, sent before the server closes the WebSocket with code 1000. | Field | Type | Description | | ------------ | ------ | ---------------------- | | `session_id` | string | The closed session id. | ### `error` | Field | Type | Description | | --------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `code` | string | Machine-readable error code from the [error reference](/reference/errors). | | `message` | string | Human-readable description. This is the WebSocket counterpart of the HTTP envelope's `detail`. | | `meta` | object (optional) | Extra context. Restart signals set `reconnect: true`, `retry_after_ms`, `reason`, and the interrupted `turn_id`. Unrecognized upstream codes are mapped to `UPSTREAM_GENERATION_ERROR` with the original code in `meta.upstream_code`. Session-limit rejections include `meta.max_sessions_per_key`; `GENERATION_CAPACITY_EXCEEDED` and `GENERATION_CONCURRENCY_EXCEEDED` include `meta.retry_after_seconds`. | ## Errors and close codes Not every error ends the session: * **Validation, protocol, and turn-capacity errors** (`VALIDATION_ERROR`, `BAD_REQUEST`, `GENERATION_CAPACITY_EXCEEDED`, `GENERATION_CONCURRENCY_EXCEEDED`) send an `error` frame and keep the session open. Errors caused by turn commands cancel the active turn and produce `turn.cancelled`. A `session.update` rejected because a turn is active is the exception: the current turn continues and no `turn.cancelled` event is sent. * **Authentication and policy errors** (`AUTH_REQUIRED`, `AUTH_SESSION_EXPIRED`, `BILLING_INSUFFICIENT_CREDITS`, `RESOURCE_NOT_FOUND`, `RATE_LIMITED`) send an `error` frame, then close with code 1008. * **Server and upstream failures** (`UPSTREAM_GENERATION_ERROR`, `UPSTREAM_TIMEOUT`, `GENERATION_INVALID_RESPONSE`, `AUTH_CONFIG_MISSING`, and other contract codes reported by the synthesis service) send an `error` frame, then close with code 1011 — or 1012 with `meta.reconnect: true` when reconnecting is expected to succeed. * **Timeouts** send an `error` frame with `IDLE_TIMEOUT` (close code 1001) or `SESSION_EXPIRED` (close code 1000). | Close code | Meaning | Client action | | ---------- | ---------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | | 1000 | Graceful close: `session.close` acknowledged, or the 30-minute session lifetime was reached (`SESSION_EXPIRED`). | Open a new session when you need one. | | 1001 | Idle timeout (`IDLE_TIMEOUT`): no client frames and no active synthesis for `inactivity_timeout_seconds`. | Reconnect when the conversation resumes; send `ping` to keep future sessions alive. | | 1008 | Rejected: invalid or expired credentials, insufficient credits, unknown voice, or too many concurrent sessions. | Fix the request; do not retry with the same credentials or token. | | 1011 | Server-side failure. | Retry with backoff. | | 1012 | Service restarting. The preceding `error` frame carries `meta.reconnect: true` and `meta.retry_after_ms`. | Reconnect with a new session after `retry_after_ms`. | ## Keepalive and timeouts The idle timer resets on every client frame and, while a turn is synthesizing, on every audio or event frame from the synthesis service. A turn that keeps producing audio longer than `inactivity_timeout_seconds` is never cut off as idle — only a session with no client traffic and no active synthesis times out. During idle gaps between turns, send `ping` well inside the `inactivity_timeout_seconds` window; the server replies with `pong`. A practical interval is half the effective idle timeout, capped at 15 seconds, with a small amount of jitter: ```json theme={null} {"type":"ping"} ``` Track the reply as well as the write. A `ping` send can succeed on a half-open connection after inbound traffic has stopped. Raw WebSocket clients should treat a missing `pong` or other inbound frame within their acknowledgement deadline as a transport failure; managed SDKs use a 5-second deadline by default. Independent of activity, a session closes after 30 minutes with a `SESSION_EXPIRED` error frame and close code 1000. Long-running products should react to `session.expiring` by creating a fresh session, waiting for its `session.ready`, routing the next turn to the replacement, and gracefully closing the old connection after any active turn reaches `turn.done` or `turn.cancelled`. ## Billing, history, and privacy Realtime turns are billed per character of appended text, counted after Unicode normalization — the same metering as HTTP text to speech. Billing is committed when the turn settles: * A completed turn bills `text_characters` and emits `usage.committed` once the charge is booked. * A cancelled turn that already delivered audio still bills the appended text; `usage.committed` is not emitted for cancelled turns. * A turn that never produced audio (for example `turn.end` with no text, or a cancel before the first audio frame) is not billed. * Delivered audio is always billed, even if your balance runs out mid-turn: the balance can go negative and is offset automatically by your next top-up. Once the balance reaches zero, new turns are rejected with `BILLING_INSUFFICIENT_CREDITS`. Each turn creates a history item (`history_item_id` in `turn.started` and `turn.done`). With logging enabled (the default), the turn text and audio are stored and retrievable through the [history endpoints](/guides/managing-history). With `enable_logging: false`, Breeze keeps the history record for accounting and billing but never stores the turn text or audio. ## Minimize time to first audio `text.flush` tells the model to synthesize the text buffered so far instead of waiting for more context. Use it at natural sentence boundaries — flushing mid-phrase can hurt prosody, while never flushing delays the first audio until `turn.end`. When the text comes from a streaming LLM, forward tokens as they arrive and flush on sentence boundaries: 1. Send `turn.start` as soon as the LLM starts responding. 2. Send each text delta as `text.append` (split deltas larger than 2 KB). 3. Send `text.flush` when a sentence completes (`.`, `?`, `!`, or newline). 4. Send `turn.end` when the LLM response finishes. Keep session setup off the critical path: create the realtime session ahead of time (for example while the user is still speaking) and open the WebSocket before the first text is ready. Time the pre-connect against the 60-second `client_secret` expiry — the token only needs to be valid at connect time, so connecting immediately and holding the open session is the safest pattern. Measure your integration with `audio.started.ttfa_ms`, which reports the server-observed time from `turn.start` to the first audio frame for every turn. ## Connection lifecycle Realtime WebSocket sessions are optimized for live conversation TTFA. They are not resumable audio streams. Network changes, service deployments, and upstream realtime worker restarts can close an active WebSocket. Between turns, Breeze may transparently replace an interrupted downstream synthesis connection. This recovery is bounded and only happens when no turn is active; the public WebSocket stays open and retains the latest acknowledged instructions together with the other session settings. If the downstream connection cannot be restored, the public session closes with the reconnect signal below. For conversations longer than one physical session, keep conversation, ASR, LLM, and playback state in your application. Open the replacement WebSocket before the current deadline, wait for `session.ready`, and switch only at a turn boundary. Keep the overlap short so planned rotation does not consume two session slots for longer than necessary. If the connection closes before `turn.done` or `turn.cancelled`, treat the active turn as interrupted. Stop playback for that turn, discard or explicitly mark any partial audio in your UI, create a new realtime session when using browser `client_secret`, reconnect, and start a new turn from your own conversation state. Do not reuse `turn_id` as a resume token; it is only a client correlation id. When Breeze can send a structured restart signal, it sends an `error` frame with `meta.reconnect: true` and `meta.retry_after_ms`, then closes the WebSocket with close code `1012`. The SDKs also surface abnormal WebSocket closes as a `session.closed` event; `session.closed.reconnect` is `true` for close code `1012`. Use short exponential backoff with jitter for reconnects, for example 250 ms, 500 ms, 1 s, then 2 s, capped at 5 s. Browser clients should request a fresh `client_secret` from your backend after a reconnect failure or token expiry. ## SDK quickstart Start consuming the connection before appending text. Audio can arrive as soon as `flush()` is processed and may continue after `endTurn()` / `end_turn()`; keep the single consumer running until `turn.done`. TypeScript events use camelCase fields (`turnId`, `historyItemId`, `ttfaMs`); the wire protocol and the Python SDK use snake\_case. ```typescript theme={null} import { writeFile } from "node:fs/promises"; import { BreezeBlueClient } from "@breeze.blue/sdk"; const client = new BreezeBlueClient({ apiKey: process.env.BREEZE_API_KEY!, }); const connection = await client.textToSpeech.realtime.connect("voc_xeh3w54cqvnp", { modelId: "breeze-tts-2", instructions: "Keep the turn warm and natural.", }); const pcmChunks: Uint8Array[] = []; const consumer = (async () => { for await (const message of connection) { if (message.type === "audio") { // Forward this chunk to your playback or transport layer immediately. pcmChunks.push(message.audio); } else if (message.type === "error") { throw new Error(`Realtime TTS failed: ${message.code}: ${message.message}`); } else if (message.type === "turn.cancelled") { throw new Error(`Turn cancelled: ${message.turnId}`); } else if (message.type === "turn.done") { return; } } throw new Error("Realtime session closed before turn.done"); })(); connection.startTurn("turn_1"); for (const delta of ["Hello ", "from Breeze."]) { connection.appendText(delta); if (/[.!?]\s*$/.test(delta)) connection.flush(); } connection.endTurn(); await consumer; await writeFile( "out.pcm", Buffer.concat(pcmChunks.map((chunk) => Buffer.from(chunk))), ); connection.close(); ``` ```python theme={null} import asyncio import os from pathlib import Path from breeze_blue import BreezeBlue client = BreezeBlue(api_key=os.environ["BREEZE_API_KEY"]) async def consume_audio(connection) -> bytes: audio = bytearray() async for event in connection: if event["type"] == "audio": # Forward this chunk to your playback or transport layer immediately. audio.extend(event["audio"]) elif event["type"] == "error": raise RuntimeError(event.get("message")) elif event["type"] == "turn.cancelled": raise RuntimeError(f"Turn cancelled: {event['turn_id']}") elif event["type"] == "turn.done": return bytes(audio) raise RuntimeError("Realtime session closed before turn.done") async def speak() -> None: async with client.text_to_speech.connect_realtime( voice_id="voc_xeh3w54cqvnp", model_id="breeze-tts-2", instructions="Keep the turn warm and natural.", ) as connection: consumer = asyncio.create_task(consume_audio(connection)) await connection.start_turn("turn_1") for delta in ("Hello ", "from Breeze."): await connection.append_text(delta) if delta.rstrip().endswith((".", "!", "?")): await connection.flush() await connection.end_turn() Path("out.pcm").write_bytes(await consumer) asyncio.run(speak()) ``` For a conversation that can outlive one physical WebSocket, use the opt-in managed connection: ```typescript theme={null} const connection = await client.textToSpeech.realtime.connectManaged( "voc_xeh3w54cqvnp", { modelId: "breeze-tts-2" }, ); await connection.startTurnWhenReady("turn_1"); connection.appendText("Hello from a long-running conversation."); connection.endTurn(); ``` ```python theme={null} async with client.text_to_speech.connect_managed( voice_id="voc_xeh3w54cqvnp", model_id="breeze-tts-2", ) as connection: await connection.start_turn("turn_1") await connection.append_text("Hello from a long-running conversation.") await connection.end_turn() ``` The managed APIs derive a safe heartbeat from `session.ready`, obtain fresh session credentials, and move the next turn to a replacement WebSocket at a turn boundary. By default, a managed physical epoch becomes eligible for rotation when it reaches 600 seconds even though the raw WebSocket server allows up to 30 minutes. An active turn remains on its original epoch and can delay the switch beyond the 600-second threshold; the 30-minute server hard limit still applies. Set Python `max_physical_session_seconds=0` or TypeScript `maxPhysicalSessionMs: 0` to disable the client age threshold and rely on the server-advertised lifetime. Both SDK connection types expose the same update operation: TypeScript uses `connection.updateInstructions(...)`, and Python uses `await connection.update_instructions(...)`. Keep consuming the existing iterator until its matching `session.updated` event arrives, then start the next turn. Managed connections retain the confirmed value across physical-session rotation and idle reconnect. SDK-created replacement sessions include it at creation; replacements created by a custom `sessionFactory` receive the confirmed value before the logical iterator exposes their next `session.ready`. Each managed heartbeat must be acknowledged by `pong` or another inbound server frame within 5 seconds. A successful WebSocket write alone is not proof that both directions are healthy. Python exposes this wait as `heartbeat_timeout_seconds`; TypeScript exposes it as `heartbeatTimeoutMs`. When the acknowledgement times out between turns, the manager opens a fresh physical epoch with bounded retry. When it times out or the transport closes during an active turn, the manager raises `TURN_INTERRUPTED` and never replays the turn; rebuild any retry from application conversation state. The same logical iterator emits another `session.ready` for each physical epoch. Python `start_turn(...)` waits when an idle replacement is already in progress. TypeScript keeps the existing synchronous `startTurn(...)` API and provides `await startTurnWhenReady(...)` for the same race-safe behavior. These methods wait only before `turn.start` has been sent and send it once on the replacement; append, flush, end, and cancel commands are not buffered. Once a turn start write begins, a connection failure remains `TURN_INTERRUPTED` and is never replayed. A managed connection represents an active realtime call, not user presence. Exit the Python async context or call TypeScript `connection.close()` when the call ends, the user leaves, or the page enters a long-lived background state. The automatic heartbeat otherwise continues to occupy a server WebSocket slot by design. Browser TypeScript clients provide a `sessionFactory` that obtains a newly minted session from their own backend for every physical epoch: ```typescript theme={null} import { BreezeBlueRealtimeError } from "@breeze.blue/sdk"; const browserClient = new BreezeBlueClient(); const connection = await browserClient.textToSpeech.realtime.connectManaged( "voc_xeh3w54cqvnp", { sessionFactory: async ({ signal }) => { const response = await fetch("/api/breeze-realtime-session", { method: "POST", signal, }); if (!response.ok) { const reconnect = [408, 425, 429].includes(response.status) || response.status >= 500; throw new BreezeBlueRealtimeError("Could not create realtime session", { code: "SESSION_FACTORY_ERROR", reconnect, }); } const session = await response.json(); return { clientSecret: session.client_secret, websocketUrl: session.websocket_url, directWebsocketUrl: session.direct_websocket_url, }; }, }, ); ``` The factory must distinguish transient setup responses (`408`, `425`, `429`, and `5xx`) from terminal `4xx` policy or configuration responses. A plain JavaScript `Error` cannot carry this distinction and is retried only inside the same bounded setup budget. Both SDKs also expose a low-level keepalive `ping()` and support connecting with a pre-created session's `client_secret` to keep setup off the critical path. Every SDK connection has exactly one event consumer; do not run separate audio and event iterators at the same time. See the SDK READMEs for error types and tuning options. ## Browser playback with the native WebSocket Browsers can talk to the realtime endpoint directly with a `client_secret` minted by your backend — no SDK and no API key in the page. This example schedules PCM chunks on an `AudioContext`: ```javascript theme={null} // Your backend calls POST /v1/text-to-speech/{voice_id}/realtime-sessions // and returns the session credentials to the page. const session = await fetch("/api/realtime-session", { method: "POST", }).then((response) => response.json()); const websocketUrl = session.direct_websocket_url ?? session.websocket_url; const protocols = session.direct_websocket_url ? ["breeze-realtime-v1", `breeze-realtime-token.${session.client_secret}`] : undefined; const audioContext = new AudioContext({ sampleRate: 24000 }); let playhead = audioContext.currentTime; const ws = new WebSocket(websocketUrl, protocols); ws.binaryType = "arraybuffer"; ws.onopen = () => { if ( session.direct_websocket_url && ws.protocol !== "breeze-realtime-v1" ) { ws.close(); throw new Error("Direct WebSocket did not negotiate breeze-realtime-v1"); } }; ws.onmessage = (event) => { if (typeof event.data === "string") { const message = JSON.parse(event.data); if (message.type === "session.ready") { ws.send(JSON.stringify({ type: "turn.start", turn_id: "turn_1" })); ws.send(JSON.stringify({ type: "text.append", text: "Hello from the browser." })); ws.send(JSON.stringify({ type: "text.flush" })); ws.send(JSON.stringify({ type: "turn.end" })); } else if (message.type === "turn.done") { ws.send(JSON.stringify({ type: "session.close" })); } return; } // Binary frame: pcm_s16le / 24000 Hz / mono. const pcm = new Int16Array(event.data); const samples = Float32Array.from(pcm, (sample) => sample / 32768); const buffer = audioContext.createBuffer(1, samples.length, 24000); buffer.copyToChannel(samples, 0); const source = audioContext.createBufferSource(); source.buffer = buffer; source.connect(audioContext.destination); playhead = Math.max(playhead, audioContext.currentTime); source.start(playhead); playhead += buffer.duration; }; ``` For production playback, prefer an `AudioWorklet` that pulls from a ring buffer — `createBufferSource` scheduling is fine for short clips but accumulates drift on long conversations. ## CLI Use the CLI as the fastest end-to-end smoke test. It validates authentication, voice access, realtime routing, PCM handling, and per-turn TTFA without requiring WebSocket code: ```bash theme={null} breeze version breeze tts realtime "Hello from Breeze" --voice voc_xeh3w54cqvnp --model breeze-tts-2 --no-play --file out.wav --agent printf 'Hello\nHow are you?\n' | breeze tts realtime --stdin --voice voc_xeh3w54cqvnp --model breeze-tts-2 printf '%s\n' \ '{"type":"turn","text":"First turn."}' \ '{"type":"session.update","instructions":"Speak faster and with more energy."}' \ '{"type":"turn","text":"Second turn."}' | breeze tts realtime --stdin --stdin-format jsonl --voice voc_xeh3w54cqvnp --model breeze-tts-2 --agent cat events.jsonl | breeze tts realtime --events --voice voc_xeh3w54cqvnp --model breeze-tts-2 --output json --no-play ``` The default `--stdin-format text` treats every non-empty line as a complete turn. `--stdin-format jsonl` provides a managed local control format: `{"type":"turn","text":"..."}` submits a turn, and `{"type":"session.update","instructions":"..."}` changes delivery instructions for later turns. Dynamic session instructions use Breeze TTS 2, so this managed JSONL mode selects `breeze-tts-2` when `--model` is omitted while preserving an explicitly selected model. The CLI waits for the current turn to finish, then waits for the exact `session.updated` acknowledgement before it processes the next record. Only confirmed instructions are retained across physical-session rotation and idle reconnect. An update that may have been written but was not acknowledged is never replayed, and later turns are not sent under an uncertain value. Managed CLI conversations use the same lifecycle as the SDK managers: a physical WebSocket becomes eligible for turn-boundary rotation at 600 seconds, each keepalive probe waits up to 5 seconds for an inbound acknowledgement, and two consecutive missed probes mark the connection unhealthy. An active turn can delay the planned switch but is never moved between sockets. An idle failure reconnects with fresh session credentials. An active-turn failure is terminal for that turn and is never replayed. `--events` is a raw, connection-scoped protocol mode. It forwards public client events such as `session.update` verbatim but does not serialize turns, wait for acknowledgements, or carry confirmed instructions to another physical session. The input owner must start another command or session at a safe turn boundary after that socket ends. See [CLI realtime conversation](/cli/reference/tts#realtime-conversation) for JSON output, multi-turn stdin, interruptions, exit codes, and file behavior. ## Continue building Mint short-lived `client_secret` tokens for browser WebSocket connections. Run realtime conversation turns from the terminal with `breeze tts realtime`. Compare one-shot HTTP streaming with the realtime WebSocket for your latency needs. Look up every error code with HTTP status, domain, and retry guidance. Use the HTTP endpoints for one-shot generation, async jobs, and `mp3` or `wav` output. Decode the fixed PCM realtime frames and pick encodings for the HTTP endpoints.