Realtime character response

Live character products often receive one language-model response over several seconds. You can begin speaking before the whole response is available by dividing each turn into three semantic segments:

Preamble — a short acknowledgement that can play immediately.
First sentence — the first complete sentence from the model response.
Remainder — the rest of the response, split only at natural sentence boundaries.

Keep the queue bounded. If a newer user message makes an old remainder irrelevant, cancel that old work instead of letting it occupy generation capacity.

Concurrency budget

Studio and Developer API generations share the account’s concurrent-generation limit. Starter accounts have six concurrent generations, so a character application should normally use an application-side semaphore of 5. The spare slot leaves room for Studio activity, cancellation overlap, and another product flow. One logical reply split into three simultaneous requests consumes three slots. A second reply can therefore reach the six-slot account limit before the first reply finishes. When that happens, Breeze returns 429 GENERATION_CONCURRENCY_EXCEEDED with Retry-After; it does not queue the request on your behalf.

Retry policy

Retry 429 after the delay in Retry-After, adding a small amount of jitter.
Retry transient 502, 503, and 504 responses with bounded exponential backoff.
Do not retry 400 or 422 without changing the request.
Set a maximum attempt count and discard remainders that have lost their playback value.
Log the Breeze request or trace ID, HTTP status, and stable error code. Do not log API keys or full private dialogue text.

Python example

import asyncio
import os
import random

import httpx

API_KEY = os.environ["BREEZE_API_KEY"]
VOICE_ID = "voc_8rsb3nhb7645"
semaphore = asyncio.Semaphore(5)


async def synthesize_segment(client: httpx.AsyncClient, text: str) -> bytes:
    url = f"https://api.breeze.blue/v1/text-to-speech/{VOICE_ID}/stream"
    for attempt in range(4):
        async with semaphore:
            response = await client.post(
                url,
                headers={"xi-api-key": API_KEY},
                json={"text": text, "model_id": "breeze-tts-2"},
            )

        if response.is_success:
            return response.content
        if response.status_code == 429:
            retry_after = float(response.headers.get("retry-after", "1"))
            await asyncio.sleep(retry_after + random.uniform(0, 0.25))
            continue
        if response.status_code in {502, 503, 504}:
            await asyncio.sleep(min(4, 0.5 * (2**attempt)) + random.uniform(0, 0.25))
            continue
        response.raise_for_status()

    raise RuntimeError("TTS retry budget exhausted")

TypeScript example

import { Semaphore } from "async-mutex";

const semaphore = new Semaphore(5);
const retryable = new Set([502, 503, 504]);

async function synthesizeSegment(text: string): Promise<ArrayBuffer> {
  const url = `https://api.breeze.blue/v1/text-to-speech/${process.env.BREEZE_VOICE_ID}/stream`;

  for (let attempt = 0; attempt < 4; attempt += 1) {
    const [, release] = await semaphore.acquire();
    let response: Response;
    let audio: ArrayBuffer | undefined;
    try {
      response = await fetch(url, {
        method: "POST",
        headers: {
          "content-type": "application/json",
          "xi-api-key": process.env.BREEZE_API_KEY!,
        },
        body: JSON.stringify({ text, model_id: "breeze-tts-2" }),
      });
      if (response.ok) audio = await response.arrayBuffer();
    } finally {
      release();
    }

    if (response.ok) return audio!;

    const jitterMs = Math.random() * 250;
    if (response.status === 429) {
      const retryAfterMs = Number(response.headers.get("retry-after") ?? "1") * 1000;
      await new Promise((resolve) => setTimeout(resolve, retryAfterMs + jitterMs));
      continue;
    }
    if (retryable.has(response.status)) {
      const backoffMs = Math.min(4000, 500 * 2 ** attempt);
      await new Promise((resolve) => setTimeout(resolve, backoffMs + jitterMs));
      continue;
    }
    throw new Error(`TTS request failed with ${response.status}`);
  }

  throw new Error("TTS retry budget exhausted");
}

The TypeScript example uses the small async-mutex package. You can replace it with any bounded semaphore already used by your application.

Language handling

language_code is optional for breeze-tts-2. Short English and Chinese text, including Hi and OK, is resolved without requiring the field. When your character has a fixed language, explicitly send language_code: "en" or language_code: "zh" to make intent unambiguous. Multilingual preview models remain conservative and can ask for an explicit language when a short input cannot be identified reliably.

Choosing HTTP or realtime WebSocket

Use HTTP streaming when you already have a complete preamble or sentence and want independent encoded streams. Use the realtime text-to-speech WebSocket when text arrives incrementally and you want one persistent synthesis connection with turn cancellation and barge-in.

Rate limits

Review account concurrency, Retry-After, and realtime session limits.

Get started

Concepts

Guides

Reference

Realtime character response

Concurrency budget

Retry policy

Python example

TypeScript example

Language handling

Choosing HTTP or realtime WebSocket

Rate limits

​Concurrency budget

​Retry policy

​Python example

​TypeScript example

​Language handling

​Choosing HTTP or realtime WebSocket

Rate limits

Concurrency budget

Retry policy

Python example

TypeScript example

Language handling

Choosing HTTP or realtime WebSocket