> ## Documentation Index
> Fetch the complete documentation index at: https://docs.breezeblue.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Speech timing

> Generate complete or streaming audio with word timing for synchronized text highlighting and seeking.

Generate speech with word/token timestamps using a BreezeBlue TTS 2 model. Choose a complete response, a background job, or a stream. Times are in seconds relative to the delivered audio; character alignment is not provided.

| Delivery   | Endpoint                                                            | Response                                                                                     |
| ---------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| Complete   | `POST /v1/text-to-speech/{voice_id}/with-timestamps`                | One JSON object with `audio_base64`, `content_type`, and the complete `word_timestamps` list |
| Background | `POST /v1/text-to-speech/{voice_id}/with-timestamps?delivery=async` | A job ID; poll the job for `word_timestamps` and an audio download URL                       |
| Streaming  | `POST /v1/text-to-speech/{voice_id}/stream/with-timestamps`         | NDJSON audio and timing updates, followed by the complete word list                          |

## Complete audio with timing

The synchronous endpoint accepts the ordinary TTS request fields and output formats, including sample-rate profiles such as `wav_48000`. Default output is MP3. Word timestamps are always included. `timestamp_mode` applies only to streaming. The `history-item-id` response header identifies the saved audio; history audio downloads contain audio bytes, not the JSON envelope.

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import base64
    from pathlib import Path
    from breeze_blue import BreezeBlue

    client = BreezeBlue()
    result = client.text_to_speech.convert_with_timestamps(
        "VOICE_ID", text="Hello world.", model_id="breeze-tts-2-multilingual",
        language_code="en", output_format="wav",
    )
    Path("speech.wav").write_bytes(base64.b64decode(result["audio_base64"]))
    print(result.get("word_timestamps"))
    ```
  </Tab>

  <Tab title="TypeScript">
    ```typescript theme={null}
    import { writeFile } from "node:fs/promises";
    import { BreezeBlueClient } from "@breeze.blue/sdk";

    const client = new BreezeBlueClient();
    const result = await client.textToSpeech.convertWithTimestamps("VOICE_ID", {
      text: "Hello world.", modelId: "breeze-tts-2-multilingual", languageCode: "en",
    }, { outputFormat: "wav" });
    await writeFile("speech.wav", Buffer.from(result.audioBase64, "base64"));
    console.log(result.wordTimestamps);
    ```
  </Tab>

  <Tab title="CLI">
    ```bash theme={null}
    breeze tts 'Hello world.' --voice VOICE_ID \
      --model breeze-tts-2-multilingual --language en \
      --no-stream --with-timestamps --format wav --file speech.wav --agent
    ```

    The CLI saves decoded audio to `speech.wav` and the complete list to `speech.wav.timestamps.json`. JSON output has `streamed: false`, `word_timestamps`, and `timestamps_file`.
  </Tab>

  <Tab title="HTTP">
    ```bash theme={null}
    curl 'https://api.breeze.blue/v1/text-to-speech/VOICE_ID/with-timestamps?output_format=wav' \
      -H "xi-api-key: $BREEZE_API_KEY" -H 'Content-Type: application/json' \
      -d '{"text":"Hello world.","model_id":"breeze-tts-2-multilingual","language_code":"en"}'
    ```
  </Tab>
</Tabs>

## Background audio with timing

Set `delivery=async` to return HTTP 202 with a `generation_job_id` immediately. Poll `GET /v1/generation-jobs/{generation_job_id}` until `status` is `ready`, then read `word_timestamps` and download the audio from `download_url`. Stop polling on `failed` or `cancelled`; use a deadline. Pending jobs and ordinary audio jobs omit `word_timestamps`.

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import time
    from breeze_blue import BreezeBlue

    client = BreezeBlue()
    job = client.text_to_speech.create_job_with_timestamps(
        "VOICE_ID", text="Hello world.", model_id="breeze-tts-2-multilingual",
        language_code="en", output_format="wav",
    )
    deadline = time.monotonic() + 120
    while time.monotonic() < deadline:
        result = client.generation_jobs.get(job["generation_job_id"])
        if result["status"] == "ready":
            print(result.get("word_timestamps"))
            break
        if result["status"] in ("failed", "cancelled"):
            raise RuntimeError(result.get("error"))
        time.sleep(2)
    else:
        raise TimeoutError("Speech job is still pending")
    # Download audio with client.generation_jobs.download_audio(job["generation_job_id"]).
    ```
  </Tab>

  <Tab title="TypeScript">
    ```typescript theme={null}
    import { BreezeBlueClient } from "@breeze.blue/sdk";

    const client = new BreezeBlueClient();
    const job = await client.textToSpeech.createJobWithTimestamps("VOICE_ID", {
      text: "Hello world.", modelId: "breeze-tts-2-multilingual", languageCode: "en",
    }, { outputFormat: "wav" });
    const deadline = Date.now() + 120_000;
    while (true) {
      if (Date.now() >= deadline) throw new Error("Speech job is still pending");
      const result = await client.generationJobs.get(job.generationJobId);
      if (result.status === "ready") {
        console.log(result.wordTimestamps);
        break;
      }
      if (["failed", "cancelled"].includes(result.status)) {
        throw new Error(JSON.stringify(result.error));
      }
      await new Promise(resolve => setTimeout(resolve, 2000));
    }
    // Download audio with client.generationJobs.downloadAudio(job.generationJobId).
    ```
  </Tab>

  <Tab title="CLI">
    ```bash theme={null}
    breeze tts 'Hello world.' --voice VOICE_ID --model breeze-tts-2-multilingual --language en --async --with-timestamps --format wav --agent
    breeze jobs wait JOB_ID --file speech.wav --agent
    ```

    Replace `JOB_ID` with the returned `generation_job_id`. Waiting with `--file` saves audio and `speech.wav.timestamps.json`. To download a completed job later, use `breeze jobs download JOB_ID --file speech.wav --with-timestamps --agent`. JSON output includes `word_timestamps` and `timestamps_file`.
  </Tab>

  <Tab title="HTTP">
    ```bash theme={null}
    curl 'https://api.breeze.blue/v1/text-to-speech/VOICE_ID/with-timestamps?delivery=async&output_format=wav' -H "xi-api-key: $BREEZE_API_KEY" -H 'Content-Type: application/json' -d '{"text":"Hello world.","model_id":"breeze-tts-2-multilingual","language_code":"en"}'
    curl 'https://api.breeze.blue/v1/generation-jobs/JOB_ID' -H "xi-api-key: $BREEZE_API_KEY"
    ```
  </Tab>
</Tabs>

## Streaming audio with timing

Use `POST /v1/text-to-speech/{voice_id}/stream/with-timestamps` with a BreezeBlue TTS 2 model. It accepts the usual text, voice, model, language, and voice settings. Audio is returned as base64 inside newline-delimited JSON (NDJSON), with word/token timestamps. The ordinary `/stream` endpoint continues to return audio bytes.

```bash theme={null}
curl 'https://api.breeze.blue/v1/text-to-speech/VOICE_ID/stream/with-timestamps?output_format=pcm' \
  -H "xi-api-key: $BREEZE_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"text":"Hello world.","model_id":"breeze-tts-2-multilingual","language_code":"en","timestamp_mode":"chunk"}'
```

`timestamp_mode` is `chunk` (default) or `lookahead`. Chunk mode sends boundaries as audio is generated; the same word may span several updates. Lookahead mode waits for later audio to confirm a boundary, so timing can arrive after its audio. It does not promise a fixed wall-clock delay.

## Consume the stream

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import base64
    from breeze_blue import BreezeBlue

    client = BreezeBlue()
    words = {}
    with client.text_to_speech.stream_with_timestamps(
        "VOICE_ID", text="Hello world.", output_format="wav",
        model_id="breeze-tts-2-multilingual", language_code="en",
    ) as stream, open("speech.wav", "wb") as audio:
        for chunk in stream:
            audio.write(base64.b64decode(chunk["audio_base64"]))
            for word in chunk["word_timestamps"]:
                previous = words.get(word["index"], word)
                words[word["index"]] = {
                    **word,
                    "start": min(previous["start"], word["start"]),
                    "end": max(previous["end"], word["end"]),
                }
    ```
  </Tab>

  <Tab title="TypeScript">
    ```typescript theme={null}
    import { BreezeBlueClient, type WordTimestamp } from "@breeze.blue/sdk";

    const client = new BreezeBlueClient();
    const words = new Map<number, WordTimestamp>();
    const stream = await client.textToSpeech.streamWithTimestamps("VOICE_ID", {
      text: "Hello world.", modelId: "breeze-tts-2-multilingual",
      languageCode: "en", timestampMode: "lookahead",
    });
    for await (const chunk of stream) {
      // Decode chunk.audioBase64 and enqueue the bytes for your audio player.
      for (const word of chunk.wordTimestamps) {
        const previous = words.get(word.index) ?? word;
        words.set(word.index, { ...word,
          start: Math.min(previous.start, word.start),
          end: Math.max(previous.end, word.end),
        });
      }
    }
    ```
  </Tab>

  <Tab title="CLI">
    ```bash theme={null}
    breeze tts 'Hello world.' --voice VOICE_ID \
      --model breeze-tts-2-multilingual --language en \
      --with-timestamps --timestamp-mode lookahead \
      --format wav --file speech.wav --no-play --agent
    ```

    The CLI writes decoded audio to `speech.wav` and merged timing to `speech.wav.timestamps.json`. JSON output includes `word_timestamps` and `timestamps_file`. Use `--no-stream --with-timestamps` for complete JSON delivery. `--timestamp-mode` applies only to streaming; use `--async --with-timestamps` for a background job.
  </Tab>
</Tabs>

SDK iterators raise on stream errors and close the connection when iteration stops. In Python, use the context manager to close even when leaving early. Each call consumes the same speech-generation credits as ordinary TTS.

## Timing and display

A word entry has this shape:

```json theme={null}
{"index":0,"word":"Hello","start":0.2,"end":0.6}
```

* Times refer to the complete output audio, including requested speed and pauses inserted between long-text segments. They are not offsets inside an HTTP chunk.
* Streaming audio output formats are `pcm` (mono 24 kHz PCM16 little-endian), `wav`, and `mp3`; complete responses support the ordinary synchronous TTS formats. Decode `audio_base64` before playback. MP3 encoders can buffer, so a record's bytes need not align with its word boundaries.
* Preserve the authored text, punctuation, whitespace, and audio tags. Chinese and other CJK languages may return tokens containing multiple characters. This API does not provide character timestamps.
* Highlight a word using the player's audio clock when `start <= currentTime < end`. Seek to its `start` when the reader selects it. Do not use network arrival time as playback time.
* A streaming record can have an empty `audio_base64` or an empty `word_timestamps` array. HTTP transport chunks can split a JSON line or a UTF-8 character; use an incremental parser.
* The final successful streaming record contains the complete merged word list and empty audio. Merge updates by `index`, keeping the earliest `start` and latest `end`, or use this final list for saved playback.
* In a stream, after HTTP 200, a failure can appear as `{"error":{"code":"UPSTREAM_GENERATION_ERROR","message":"..."}}`. A disconnect, parsing failure, or error record means the output is incomplete.

The API follows the normal authentication, voice visibility, generation concurrency, history, and billing rules. Old history audio is not retroactively aligned.

## Related endpoints and tools

* [Convert with timestamps](https://docs.breezeblue.ai/api-reference/text-to-speech/convert-with-timestamps)
* [Convert text to speech](https://docs.breezeblue.ai/api-reference/text-to-speech/convert-text-to-speech)
* [Stream speech with word timestamps](https://docs.breezeblue.ai/api-reference/text-to-speech/stream-speech-with-word-timestamps)
* [Output formats](https://docs.breezeblue.ai/concepts/output-format)
* [CLI text to speech](https://docs.breezeblue.ai/cli/reference/tts)