Streaming and chunks

Streaming and chunks#

The ABR SDK’s streaming pipeline is built on three low-level primitives: Buffer, EventSet, and Event. The push() and wait_for_completion() manage these primitives internally, so most callers never interact with them directly. This page explains how they work and how they fit together underneath the high-level API.

The layered model#

The streaming pipeline has two layers, from lowest to highest:

  1. Buffers: FIFO byte queues that carry audio in and text chunks out.

  2. Events and EventSets: a signaling system for blocking efficiently until a buffer reaches a threshold or the neural network becomes idle.

push() and push() manage both layers internally. Most callers never need to interact with them directly.

Buffers#

A Buffer is a fixed-capacity FIFO byte queue inside the neural network pipeline. The Asr class exposes two:

Property or method

Description

buffer.size

Total capacity in bytes (fixed for the lifetime of the buffer).

buffer.level

Current fill level in bytes.

buffer.push(data)

Write bytes into the buffer. Returns the number of bytes actually written, which may be less than len(data) if the buffer is full.

buffer.pull(max_n_bytes)

Read up to max_n_bytes bytes from the buffer.

buffer.clear()

Discard all data currently in the buffer.

In normal usage you do not call push(), pull(), or clear() directly. push() manages both buffers for you.

Events and EventSets#

Polling level in a loop would waste CPU cycles. The event system provides a blocking interface: your code waits until a condition is met, then acts.

An EventSet groups one or more Event objects into a single poll target. Call evset.poll() to block until at least one event fires or the timeout elapses. It returns True if an event fired, False if the timeout expired. A negative timeout waits forever; a timeout of zero returns immediately without blocking. poll() raises InterruptedError if interrupted by a signal.

Call evset.interrupt() to unblock a poll() call that is waiting on another thread.

Create events from an EventSet using these factory methods:

create_buffer_level_event() fires when at least threshold bytes are available. For an output buffer, “available” means threshold bytes of data ready to read. For an input buffer, “available” means threshold bytes of write space free. Use an output buffer event to know when a chunk is ready; use an input buffer event to know when the buffer has space to accept more audio.

create_application_idle_event() fires when all neural networks and all processing stages are idle, meaning no further output will be written to the output buffer until new input arrives. This is the signal to use after flushing, to know that the model has finished. Note that the output buffer may still contain unread data when this event fires; drain the buffer after receiving it.

Event flags and state#

Pass EventFlags when creating an event to control its initial behavior:

Flag

Effect

EventFlags(auto_disable=True)

The event disables itself after firing. Call enable() to arm it again before the next wait.

EventFlags(disabled=True)

The event starts disabled and will not cause poll() to return until you call enable().

Call enable() and disable() at any time to include or exclude an event from the next poll(). is_triggered reflects whether the event’s condition currently holds, independent of whether the event is enabled.

Next steps

  • Transcription stages: how the model produces and revises text, and how to handle chunk overwrites.

  • Overview: TTS overview and the PCM output format.