Overview#

The ABR SDK transcribes speech to text on-device, with no network round-trip. You push audio in as it arrives and receive text chunks back through a callback in real time.

The ASR API is provided by the asrabr_lib module. The main class you interact with is AsrAsr.

The ASR API is the abr_lib function table, loaded from an ASR package with abr_lib_open(). An abr_app_t created from it is the application you stream audio through; the session helper from the tarball’s examples/common directory wraps the streaming in abr_session_push() and abr_session_wait_for_completion().

How a transcript is built#

ASR produces text incrementally as audio arrives. Each piece of output is an AsrChunkAsrChunk; the AsrTranscriptAsrTranscript helper collects chunks and applies all overwrites automatically if you only need the final text. The full mechanics (chunk types, model output modes, and replace ranges) are covered in Transcription stages.

Streaming transcription#

Streaming transcription is the primary way to use ASR in the SDK. You push audio in chunks as it arrives, and a callback fires each time the model produces new text. This works for live microphone input, network audio streams, and long files you want to start transcribing before they finish loading, as well as for short clips and complete recordings.

Attention

The function below shows the pattern. audio_source can be any iterable that yields raw PCM bytes (a microphone reader, file reader, or network stream). For copy-paste-ready code, see Examples.

streaming.py#
from collections.abc import Iterable

from abr_sdk.asr import Asr, AsrTranscript

LIBRARY_PATH = "/path/to/niagara-38m-live.en-linux-x86_64/libniagara_38m_live.so"


def transcribe(audio_source: Iterable[bytes]) -> str:
    """Transcribe raw PCM bytes from any iterable."""
    transcript = AsrTranscript()
    with Asr(LIBRARY_PATH) as asr:
        for audio_chunk in audio_source:
            asr.push(audio_chunk, on_chunk=transcript.chunks.append)
        asr.wait_for_completion()
    return transcript.text

The push()abr_session_push() call returns as soon as all input bytes have been accepted; it can block briefly when the model’s input buffer is full, and the on_chunk callback may fire during the call as chunks are produced. The callback passed to the first push() of a session is used for the whole session; the on_chunk argument on later calls is ignored (passing the same callback each time, as the examples do, is harmless). wait_for_completion()abr_session_wait_for_completion() blocks until the model has finished processing all audio that has been pushed, and may also invoke on_chunk.

streaming.c#
#include "common/asr_transcript.h"
#include "common/session.h"

#define LIBRARY_PATH \
    "/path/to/niagara-38m-live.en-linux-x86_64/libniagara_38m_live.so"
#define CHUNK_BYTES (16000 * 2 / 10)  // 100 ms of 16 kHz mono S16_LE audio

/**
 * Any source of raw PCM: fills `data` and returns the number of bytes, or 0
 * when the audio has ended.
 */
typedef size_t (*audio_source_t)(void *user_data, uint8_t *data,
                                 size_t max_bytes);

/**
 * Transcribes raw PCM bytes from any source into `transcript`.
 */
// NOLINTNEXTLINE(misc-use-internal-linkage)
abr_status_t transcribe(audio_source_t source, void *source_data,
                        abr_asr_transcript_t *transcript) {
    abr_session_t session;
    if (abr_session_open(&session, LIBRARY_PATH, NULL, 0) != ABR_OK) {
        return session.status;
    }
    uint8_t chunk[CHUNK_BYTES];
    size_t n_bytes;
    while ((n_bytes = source(source_data, chunk, sizeof(chunk))) > 0) {
        abr_session_push(&session, chunk, n_bytes, abr_asr_transcript_on_output,
                         transcript);
    }
    abr_session_wait_for_completion(&session, abr_asr_transcript_on_output,
                                    transcript);
    abr_session_close(&session);
    return session.status;
}

abr_session_push returns as soon as all input bytes have been accepted; it blocks while the model’s input buffer is full, and on_chunk fires during the call as chunks are produced. abr_session_wait_for_completion pushes a flush marker and blocks until the model has finished processing all audio that has been pushed, invoking on_chunk for the last chunks. Both drive abr_lib.buf_push, abr_lib.buf_pull and an event set; examples/asr.c shows that loop in full.

For benchmarking against a fixed dataset, see Benchmarking.

Transcription modes#

The SDK offers two transcription modes:

  • AsrMode.FASTAsrMode.FAST emits text as soon as possible, around 150 milliseconds after the audio arrives. It does not wait for additional context.

  • AsrMode.ACCURATEAsrMode.ACCURATE uses a small amount of subsequent audio as context before emitting, roughly one second after the audio arrives, which improves accuracy at the cost of latency.

If you do not specify a mode, the library defaults to ACCURATE.

Both modes run the same network at the same compute cost, so FAST is not faster for batch processing. The mode only changes when the model commits to output.

You set the mode when constructing the AsrAsr instance:

from abr_sdk.asr import Asr, AsrMode

with Asr(LIBRARY_PATH, mode=AsrMode.FAST) as asr:
    ...

The mode is the asr_mode configuration option of abr_lib.app_create, "fast" or "accurate":

abr_config_t cfg;
cfg.key = "asr_mode";
cfg.value.type = ABR_VAR_STR;
cfg.value.value.str = "fast";

abr_session_t session;
abr_session_open(&session, LIBRARY_PATH, &cfg, 1);

The right choice depends on what consumes the transcript. AsrMode.FASTAsrMode.FAST suits a live display where text must appear as soon as possible. AsrMode.ACCURATEAsrMode.ACCURATE suits everything else, including subtitles and transcripts sent to a downstream language model.

See Transcription stages for guidance on picking between them.

The overwrite model#

Text appears early and is revised in place. The model output, FAST or ACCURATE depending on your mode setting, appears first. Later chunks overwrite it with punctuated, spell-corrected text. This is how the SDK delivers low-latency output while still refining it before it is considered final.

How you handle the overwrites depends on where the text is rendered. A terminal can rewrite a line in place; a web UI tracks byte ranges and updates DOM spans; a downstream language model should receive the final transcript after wait_for_completion()abr_session_wait_for_completion().

The full mechanics (model output modes, post-processing, and the replace-range fields) are covered in Transcription stages.

What you need to use ASR#

To run ASR you need three things:

  • The SDK Python package (pip install abr-sdk).

  • The SDK C binding, compiled into your program (see Installation).

  • An application package for your target platform. This archive contains the compiled model, the network weights, and supporting files. You download it from the ABR developer portal and extract it on the device. See Application packages.

  • Audio in the expected format. ASR consumes raw PCM bytes at a specific sample rate. The required format and sample rates are listed in Input format.

The AsrAsr constructor takes the path to the package’s shared library.

abr_lib_open() takes the path to the package’s shared library, and abr_lib.app_create creates the ASR application from it.

Next steps

  • New to the SDK? Start with ASR quickstart.

  • Transcription stages: how the model produces and revises output, and how to handle chunk overwrites.

  • Examples: full, runnable examples for microphone, file, LLM feed, and more.