ABR SDK Introduction#

The ABR SDK is an on-device AI engine that enables streaming automatic speech recognition (ASR) and text-to-speech (TTS). It is:

  • Fully streaming, delivering the lowest latency while achieving maximum accuracy

  • Computationally efficient, with a small memory footprint

  • Supported on multiple OS and hardware platforms: Linux x86-64 Linux arm64 Android arm64

  • Available in multiple languages, including English, Spanish, German, Chinese (Mandarin), Japanese, Korean, and more; coverage varies by model and model size, see the Niagara ASR and Nith TTS model matrices

Getting set up#

You need an ABR Developer account and an SDK access token to access and operate the SDK on your device. You can start your journey as an ABR developer on the ABR developer portal.

The SDK is available as a Python package, for Python 3.10 or later, and as a C binding you compile into your program. See Installation for full setup instructions, and pick your language in the navigation bar to see the documentation for it.

SDK models and features#

The SDK gives you the following on-device speech AI capabilities:

Niagara ASR

Transcribe streaming voice audio to text in real time.

  • Punctuation and capitalization

  • Custom vocabulary (coming soon)

Niagara Automatic Speech Recognition
Nith TTS

Synthesize natural-sounding speech from a text stream.

  • Custom vocabulary pronunciation dictionary

  • Normalizes text to correctly pronounce dates, temperatures, currencies, and more

Nith Text-to-Speech

Both run entirely on your device. No audio or transcript data is sent to an external service during processing.

SDK model technology#

ABR’s speech AI models are built on state-space models (SSMs), a technology we invented with the Legendre Memory Unit (LMU), which we introduced in 2019. Unlike transformer models, SSMs are derived from continuous-time representations and require significantly less computation. Our selective use of attention layers lets our SSMs achieve much higher performance than transformers of similar size, so ABR’s models deliver competitive performance while remaining much smaller. These models also deliver true streaming performance, with no look-ahead or batch accumulation before producing output.

These properties make SSM-based models practical and ideal on devices with limited compute and memory. You do not need to understand SSMs to use the SDK, but if you want to learn more you can read about it here. The distinction matters if you are evaluating whether ABR’s models fit your device’s hardware budget.

SDK architecture#

The SDK is a thin API layer over a stable C application binary interface (ABI). Neural network execution and platform-specific optimizations happen inside a compiled library you download separately, called an application package.

Once you extract the package to a directory on your device, the SDK loads the library and weights directly from that directory path when you create an Asr or Tts object. There is no need to select a hardware backend or configure execution targets; the application package already contains the correct code for the platform you selected.

Who this documentation is for#

This documentation assumes you are a developer integrating speech AI into a device or application, and that you have the requisite Python or C skills to integrate this SDK.