Transcription stages#
The ASR model begins processing audio immediately with the first input and emits text incrementally as audio arrives. Text appears as soon as the model recognizes it, and later chunks revise it in place as post-processing refines it.
This page is for developers building applications that display text as it arrives: a terminal renderer or a subtitle overlay.
If you only need the final transcript after all audio has been processed,
AsrTranscriptAsrTranscript handles the bookkeeping automatically. See
Examples for complete runnable implementations.
Model output and post-processing#
The model output is the text the network recognizes. Your mode setting selects which of the model’s two outputs feeds the pipeline:
FAST: uses no look-ahead, so text appears roughly 150 milliseconds after the audio arrives. Lowest latency.
ACCURATE (the default): uses a small window of subsequent audio as context, so text appears roughly one second after the audio arrives. Higher accuracy, higher latency.
Post-processing then revises the model output in place, regardless of mode. It adds punctuation, capitalization, and spell correction.
Punctuation and capitalization can be turned off with enable_punctuation=False, and spell
correction with enable_spellcheck=False, when constructing the AsrAsr instance; left unset,
both are enabled.
Punctuation and capitalization can be turned off with the asr_enable_pcr configuration option,
and spell correction with asr_enable_spellcheck, both ABR_VAR_BOOLEAN options of
abr_lib.app_create; left unset, both are enabled.
Some language packages ship models that emit punctuation directly rather than in post-processing.
Every chunk is an edit to the running transcript. Apply all of them, in order, whatever their type; skipping one corrupts the replace offsets of the chunks that follow.
See Choosing a mode for guidance on which mode to use.
How replace ranges work#
Each AsrChunkAsrChunk carries two byte offset fields that tell you where its text
belongs in the running transcript. All text is encoded as UTF-8, and the byte offsets always index
to UTF-8 code-point boundaries: you will never receive an offset that splits a multi-byte character.
Each chunk’s data also holds whole characters only, so both the chunk and the transcript decode
cleanly after every callback.
replace_byte_offset_beginandreplace_byte_offset_endare negative byte offsets measured from the current end of the transcript buffer.A value of
0means “the current end,” so a chunk with both offsets at0appends new text without replacing anything.A chunk with
replace_byte_offset_begin = -Nandreplace_byte_offset_end = 0replaces the lastNbytes.
Worked example#
Consider the phrase “hello world” arriving in FAST mode. The transcript starts as an empty buffer.
Step 1: model output arrives. Both offsets are 0, the text is appended:
buffer: b"helo wrold" (10 bytes)
Step 2: post-processing revision arrives. replace_byte_offset_begin = -10,
replace_byte_offset_end = 0.
The last 10 bytes are replaced:
buffer: b"Hello, world." (13 bytes)
In ACCURATE mode the steps are the same, but the model output in step 1 is more accurate before post-processing arrives.
Applying chunks to a buffer#
update()abr_asr_transcript_apply() applies a chunk to a bytearray, handling the slice
replacement in place:
buf = bytearray()
def on_chunk(chunk: AsrChunk) -> None:
chunk.update(buf)
This is equivalent to what AsrTranscriptAsrTranscript does internally when you call
.text. Use AsrChunk.update() directly only when you are maintaining a custom buffer for a
renderer that needs per-chunk control.
abr_asr_transcript_apply from asr_transcript.h, in the tarball’s examples/common directory,
applies a chunk to an abr_asr_transcript_t, handling the replacement in place and keeping the
text NUL-terminated:
static char text[ABR_ASR_TRANSCRIPT_MIN_SIZE];
static abr_asr_transcript_t transcript;
static void on_chunk(void *user_data, const uint8_t *data, size_t n_bytes) {
abr_asr_text_chunk_t chunk;
memcpy(&chunk, data, sizeof(chunk));
abr_asr_transcript_apply(&transcript, &chunk, NULL);
}
The transcript drops its oldest text once the buffer is full, keeping at least the last 1024 bytes that a chunk can still replace, so size the buffer for the text you want to keep. Its source is a compact reference if you maintain a buffer of your own, for example a text widget’s.
Rendering#
For a terminal, subtitle, or caption display, apply every chunk and redraw. The text settles on the post-processed version as revisions arrive.
from abr_sdk.asr import AsrChunk
buf = bytearray()
def on_chunk(chunk: AsrChunk) -> None:
chunk.update(buf)
print(buf.decode("utf-8"), end="\r", flush=True)
abr_asr_transcript_apply also reports each change as an edit at the end of the text: how many
bytes were taken back and what follows them. abr_asr_printer_show, from the same header, applies
that edit to a terminal line with backspaces and appends:
static abr_asr_printer_t printer;
static void on_chunk(void *user_data, const uint8_t *data, size_t n_bytes) {
abr_asr_text_chunk_t chunk;
memcpy(&chunk, data, sizeof(chunk));
abr_asr_transcript_edit_t edit;
abr_asr_transcript_apply(&transcript, &chunk, &edit);
abr_asr_printer_show(&printer, &edit);
}
Text already delivered can still be revised, so send a downstream language model the final
transcript. Call wait_for_completion()abr_session_wait_for_completion() first; it flushes the last words,
which would otherwise be cut off. See Examples for a runnable example.
Choosing a mode#
The mode selects FAST or ACCURATE model output. Post-processing is applied in both modes.
Pass mode to the AsrAsr constructor:
from abr_sdk.asr import Asr, AsrMode
with Asr(LIBRARY_PATH, mode=AsrMode.ACCURATE) as asr:
...
If mode is not specified, the library defaults to ACCURATE.
Pass the asr_mode configuration option, "fast" or "accurate", to
abr_lib.app_create:
abr_config_t cfg;
cfg.key = "asr_mode";
cfg.value.type = ABR_VAR_STR;
cfg.value.value.str = "accurate";
abr_session_t session;
abr_session_open(&session, LIBRARY_PATH, &cfg, 1);
If asr_mode is not given, the library defaults to "accurate".
Situation |
Recommended mode |
|---|---|
Transcript consumed by a language model |
|
Transcript displayed to a user (subtitles, live captions) |
|
Latency is the primary constraint |
|
Accuracy is the primary constraint |
|
Measuring WER/CER (accuracy benchmarking) |
|
Both modes use the same underlying model. The difference is in when the model commits to its output.