Library loading and the ABI functions#

Types and utilities for dynamically loading ABR SDK libraries.

Author

Andreas Stöckel (Applied Brain Research)

Typedefs

typedef struct abr_lib abr_lib_t#

Collection of all functions defined in the ABR SDK ABI.

This structure also includes the system handle for the opened shared library.

Functions

abr_status_t abr_lib_open(abr_lib_t *lib, const char *path, abr_log_cb_t log_cb, void *log_cb_user_data)#

Opens an ABR SDK shared library and places pointers at the SDK functions in the provided abr_lib_t structure.

Libraries opened by abr_lib_open() must be closed by calling abr_lib_close() to avoid resource leaks.

Parameters:
  • lib – Pointer at a abr_lib_t structure that should be initialized. Note that abr_lib_open() resets the lib structure independent of the return value of this function.

  • path – File path to the ABR SDK shared library or NULL. In case NULL is provided, then a library linked at compile time is used instead. Note that, depending on the operating system, providing a non-absolute path may result in the OS searching for the shared library in standard locations. Pass an absolute path to avoid this behaviour.

  • log_cb – Log callback that should be used to log events that happen during loading.

  • log_cb_user_data – Pointer passed as a first parameter to log_cb.

Returns:

Returns one of the following status codes. Supply a log callback to gather more information.

abr_status_t abr_lib_close(abr_lib_t *lib)#

Closes a previously opened ABR SDK shared library.

Calling this function multiple times on the same lib object has no effect.

Parameters:
  • lib – Pointer at the abr_lib_t structure that should be closed. Passing NULL or an already closed library has no effect.

Returns:

One of the following status codes.

struct abr_lib#

Collection of all functions defined in the ABR SDK ABI.

This structure also includes the system handle for the opened shared library.

Public Members

uintptr_t handle#

Platform-specific handle pointing at the underlying shared library.

A value of zero indicates that this abr_lib_t instance does not own a handle at a shared library. This might happen after calling abr_lib_close() or when using compile-time linking.

uint32_t (*abi_version)()#

Return an identifier describing the ABI version.

The returned integer is structured as follows:

  • Bits 16-31: Major version (incompatible ABIs)

  • Bits 0-15: Minor version (new features, back and forward-compatible)

Return:

The ABI version as a 32-bit unsigned integer.

abr_status_t (*app_create)(abr_app_t **app_p, const abr_config_t *cfg, uint32_t n_cfg, abr_log_cb_t log_cb, void *log_cb_user_data)#

Create a new time-series neural network application.

Users may instantiate an arbitrary number of application instances, as long as the underlying accelerator supports this, and there are enough resources (i.e., memory) left.

Every application instance is optionally linked to a logger. The logger is used to capture log messages from the application.

Application-agnostic keys:

  • “min_log_level” (ABR_VAR_SINT, see abr_log_level): minimum severity threshold passed to the logger. Defaults to ABR_LOG_DEFAULT.

  • ”resources_dir” (ABR_VAR_STR): filesystem directory the loaded library reads model weights from. Required unless “resources_addr” is given.

  • ”resources_addr” (ABR_VAR_PTR): address of the application data container in memory, for a platform without a filesystem. Given in place of “resources_dir”. The address must be aligned to 16384 bytes, and the memory must remain valid and unchanged until the application is freed.

  • ”verify_resources” (ABR_VAR_BOOLEAN): whether to check the hash of every blob in the application data container, such as the model weights, when the application is created. This reads all of the container. Defaults to false.

  • ”license_dir” (ABR_VAR_STR): filesystem directory in which license data is stored. Required where there is a filesystem, and not accepted on a platform without one.

  • ”license_envelope” (ABR_VAR_STR): the license itself, as written by abr-sdk activate --fingerprint. Only accepted on a platform without a filesystem, in place of “license_dir”.

  • ”alif_se_handle” (ABR_VAR_UINT): handle of the Secure Enclave services of an Alif Ensemble, as SERVICES_register_channel() returned it to the application. Required under FreeRTOS on an Alif Ensemble, and not accepted elsewhere. The library asks the Secure Enclave for random bytes and for the identity of the device, which abr_get_fingerprint() and abr_activate_license() need too.

  • ”litert_delegate” (ABR_VAR_STR): LiteRT kernel path used to run the neural networks. One of:

    • ”builtin”: the TFLite built-in kernels, with no delegate.

    • ”xnnpack”: the XNNPACK delegate.

    • a filesystem path to an external delegate library. Defaults to the delegate the library was built for. Libraries built for TFLite Micro log that the key is ignored.

  • ”cache_dir” (ABR_VAR_STR): existing, writable directory in which the library stores data that it can recompute, such as neural networks compiled for the target hardware. A directory that does not exist or cannot be written to is rejected. If omitted, the library uses the abr-sdk subdirectory of the first of the directories listed below, and may create that subdirectory including any missing parent directories. If none of them applies, the library logs a warning and computes that data again on every run.

    • the directory Android assigns to the calling application, Context.getCacheDir(). Android only, and only in an app process.

    • $XDG_CACHE_HOME, if that environment variable holds an absolute path. Linux only.

    • $HOME/.cache, if the HOME environment variable holds an absolute path the calling process can write to. Linux only.

    • $TMPDIR, if that environment variable is set and not empty.

    • /tmp. Linux only.

ASR-specific keys:

  • “asr_mode” (ABR_VAR_STR): which decoder branch is surfaced. One of:

    • ”fast”: causal decoder (lower-quality, sooner output).

    • ”accurate”: noncausal decoder (higher-quality, lookahead latency).

    Defaults to “accurate”.

  • ”asr_enable_spellcheck” (ABR_VAR_BOOLEAN): whether the spell checker corrects the transcript. Ignored if the library has no spell checker. Defaults to true.

  • ”asr_enable_pcr” (ABR_VAR_BOOLEAN): whether the punctuation and capitalization (PCR) network post-processes the transcript. Ignored if the library has no PCR network. Defaults to true.

  • ”asr_vocabulary” (ABR_VAR_STR): path of the custom vocabulary artifact whose terms the vocabularizer substitutes into the transcript. The artifact is an .abrdb plugin of this application, built for its network (otherwise ABR_ERR_INCOMPATIBLE_APP). A relative path is resolved within “resources_dir”. Ignored if the library has no vocabularizer. Defaults to none.

Depending on the error code, additional information may have been logged.

Param app_p:

Pointer to the variable where the newly created application handle will be stored. This will be set to NULL if initialization fails.

Param cfg:

Pointer to an array of configuration options. May be NULL if no additional configuration is needed. Recognized keys depend on the loaded application; the library picks the default for any key the caller omits. Application-specific keys are namespaced by the application name (e.g. ASR exposes “asr_*”).

Param n_cfg:

Number of configuration options in the cfg array.

Param log_cb:

Pointer to the log callback function. Passing NULL disables logging.

Param log_cb_user_data:

Arbitrary pointer passed as the first argument to log_cb. Useful for object-oriented patterns where log_cb is a member function and log_cb_user_data is a class instance.

Return:

One of the following status codes:

  • ABR_OK: A valid pointer has been written to abr_p, and the application is ready to use.

  • ABR_ERR_LICENSE: No application has been created because this device does not hold a valid license. Call abr_activate_license() to activate the device. See the log for more details.

  • ABR_ERR_PLATFORM_UNSUPPORTED: This application library is not supported on this machine. You may be attempting to open an application library that uses a hardware accelerator that is not available on your machine.

  • ABR_ERR_BUSY: The targeted accelerator is already busy executing another application.

  • ABR_ERR_SYSTEM: A system call returned with an error.

abr_status_t (*get_fingerprint)(abr_fingerprint_t *fingerprint, const abr_config_t *cfg, uint32_t n_cfg, abr_log_cb_t log_cb, void *log_cb_user_data)#

Returns this device’s fingerprint, as reported by the library.

Does not require an application instance. A device without a hardware identity is identified by a random identity that is created in license_dir on first use and reused afterwards; the directory is created if it does not exist.

Param fingerprint:

Pointer to the structure where the fingerprint will be stored.

Param cfg:

Configuration options that would normally be passed to abr_app_create(). The relevant keys are license_dir, alif_se_handle and min_log_level; all other keys are ignored.

Param n_cfg:

Number of configuration options in the cfg array.

Param log_cb:

Pointer to the log callback function. Passing NULL disables logging.

Param log_cb_user_data:

Arbitrary pointer passed as the first argument to log_cb. Useful for object-oriented patterns where log_cb is a member function and log_cb_user_data is a class instance.

Return:

One of the following status codes:

abr_status_t (*activate_license)(const char *envelope, const abr_config_t *cfg, uint32_t n_cfg, abr_log_cb_t log_cb, void *log_cb_user_data)#

Activate a license for this device given the server response.

Does not require an application instance; activation only depends on the license directory and the device this function is called on.

Param envelope:

Pointer at a zero-terminated string containing the signed license envelope.

Param cfg:

Configuration options that would normally be passed to abr_app_create(). The relevant keys are license_dir, alif_se_handle and min_log_level; all other keys are ignored and logged at the debug level.

Param n_cfg:

Number of configuration options in the cfg array.

Param log_cb:

Pointer to the log callback function. Passing NULL disables logging.

Param log_cb_user_data:

Arbitrary pointer passed as the first argument to log_cb. Useful for object-oriented patterns where log_cb is a member function and log_cb_user_data is a class instance.

Return:

One of the following status codes:

void (*app_free)(abr_app_t *app)#

Free the application instance and all resources held by it.

Note that this invalidates all other handles that (directly or indirectly) have this application instance as a parent.

This function does nothing if app is NULL; i.e., it is safe to call abr_app_free() independent of whether the library has been initialized correctly or not.

Param app:

Pointer to the application instance to free, or NULL.

uint32_t (*app_get_buffer_count)(const abr_app_t *app)#

Return the number of I/O buffers available in the application.

Param app:

Pointer to the application instance for which the buffer count should be obtained.

Return:

The number of buffers, or zero if app is NULL.

abr_buf_t *(*app_get_buffer)(abr_app_t *app, uint32_t idx)#

Fetch the i-th buffer from the application.

Calling this function multiple times with the same index will always return the same buffer object. There is no need to free these objects after use.

Input buffers come first, in graph order, followed by the output buffers. ASR, TTS and PCR libraries expose exactly one of each.

Param app:

Pointer to the application instance.

Param idx:

Zero-based index of the buffer object to retrieve.

Return:

A pointer to the buffer, or NULL if app is NULL or idx is out of range.

void *(*get_parent)(void *obj)#

Return the parent object of the specified object.

Param obj:

Pointer to any ABR object; this may be a abr_app_t, a abr_buf_t, etc. May be NULL.

Return:

The parent object of the given object, or NULL if the object has no known parent object.

uint32_t (*get_property_count)(void *obj)#

Return the number of properties attached to the given object.

Every ABR object may have a series of runtime-defined properties attached. These properties are used for values that do not need to be accessed very often. Using a generic interface avoids having to add new functions to the C ABI every time a new property is exported.

Passing NULL provides information about the application library itself.

Param obj:

Pointer to a ABR object; this may be a abr_app_t, a abr_buf_t, etc. Passing NULL provides access to the global “meta” scope.

Return:

The number of properties.

const char *(*get_property_name)(void *obj, uint32_t idx)#

Return the property name by index.

Param obj:

Pointer to a ABR object, or NULL for the global “meta” scope.

Param idx:

Zero-based index of the property.

Return:

The property name, or NULL if the index is out of range.

abr_variant_t (*get_property_value_by_idx)(void *obj, uint32_t idx)#

Return a property value by index.

Param obj:

Pointer to a ABR object, or NULL for the global “meta” scope.

Param idx:

Zero-based index of the property.

Return:

The property value, or an “undefined” variant if the index is out of range or the object is not known.

abr_variant_t (*get_property_value_by_name)(void *obj, const char *key)#

Return a property value by name.

Param obj:

Pointer to a ABR object, or NULL for the global “meta” scope.

Param key:

The property name to look up. Keys are case-sensitive.

Return:

The property value, or an “undefined” variant if the name is not found or the object is not known.

uint32_t (*buf_get_size)(const abr_buf_t *buf)#

Return the total buffer capacity in bytes.

Param buf:

Pointer to the buffer.

Return:

The buffer capacity in bytes, or zero if buf is NULL.

uint32_t (*buf_get_level)(const abr_buf_t *buf)#

Return the number of bytes that can be pushed or pulled.

Param buf:

Pointer to the buffer.

Return:

The number of bytes that can be pushed, or pulled up to the next flush marker; zero if buf is NULL.

int32_t (*buf_push)(abr_buf_t *buf, const uint8_t *data, uint32_t n_bytes, bool flush)#

Push (write) data onto a buffer.

Does nothing if buf is an output buffer.

Param buf:

Pointer to the buffer.

Param data:

Pointer to the data to push.

Param n_bytes:

Number of bytes to push.

Param flush:

If true, pushes a “flush marker” into the buffer. This indicates that this is the end of a data segment, and that the application should process all data made available so far. If not all n_bytes fit, the marker is dropped; push the rest with flush set again. A push of zero bytes always places the marker.

Return:

The number of bytes actually pushed onto the buffer, or a negative error code (compatible with abr_status_t) on failure. Once the application has failed, every call returns the status of that failure; free the application to recover.

int32_t (*buf_pull)(abr_buf_t *buf, uint8_t *data, uint32_t n_bytes, bool *flush)#

Pull (read) data from a buffer.

Does nothing if buf is an input buffer.

Param buf:

Pointer to the buffer.

Param data:

Pointer to the buffer where pulled data will be stored.

Param n_bytes:

Maximum number of bytes to pull.

Param flush:

Pointer to a boolean flag, or NULL. Set to true if the pull stopped at a flush marker and consumed it, which typically happens once for every flush flag pushed into the pipeline. A pull with n_bytes of zero never reports or consumes a marker; a pull that returns zero bytes may still report one, at the end of an empty segment.

Return:

The number of bytes actually pulled from the buffer, or a negative error code (compatible with abr_status_t) on failure. Once the application has failed, every call returns the status of that failure; free the application to recover.

abr_evset_t *(*evset_create)(abr_app_t *app)#

Create a new event set connected to the given application instance.

The event set (short “evset”) may be used by a single user thread to wait for events on the application’s buffers.

Param app:

Pointer to the application instance.

Return:

Pointer to the newly created event set, or NULL on failure.

int64_t (*evset_get_platform_handle)(abr_evset_t *evset)#

Return the platform-specific event handle.

The event handle may be used to yield to the operating system until a buffer event (configured through the buffer interface) happens.

On POSIX-like systems, the returned handle is a file descriptor of a socketpair() or a compatible mechanism such as eventfd() on Linux. An event results in eight bytes becoming readable from the file descriptor. The user must read those eight bytes to confirm the event.

The file descriptor is meant to be used in conjunction with poll()-like event loops typically used by event-runtimes, such as Python async or libuv (NodeJS/JavaScript), or Boost ASIO (C++).

The exact semantics of the returned handle on Windows platforms is yet to be determined.

Param evset:

Pointer to the event set.

Return:

The platform-specific handle, or a negative value on error.

abr_status_t (*evset_poll)(abr_evset_t *evset, int32_t timeout_ms)#

Block the current thread until an event is triggered or timeout.

This function returns once at least one event in the event set is in the “enabled” and “triggered” state.

Events created with the ABR_EVENT_FLAGS_AUTO_DISABLE flag are disabled if they cause abr_evset_poll() to return. Users must explicitly enable events again once they are interested in that event again. This simplifies usage patterns where users need to wait for multiple events to become triggered before something can happen.

Under the hood, this function uses the event platform handle to yield to the operating system. This function may be used if the runtime does not provide an easy way to integrate an event handle into the main asynchronous I/O loop.

Param evset:

Pointer to the event set to poll.

Param timeout_ms:

Timeout in milliseconds. A positive value indicates the maximum time to wait. A value of zero returns immediately with ABR_OK if an event is ready, or ABR_ERR_TIMEOUT otherwise. A negative value denotes an infinite timeout.

Return:

One of the following status codes:

  • ABR_OK: At least one event is triggered. Use abr_event_is_triggered() to determine which event.

  • ABR_ERR_TIMEOUT: The timeout expired.

  • ABR_ERR_INTERRUPT: The process received a signal/interrupt while waiting or abr_evset_interrupt() was called. It is safe to call abr_evset_poll() again in this case.

  • Any other error code: The call failed. If the application failed, every later call returns the same status; free the application to recover.

void (*evset_interrupt)(abr_evset_t *evset)#

Interrupt a thread waiting on the event set platform handle.

The abr_evset_poll() function will unblock and return with ABR_ERR_INTERRUPT if no event has been triggered.

Param evset:

Event set that should be interrupted.

void (*evset_free)(abr_evset_t *evset)#

Free a previously created event set.

Frees the event set and all events connected to it.

Param evset:

Pointer to the event set to free, or NULL.

abr_event_t *(*event_create_buf_level_event)(abr_evset_t *evset, abr_buf_t *buf, uint32_t threshold, abr_event_flags_t flags)#

Register a buffer level event for the given buffer.

The I/O event is triggered if there are threshold bytes available for reading from or writing to the buffer, or if an output buffer holds a flush marker.

Param evset:

Pointer to the event set.

Param buf:

Pointer to the buffer to monitor.

Param threshold:

Number of bytes that must be available to trigger the event.

Param flags:

Bitwise combination of abr_event_flags_t enum values that influence how the event behaves.

Return:

Pointer to the newly created event, or NULL on failure.

void (*event_free)(abr_event_t *event)#

Free a previously created event.

This function does nothing if event is NULL or does not point at a valid event.

Param event:

Pointer to the event to free, or NULL.

bool (*event_is_triggered)(const abr_event_t *event)#

Return true if the event has been triggered.

Returns whether the event has reached its trigger condition. This function is independent of the event’s enabled/disabled status.

Param event:

Pointer to the event to check.

Return:

true if the event is in the triggered state, false otherwise or if event is NULL.

bool (*event_is_enabled)(const abr_event_t *event)#

Return true if the event is currently enabled.

Only enabled events can cause the underlying event set to wake up, i.e., cause the platform handle to become readable or abr_evset_poll() to return. Note that events can only be disabled if the event was created with the ABR_EVENT_FLAGS_AUTO_DISABLE flag, or if it was manually disabled by calling abr_event_disable().

To re-enable an event, call abr_event_enable(). To disable an event call abr_event_disable().

Param event:

Pointer to the event to check.

Return:

true if the event is enabled, false otherwise or if event is NULL.

void (*event_enable)(abr_event_t *event)#

(Re-)enable an event.

Ensures that the given event can cause poll to return, or, when using the platform handle, the handle to become readable.

Param event:

Pointer to the event to enable. Does nothing if the pointer is NULL or the pointer does not point at an event object.

void (*event_disable)(abr_event_t *event)#

Disable an event.

After having been disabled, the event cannot cause poll to return, or, when using the platform handle, the handle to become readable.

Param event:

Pointer to the event to disable. Does nothing if the pointer is NULL or the pointer does not point at an event object.