Library loading and the ABI functions#
Types and utilities for dynamically loading ABR SDK libraries.
- Author
Andreas Stöckel (Applied Brain Research)
Typedefs
Functions
-
abr_status_t abr_lib_open(abr_lib_t *lib, const char *path, abr_log_cb_t log_cb, void *log_cb_user_data)#
Opens an ABR SDK shared library and places pointers at the SDK functions in the provided
abr_lib_tstructure.Libraries opened by abr_lib_open() must be closed by calling abr_lib_close() to avoid resource leaks.
ABR_OK The library has been successfully opened.
ABR_ERR_INCOMPATIBLE_LIBRARY The library is not compatible with this version of the ABR C SDK.
ABR_ERR_NOT_STATICALLY_LINKED Passed
NULLtopath, but the ABR SDK library has not been linked at compile time.ABR_ERR_SYSTEM The operating system reported an error while attempting to load the library.
ABR_ERR_PLATFORM_UNSUPPORTED Passed a
pathon an embedded platform, which has no shared libraries.
- Parameters:
lib – Pointer at a abr_lib_t structure that should be initialized. Note that abr_lib_open() resets the
libstructure independent of the return value of this function.path – File path to the ABR SDK shared library or
NULL. In caseNULLis provided, then a library linked at compile time is used instead. Note that, depending on the operating system, providing a non-absolute path may result in the OS searching for the shared library in standard locations. Pass an absolute path to avoid this behaviour.log_cb – Log callback that should be used to log events that happen during loading.
log_cb_user_data – Pointer passed as a first parameter to
log_cb.
- Returns:
Returns one of the following status codes. Supply a log callback to gather more information.
-
abr_status_t abr_lib_close(abr_lib_t *lib)#
Closes a previously opened ABR SDK shared library.
Calling this function multiple times on the same
libobject has no effect.- Parameters:
lib – Pointer at the abr_lib_t structure that should be closed. Passing
NULLor an already closed library has no effect.
- Returns:
One of the following status codes.
ABR_OK The library has been closed, or was already closed.
ABR_ERR_INVALID_ARGS Passed
NULLtolib.ABR_ERR_SYSTEM The operating system reported an error while attempting to unload the library.
-
struct abr_lib#
Collection of all functions defined in the ABR SDK ABI.
This structure also includes the system handle for the opened shared library.
Public Members
-
uintptr_t handle#
Platform-specific handle pointing at the underlying shared library.
A value of zero indicates that this abr_lib_t instance does not own a handle at a shared library. This might happen after calling abr_lib_close() or when using compile-time linking.
-
uint32_t (*abi_version)()#
Return an identifier describing the ABI version.
The returned integer is structured as follows:
Bits 16-31: Major version (incompatible ABIs)
Bits 0-15: Minor version (new features, back and forward-compatible)
- Return:
The ABI version as a 32-bit unsigned integer.
-
abr_status_t (*app_create)(abr_app_t **app_p, const abr_config_t *cfg, uint32_t n_cfg, abr_log_cb_t log_cb, void *log_cb_user_data)#
Create a new time-series neural network application.
Users may instantiate an arbitrary number of application instances, as long as the underlying accelerator supports this, and there are enough resources (i.e., memory) left.
Every application instance is optionally linked to a logger. The logger is used to capture log messages from the application.
Application-agnostic keys:
“min_log_level” (ABR_VAR_SINT, see abr_log_level): minimum severity threshold passed to the logger. Defaults to ABR_LOG_DEFAULT.
”resources_dir” (ABR_VAR_STR): filesystem directory the loaded library reads model weights from. Required unless “resources_addr” is given.
”resources_addr” (ABR_VAR_PTR): address of the application data container in memory, for a platform without a filesystem. Given in place of “resources_dir”. The address must be aligned to 16384 bytes, and the memory must remain valid and unchanged until the application is freed.
”verify_resources” (ABR_VAR_BOOLEAN): whether to check the hash of every blob in the application data container, such as the model weights, when the application is created. This reads all of the container. Defaults to false.
”license_dir” (ABR_VAR_STR): filesystem directory in which license data is stored. Required where there is a filesystem, and not accepted on a platform without one.
”license_envelope” (ABR_VAR_STR): the license itself, as written by
abr-sdk activate --fingerprint. Only accepted on a platform without a filesystem, in place of “license_dir”.”alif_se_handle” (ABR_VAR_UINT): handle of the Secure Enclave services of an Alif Ensemble, as
SERVICES_register_channel()returned it to the application. Required under FreeRTOS on an Alif Ensemble, and not accepted elsewhere. The library asks the Secure Enclave for random bytes and for the identity of the device, which abr_get_fingerprint() and abr_activate_license() need too.”litert_delegate” (ABR_VAR_STR): LiteRT kernel path used to run the neural networks. One of:
”builtin”: the TFLite built-in kernels, with no delegate.
”xnnpack”: the XNNPACK delegate.
a filesystem path to an external delegate library. Defaults to the delegate the library was built for. Libraries built for TFLite Micro log that the key is ignored.
”cache_dir” (ABR_VAR_STR): existing, writable directory in which the library stores data that it can recompute, such as neural networks compiled for the target hardware. A directory that does not exist or cannot be written to is rejected. If omitted, the library uses the
abr-sdksubdirectory of the first of the directories listed below, and may create that subdirectory including any missing parent directories. If none of them applies, the library logs a warning and computes that data again on every run.the directory Android assigns to the calling application,
Context.getCacheDir(). Android only, and only in an app process.$XDG_CACHE_HOME, if that environment variable holds an absolute path. Linux only.$HOME/.cache, if theHOMEenvironment variable holds an absolute path the calling process can write to. Linux only.$TMPDIR, if that environment variable is set and not empty./tmp. Linux only.
ASR-specific keys:
“asr_mode” (ABR_VAR_STR): which decoder branch is surfaced. One of:
”fast”: causal decoder (lower-quality, sooner output).
”accurate”: noncausal decoder (higher-quality, lookahead latency).
Defaults to “accurate”.
”asr_enable_spellcheck” (ABR_VAR_BOOLEAN): whether the spell checker corrects the transcript. Ignored if the library has no spell checker. Defaults to
true.”asr_enable_pcr” (ABR_VAR_BOOLEAN): whether the punctuation and capitalization (PCR) network post-processes the transcript. Ignored if the library has no PCR network. Defaults to
true.”asr_vocabulary” (ABR_VAR_STR): path of the custom vocabulary artifact whose terms the vocabularizer substitutes into the transcript. The artifact is an
.abrdbplugin of this application, built for its network (otherwise ABR_ERR_INCOMPATIBLE_APP). A relative path is resolved within “resources_dir”. Ignored if the library has no vocabularizer. Defaults to none.
Depending on the error code, additional information may have been logged.
- Param app_p:
Pointer to the variable where the newly created application handle will be stored. This will be set to
NULLif initialization fails.- Param cfg:
Pointer to an array of configuration options. May be
NULLif no additional configuration is needed. Recognized keys depend on the loaded application; the library picks the default for any key the caller omits. Application-specific keys are namespaced by the application name (e.g. ASR exposes “asr_*”).- Param n_cfg:
Number of configuration options in the
cfgarray.- Param log_cb:
Pointer to the log callback function. Passing
NULLdisables logging.- Param log_cb_user_data:
Arbitrary pointer passed as the first argument to
log_cb. Useful for object-oriented patterns wherelog_cbis a member function andlog_cb_user_datais a class instance.- Return:
One of the following status codes:
ABR_OK: A valid pointer has been written to
abr_p, and the application is ready to use.ABR_ERR_LICENSE: No application has been created because this device does not hold a valid license. Call abr_activate_license() to activate the device. See the log for more details.
ABR_ERR_PLATFORM_UNSUPPORTED: This application library is not supported on this machine. You may be attempting to open an application library that uses a hardware accelerator that is not available on your machine.
ABR_ERR_BUSY: The targeted accelerator is already busy executing another application.
ABR_ERR_SYSTEM: A system call returned with an error.
-
abr_status_t (*get_fingerprint)(abr_fingerprint_t *fingerprint, const abr_config_t *cfg, uint32_t n_cfg, abr_log_cb_t log_cb, void *log_cb_user_data)#
Returns this device’s fingerprint, as reported by the library.
Does not require an application instance. A device without a hardware identity is identified by a random identity that is created in
license_diron first use and reused afterwards; the directory is created if it does not exist.- Param fingerprint:
Pointer to the structure where the fingerprint will be stored.
- Param cfg:
Configuration options that would normally be passed to abr_app_create(). The relevant keys are
license_dir,alif_se_handleandmin_log_level; all other keys are ignored.- Param n_cfg:
Number of configuration options in the
cfgarray.- Param log_cb:
Pointer to the log callback function. Passing
NULLdisables logging.- Param log_cb_user_data:
Arbitrary pointer passed as the first argument to
log_cb. Useful for object-oriented patterns wherelog_cbis a member function andlog_cb_user_datais a class instance.- Return:
One of the following status codes:
ABR_OK: The fingerprint has been written to
fingerprint.ABR_ERR_INVALID_ARGS: The
fingerprintpointer is NULL, orcfgis NULL whilen_cfgis non-zero.ABR_ERR_INVALID_CONFIG: The
license_dirconfiguration key is missing.ABR_ERR_LICENSE: This device has no usable identity. See the log for more details.
ABR_ERR_SYSTEM: The random identity could not be stored in
license_dir.
-
abr_status_t (*activate_license)(const char *envelope, const abr_config_t *cfg, uint32_t n_cfg, abr_log_cb_t log_cb, void *log_cb_user_data)#
Activate a license for this device given the server response.
Does not require an application instance; activation only depends on the license directory and the device this function is called on.
- Param envelope:
Pointer at a zero-terminated string containing the signed license envelope.
- Param cfg:
Configuration options that would normally be passed to abr_app_create(). The relevant keys are
license_dir,alif_se_handleandmin_log_level; all other keys are ignored and logged at the debug level.- Param n_cfg:
Number of configuration options in the
cfgarray.- Param log_cb:
Pointer to the log callback function. Passing
NULLdisables logging.- Param log_cb_user_data:
Arbitrary pointer passed as the first argument to
log_cb. Useful for object-oriented patterns wherelog_cbis a member function andlog_cb_user_datais a class instance.- Return:
One of the following status codes:
ABR_OK: The device has been activated.
ABR_ERR_INVALID_ARGS: The
envelopepointer is NULL, orcfgis NULL whilen_cfgis non-zero.ABR_ERR_INVALID_CONFIG: The
license_dirconfiguration key is missing.ABR_ERR_LICENSE: The license could not be validated. See the log for more details.
ABR_ERR_SYSTEM: The random identity could not be stored in
license_dir.
-
void (*app_free)(abr_app_t *app)#
Free the application instance and all resources held by it.
Note that this invalidates all other handles that (directly or indirectly) have this application instance as a parent.
This function does nothing if
appisNULL; i.e., it is safe to call abr_app_free() independent of whether the library has been initialized correctly or not.- Param app:
Pointer to the application instance to free, or
NULL.
-
uint32_t (*app_get_buffer_count)(const abr_app_t *app)#
Return the number of I/O buffers available in the application.
- Param app:
Pointer to the application instance for which the buffer count should be obtained.
- Return:
The number of buffers, or zero if
appisNULL.
-
abr_buf_t *(*app_get_buffer)(abr_app_t *app, uint32_t idx)#
Fetch the i-th buffer from the application.
Calling this function multiple times with the same index will always return the same buffer object. There is no need to free these objects after use.
Input buffers come first, in graph order, followed by the output buffers. ASR, TTS and PCR libraries expose exactly one of each.
- Param app:
Pointer to the application instance.
- Param idx:
Zero-based index of the buffer object to retrieve.
- Return:
A pointer to the buffer, or
NULLifappisNULLoridxis out of range.
-
void *(*get_parent)(void *obj)#
Return the parent object of the specified object.
-
uint32_t (*get_property_count)(void *obj)#
Return the number of properties attached to the given object.
Every ABR object may have a series of runtime-defined properties attached. These properties are used for values that do not need to be accessed very often. Using a generic interface avoids having to add new functions to the C ABI every time a new property is exported.
Passing
NULLprovides information about the application library itself.
-
const char *(*get_property_name)(void *obj, uint32_t idx)#
Return the property name by index.
- Param obj:
Pointer to a ABR object, or
NULLfor the global “meta” scope.- Param idx:
Zero-based index of the property.
- Return:
The property name, or
NULLif the index is out of range.
-
abr_variant_t (*get_property_value_by_idx)(void *obj, uint32_t idx)#
Return a property value by index.
- Param obj:
Pointer to a ABR object, or
NULLfor the global “meta” scope.- Param idx:
Zero-based index of the property.
- Return:
The property value, or an “undefined” variant if the index is out of range or the object is not known.
-
abr_variant_t (*get_property_value_by_name)(void *obj, const char *key)#
Return a property value by name.
- Param obj:
Pointer to a ABR object, or
NULLfor the global “meta” scope.- Param key:
The property name to look up. Keys are case-sensitive.
- Return:
The property value, or an “undefined” variant if the name is not found or the object is not known.
-
uint32_t (*buf_get_size)(const abr_buf_t *buf)#
Return the total buffer capacity in bytes.
- Param buf:
Pointer to the buffer.
- Return:
The buffer capacity in bytes, or zero if
bufisNULL.
-
uint32_t (*buf_get_level)(const abr_buf_t *buf)#
Return the number of bytes that can be pushed or pulled.
- Param buf:
Pointer to the buffer.
- Return:
The number of bytes that can be pushed, or pulled up to the next flush marker; zero if
bufisNULL.
-
int32_t (*buf_push)(abr_buf_t *buf, const uint8_t *data, uint32_t n_bytes, bool flush)#
Push (write) data onto a buffer.
Does nothing if
bufis an output buffer.- Param buf:
Pointer to the buffer.
- Param data:
Pointer to the data to push.
- Param n_bytes:
Number of bytes to push.
- Param flush:
If
true, pushes a “flush marker” into the buffer. This indicates that this is the end of a data segment, and that the application should process all data made available so far. If not alln_bytesfit, the marker is dropped; push the rest withflushset again. A push of zero bytes always places the marker.- Return:
The number of bytes actually pushed onto the buffer, or a negative error code (compatible with abr_status_t) on failure. Once the application has failed, every call returns the status of that failure; free the application to recover.
-
int32_t (*buf_pull)(abr_buf_t *buf, uint8_t *data, uint32_t n_bytes, bool *flush)#
Pull (read) data from a buffer.
Does nothing if
bufis an input buffer.- Param buf:
Pointer to the buffer.
- Param data:
Pointer to the buffer where pulled data will be stored.
- Param n_bytes:
Maximum number of bytes to pull.
- Param flush:
Pointer to a boolean flag, or
NULL. Set totrueif the pull stopped at a flush marker and consumed it, which typically happens once for every flush flag pushed into the pipeline. A pull withn_bytesof zero never reports or consumes a marker; a pull that returns zero bytes may still report one, at the end of an empty segment.- Return:
The number of bytes actually pulled from the buffer, or a negative error code (compatible with abr_status_t) on failure. Once the application has failed, every call returns the status of that failure; free the application to recover.
-
abr_evset_t *(*evset_create)(abr_app_t *app)#
Create a new event set connected to the given application instance.
The event set (short “evset”) may be used by a single user thread to wait for events on the application’s buffers.
- Param app:
Pointer to the application instance.
- Return:
Pointer to the newly created event set, or
NULLon failure.
-
int64_t (*evset_get_platform_handle)(abr_evset_t *evset)#
Return the platform-specific event handle.
The event handle may be used to yield to the operating system until a buffer event (configured through the buffer interface) happens.
On POSIX-like systems, the returned handle is a file descriptor of a
socketpair()or a compatible mechanism such aseventfd()on Linux. An event results in eight bytes becoming readable from the file descriptor. The user must read those eight bytes to confirm the event.The file descriptor is meant to be used in conjunction with
poll()-likeevent loops typically used by event-runtimes, such as Pythonasyncorlibuv(NodeJS/JavaScript), or Boost ASIO (C++).The exact semantics of the returned handle on Windows platforms is yet to be determined.
- Param evset:
Pointer to the event set.
- Return:
The platform-specific handle, or a negative value on error.
-
abr_status_t (*evset_poll)(abr_evset_t *evset, int32_t timeout_ms)#
Block the current thread until an event is triggered or timeout.
This function returns once at least one event in the event set is in the “enabled” and “triggered” state.
Events created with the ABR_EVENT_FLAGS_AUTO_DISABLE flag are disabled if they cause abr_evset_poll() to return. Users must explicitly enable events again once they are interested in that event again. This simplifies usage patterns where users need to wait for multiple events to become triggered before something can happen.
Under the hood, this function uses the event platform handle to yield to the operating system. This function may be used if the runtime does not provide an easy way to integrate an event handle into the main asynchronous I/O loop.
- Param evset:
Pointer to the event set to poll.
- Param timeout_ms:
Timeout in milliseconds. A positive value indicates the maximum time to wait. A value of zero returns immediately with ABR_OK if an event is ready, or ABR_ERR_TIMEOUT otherwise. A negative value denotes an infinite timeout.
- Return:
One of the following status codes:
ABR_OK: At least one event is triggered. Use abr_event_is_triggered() to determine which event.
ABR_ERR_TIMEOUT: The timeout expired.
ABR_ERR_INTERRUPT: The process received a signal/interrupt while waiting or
abr_evset_interrupt()was called. It is safe to call abr_evset_poll() again in this case.Any other error code: The call failed. If the application failed, every later call returns the same status; free the application to recover.
-
void (*evset_interrupt)(abr_evset_t *evset)#
Interrupt a thread waiting on the event set platform handle.
The
abr_evset_poll()function will unblock and return with ABR_ERR_INTERRUPT if no event has been triggered.- Param evset:
Event set that should be interrupted.
-
void (*evset_free)(abr_evset_t *evset)#
Free a previously created event set.
Frees the event set and all events connected to it.
- Param evset:
Pointer to the event set to free, or
NULL.
-
abr_event_t *(*event_create_buf_level_event)(abr_evset_t *evset, abr_buf_t *buf, uint32_t threshold, abr_event_flags_t flags)#
Register a buffer level event for the given buffer.
The I/O event is triggered if there are
thresholdbytes available for reading from or writing to the buffer, or if an output buffer holds a flush marker.- Param evset:
Pointer to the event set.
- Param buf:
Pointer to the buffer to monitor.
- Param threshold:
Number of bytes that must be available to trigger the event.
- Param flags:
Bitwise combination of
abr_event_flags_tenum values that influence how the event behaves.- Return:
Pointer to the newly created event, or
NULLon failure.
-
void (*event_free)(abr_event_t *event)#
Free a previously created event.
This function does nothing if
eventisNULLor does not point at a valid event.- Param event:
Pointer to the event to free, or
NULL.
-
bool (*event_is_triggered)(const abr_event_t *event)#
Return
trueif the event has been triggered.Returns whether the event has reached its trigger condition. This function is independent of the event’s enabled/disabled status.
- Param event:
Pointer to the event to check.
- Return:
trueif the event is in the triggered state,falseotherwise or ifeventisNULL.
-
bool (*event_is_enabled)(const abr_event_t *event)#
Return
trueif the event is currently enabled.Only enabled events can cause the underlying event set to wake up, i.e., cause the platform handle to become readable or abr_evset_poll() to return. Note that events can only be disabled if the event was created with the ABR_EVENT_FLAGS_AUTO_DISABLE flag, or if it was manually disabled by calling abr_event_disable().
To re-enable an event, call abr_event_enable(). To disable an event call abr_event_disable().
- Param event:
Pointer to the event to check.
- Return:
trueif the event is enabled,falseotherwise or ifeventisNULL.
-
void (*event_enable)(abr_event_t *event)#
(Re-)enable an event.
Ensures that the given event can cause
pollto return, or, when using the platform handle, the handle to become readable.- Param event:
Pointer to the event to enable. Does nothing if the pointer is
NULLor the pointer does not point at an event object.
-
void (*event_disable)(abr_event_t *event)#
Disable an event.
After having been disabled, the event cannot cause
pollto return, or, when using the platform handle, the handle to become readable.- Param event:
Pointer to the event to disable. Does nothing if the pointer is
NULLor the pointer does not point at an event object.
-
uintptr_t handle#