Skip to content

Latest commit

 

History

History
645 lines (430 loc) · 25.4 KB

README.md

File metadata and controls

645 lines (430 loc) · 25.4 KB
Raw

libASPL Build Doxygen License

Synopsis

libASPL (Audio Server PLugin library) is a C++17 library helping to create macOS CoreAudio Audio Server Plug-In (a.k.a User-Space CoreAudio Driver) with your custom virtual device.

The library acts as a thin shim between Audio Server and your code and takes care of the boilerplate part:

  • Instead of implementing dynamic property dispatch and handling dozens of properties, you inherit classes provided by this library and override statically typed getters and setters that you're interested in.

  • Instead of coping with Core Foundation types like CFString, you mostly work with convenient C++ types like std::string and std::vector.

  • All properties have reasonable default implementation, so typically you need to override only a small subset which is specific to your driver.

  • The library does not hide any Audio Server functionality from you. If necessary, you can customize every aspect of the plugin by overriding corresponding virtual methods.

  • The library also does not introduce any new abstractions. Audio Server properties and callbacks are mapped almost one-to-one to C++ methods.

  • As a bonus, the library performs verbose tracing of everything that happens with your driver. The output and the format of the trace can be customized.

Instructions

Install recent CMake:

brew install cmake

Clone project:

git clone https://github.com/gavv/libASPL.git
cd libASPL

Build and install into /usr/local (headers, static library, and cmake package):

make
sudo make install

You can do more precise configuration by using CMake directly:

mkdir -p build/Release
cd build/Release
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/my/install/path ../..
make -j4
make install

Status

This library is used in commercial and open-source applications and can be considered production-ready.

You can find a real-world usage example here: Roc Virtual Audio Device for macOS.

Versioning

The library uses semantic versioning.

Only source-level compatibility is maintained. There is no binary compatibility even between minor releases. In other words, if you update the library, you should recompile your code with the new version.

Changelog file can be found here: changelog.

API reference

Doxygen-generated documentation is available here.

Example drivers

A complete standalone example drivers with comments can be found in examples directory.

driver name device type description
NetcatDevice output Sound from apps that write to device is mixed and sent over UDP, and can be recorded using netcat tool
SinewaveDevice input Apps that read from device receive infinite sine wave (a loud beep).

You can build examples with this command:

make examples [CODESIGN_ID=...]

You can then (un)install drivers into the system using:

sudo ./examples/install.sh [-u]

The devices should appear in the device list:

$ system_profiler SPAudioDataType
Audio:

    Devices:

        ...

        Netcat Device (libASPL):

          Manufacturer: libASPL
          Output Channels: 2
          Current SampleRate: 44100
          Transport: Virtual
          Output Source: Default

        Sinewave Device (libASPL):

          Input Channels: 2
          Manufacturer: libASPL
          Current SampleRate: 44100
          Transport: Virtual
          Input Source: Default

You can also gather driver logs:

log stream --predicate 'sender == "NetcatDevice"'

Or, for more compact output:

log stream --predicate 'sender == "NetcatDevice"' | sed -e 's,.*\[aspl\],,'

Netcat Device driver sends sound written to it via UDP to 127.0.0.1:4444. The following command receives 1M samples, decodes them, and stores to a WAV file:

nc -u -l 127.0.0.1 4444 | head -c 1000000 | sox -t raw -r 44100 -e signed -b 16 -c 2 - test.wav

Sinewave Device driver writes an infinite sine wave to all apps connected to it. The following command records 5 seconds of audio from device and stores it to a WAV file:

sox -t coreaudio "Sinewave Device (libASPL)" test.wav trim 0 5

The above commands use sox tool, which can be installed using:

brew install sox

Quick start

Add to CMake project using ExternalProject

ExternalProject_Add(libASPL
  URL "https://github.com/gavv/libASPL.git"
  SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/libASPL-src
  BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/libASPL-build
  INSTALL_DIR ${CMAKE_CURRENT_BINARY_DIR}/libASPL-prefix
  CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=<INSTALL_DIR>
  )

target_include_directories(YourDriver
    PRIVATE ${CMAKE_CURRENT_BINARY_DIR}/libASPL-prefix/include
  )
target_link_libraries(YourDriver
    PRIVATE ${CMAKE_CURRENT_BINARY_DIR}/libASPL-prefix/lib/libASPL.a
  )

Add to CMake project using FindPackage

# if libASPL is pre-installed into the system:
find_package(libASPL REQUIRED)

# if libASPL is pre-installed into a directory:
find_package(libASPL REQUIRED
  PATHS /your/install/directory
  NO_DEFAULT_PATH
  )

target_include_directories(YourDriver PRIVATE aspl::libASPL)
target_link_libraries(YourDriver PRIVATE aspl::libASPL)

Minimal driver with no-op device

std::shared_ptr<aspl::Driver> CreateDriver()
{
    auto context = std::make_shared<aspl::Context>();

    auto device = std::make_shared<aspl::Device>(context);
    device->AddStreamWithControlsAsync(aspl::Direction::Output);

    auto plugin = std::make_shared<aspl::Plugin>(context);
    plugin->AddDevice(device);

    return std::make_shared<aspl::Driver>(context, plugin);
}

extern "C" void* EntryPoint(CFAllocatorRef allocator, CFUUIDRef typeUUID)
{
    if (!CFEqual(typeUUID, kAudioServerPlugInTypeUUID)) {
        return nullptr;
    }

    static std::shared_ptr<aspl::Driver> driver = CreateDriver();

    return driver->GetReference();
}

Handler for control and I/O requests

class MyHandler : public aspl::ControlRequestHandler, public aspl::IORequestHandler
{
public:
    // Invoked on control thread before first I/O request.
    OSStatus OnStartIO() override
    {
        // prepare to start I/O
        return kAudioHardwareNoError;
    }

    // Invoked on control thread after last I/O request.
    void OnStopIO() override
    {
        // finish I/O
    }

    // Invoked on realtime I/O thread to read data from device to client.
    virtual void OnReadClientInput(const std::shared_ptr<Client>& client,
        const std::shared_ptr<Stream>& stream,
        Float64 zeroTimestamp,
        Float64 timestamp,
        void* buff,
        UInt32 buffBytesSize)
    {
        // fill data for client
    }

    // Invoked on realtime I/O thread to write mixed data from clients to device.
    void OnWriteMixedOutput(const std::shared_ptr<aspl::Stream>& stream,
        Float64 zeroTimestamp,
        Float64 timestamp,
        const void* buff,
        UInt32 buffBytesSize) override
    {
        // handle data from clients
    }
};

device->AddStreamWithControlsAsync(aspl::Direction::Input);
device->AddStreamWithControlsAsync(aspl::Direction::Output);

auto handler = std::make_shared<MyHandler>();

device->SetControlHandler(handler);
device->SetIOHandler(handler);

Streams and controls

If you want to configure streams and controls more precisely, then instead of:

device->AddStreamWithControlsAsync(aspl::Direction::Output);

you can write:

auto stream = device->AddStreamAsync(aspl::Direction::Output);

auto volumeControl = device->AddVolumeControlAsync(kAudioObjectPropertyScopeOutput);
auto muteControl = device->AddMuteControlAsync(kAudioObjectPropertyScopeOutput);

stream->AttachVolumeControl(volumeControl);
stream->AttachMuteControl(muteControl);

Furthermore, all AddXXX() methods have overloads that allow you to specify custom parameters or provide manually created object. The latter is useful if you want to use your own subclass.

Initialization callback

Driver object is not fully functional until plugin returns from entry point and HAL performs asynchronous driver initialization. If part of your plugin initialization requires functioning driver and so can not be done in entry point (e.g. it uses persistent storage), you can use DriverRequestHandler:

class MyHandler : public aspl::DriverRequestHandler
{
public:
    // Invoked when HAL performs asynchrnous initialization.
    OSStatus OnInitialize() override
    {
        // do initialization stuff that requires functioning driver
        return kAudioHardwareNoError;
    }
};

auto handler = std::make_shared<MyHandler>();

driver->SetDriverHandler(handler);

Tracing

By default, the library traces all operations to syslog. To disable it, construct tracer with Noop mode:

auto tracer = std::make_shared<aspl::Tracer>(aspl::Tracer::Mode::Noop);
auto context = std::make_shared<aspl::Context>(tracer);

// pass context to all objects

You can provide custom tracer implementation:

class MyTracer : public aspl::Tracer
{
protected:
    void Print(const char* message) override
    {
        // ...
    }
};

auto tracer = std::make_shared<MyTracer>();
auto context = std::make_shared<aspl::Context>(tracer);

// pass context to all objects

Persistent storage

libASPL provides a convenient wrapper for CoreAudio Storage API.

Since plugins are running inside a sandbox where filesystem is mostly unavailable, this API may be the most convenient way for storing plugin configuration that should persist across audio server restart and rebooting.

Usage:

auto ok = driver->GetStorage()->WriteString("key", "value");
// ...
auto [str, ok] = driver->GetStorage()->ReadString("key");
if (ok) {
    // ...
}

You can also construct storage manually and pass it to driver:

// ...
auto storage = std::make_shared<aspl::Storage>(context);
// ...
auto driver = std::make_shared<aspl::Driver>(context, plugin, storage);

Object model

Typical AudioServer Plug-In consists of the following components:

  • Factory function, which name is defined in Info.plist, and which should return the function table of the driver.
  • Driver, which consists of the audio object tree and the function pointers table with operations on objects.
  • Audio object tree, consisting of objects such as plugin, device, stream, etc., organized in a tree with the plugin object in the root.

The Audio Server invokes the factory function to obtain the function pointer table of the driver and then issues operations on driver's audio objects.

Each audio object has the following characteristics:

  • it has a numeric identifier, unique within a driver
  • it belongs to one of the predefined audio object classes, also identified by numeric identifiers
  • it has a list of owned audio objects (e.g. plugin owns devices, device owns streams, etc.)
  • it has a set of properties that can be read and written
  • in case of some objects, it has some additional operations (e.g. device has I/O operations)

This diagram shows driver and audio object tree in libASPL:

Audio object classes are organized in a hierarchy as well, with the AudioObject class in the root of the hierarchy. For example, AudioVolumeControl class inherits more generic AudioLevelControl, which inherits even more generic AudioControl, which finally inherits AudioObject.

Audio objects "classes" are more like "interfaces": they define which properties and operations should be supported by an object. Inheritance means supporting of all properties and operations of the parent class too.

In libASPL, there are C++ classes corresponding to some of the leaf audio object classes. All of them inherit from aspl::Object which corresponds to AudioObject and provide basic services liker identification, ownership, property dispatch, etc.

This diagram shows audio object classes and and corresponding libASPL classes.

Note that the classes in the middle of the tree may not have corresponding C++ classes, e.g. there is aspl::VolumeControl for AudioVolumeControl, but there are no C++ classes for AudioLevelControl and AudioControl. Instead, aspl::VolumeControl implements everything needed.

All objects inherited from aspl::Object are automatically registered in aspl::Dispatcher. Driver uses it to find object by identifier when it redirects Audio Server operations to C++ objects.

Types of setters

libASPL objects provide setters of two types:

  • Synchronous setters, named SetXXX().

    Such setters are used for properties which are allowed to be changed on fly at any point. They typically do two things: call protected virtual method SetXXXImpl() to actually change the value (you can override it), and then notify HAL that the property was changed.

  • Asynchronous setters, named SetXXXAsync().

    Such setters are used for properties that can't be changed at arbitrary point of time, but instead the change should be negotiated with HAL. They typically request HAL to schedule configuration change. When the time comes (e.g. after I/O cycle end), the HAL invokes the scheduled code, and the change is actually applied by invoking protected virtual method SetXXXImpl() (which you can override).

Note 1: if you invoke asynchronous setter before you've published plugin to HAL by returning driver from the entry point, the setter applies the change immediately without involving HAL, because it's safe to change anything at this point.

Note 2: if you invoke an asynchronous setter while you're already applying some asynchronous change, i.e. from some SetXXXImpl() method, again the setter applies the change immediately without scheduling it, because we're already at the point where it's safe to apply such changes.

Customization

The library allows several ways of customization.

Builtin properties:

  1. Each object (Plugin, Device, Stream, etc.) can be provided with the custom config at construction time (PluginParameters, DeviceParameters, etc), which defines initial values of the properties.

  2. Each object also provides setters for the properties that can be changed on fly.

  3. If desired, you can subclass any object type (Plugin, Device, etc.) and override statically typed getters and setters. Dynamic property dispatch will automatically use overridden versions.

  4. For even more precise control, you can override dynamic dispatch methods themselves (HasProperty, GetPropertyData, etc).

Custom properties:

  1. Each object allows to register custom properties, i.e. properties not known by HAL but available for apps. Such properties support only limited number of types.

  2. Again, for more precise control of custom properties, you can override dynamic dispatch methods (same as for builtin properties).

Control and I/O requests:

  1. You can provide custom implementations of ControlRequestHandler and IORequestHandler. Device will invoke their methods when serving requests from HAL.

  2. You can also provide custom implementation of Client if you want to associate some state with every client connected to device. See ControlRequestHandler for details.

  3. For more precise control of request handling, you can subclass Device and override some of its control and I/O methods directly. (StartIOImpl, StopIOImpl, WillDoIOOperationImpl, etc.).

  4. You can also replace Device I/O handling entirely with your own logic by overriding top-level I/O methods (StartIO, StopIO, WillDoIOOperation, etc. - ones without "Impl" suffix). In this case you typically need to override all of the methods together, because of their coupling.

Driver requests:

  1. You can provide custom implementations of DriverRequestHandler. Driver will invoke its methods when serving requests from HAL.

  2. For more precise request handling, you can subclass Driver and override its protected methods directly.

Thread and realtime safety

An operation is thread-safe if it can be called from concurrent threads without a danger of a data race.

An operation is realtime-safe if it can be called from realtime threads without a danger of blocking on locks held by non-realtime threads. This problem is known as priority inversion.

In CoreAudio, I/O is performed on realtime threads, and various control operations are performed in non-realtime threads. As a plugin implementer, you should ensure that all code is thread-safe and I/O operations are also realtime-safe. Every non-realtime safe call from an I/O handler would increase probability of audio glitches.

To help with this, libASPL follows the following simple rules:

  • All operations are thread-safe.

  • All operations that don't modify object state, e.g. all GetXXX() and ApplyProcessing() methods, are also non-blocking and realtime-safe. You can call them from any thread. This, however, excludes getters that return STL strings and containers.

  • All operations that modify object state, e.g. all SetXXX(), AddXXX(), and RegisterXXX() methods, are allowed to block and should be called only from non-realtime threads. So avoid calling them during I/O.

  • All operations that are invoked on realtime threads are grouped into a single class IORequestHandler. So typically you need to be careful only when overriding methods of that class.

When overriding libASPL methods, you can either follow the same rules for simplicity, or revise each method you override and make sure it's realtime-safe if there are paths when it's called on realtime threads.

Note that various standard library functions, which implicitly use global locks shared among threads, are typically not realtime-safe. Some examples are: everything that allocates or deallocates memory, including copy constructors of STL containers; stdio functions; atomic overloads for shared_ptr; etc. Basically, you need to carefully check each function you call.

Internally, realtime safety is achieved by using atomics and double buffering combined with a couple of simple lock-free algorithms. There is a helper class aspl::DoubleBuffer, which implements a container with blocking setter and non-blocking lock-free getter. You can use it to implement the described approach in your own code.

Driver initialization

Right after the Driver object is created, it is not fully initialized yet. The final initialization is performed by HAL asynchrnously, after returning from plugin entry point.

Until asynchronous initialization is done, most driver services are not really functioning yet. You can create objects (devices, streams), but they won't actually appear on system until initialization. Persistent storage service is not available yet and trying to use it will produce errors.

If part of your plugin initialization requires fully functioning driver, and so it can not be done at entry point time, you can move initialization code to DriverRequestHandler (see Quick start section). Alternatively, you can subclass Driver and override its Initialize() method.

Sandboxing

AudioServer plugin operates in its own sandboxed process separate from the system daemon.

These things are no specific to libASPL, but worth mentioning:

  • Filesystem access is restricted. You can access only your own bundle, system libraries and frameworks, and temporary and cache directories.

  • IPC is allowed, including semaphores and shared memory. The plugin should list the mach services to be accessed in Info.plist.

  • Networking is allowed, including sockets and syslog. The plug-in should declare this in Info.plist as well.

  • IOKit access is allowed too.

Missing features

Below you can find the list of things that are NOT supported out of the box because so far authors had no need for any of them.

Element-related properties are currently not supported (ElementName, ElementCategoryName, ElementNumberName).

Currently unsupported device-related object types:

  • AudioBox
  • AudioClockDevice
  • AudioTransportManager
  • AudioEndPoint
  • AudioEndPointDevice

Currently unsupported control object types:

  • AudioClipLightControl
  • AudioClockSourceControl
  • AudioDataDestinationControl
  • AudioDataSourceControl
  • AudioHighPassFilterControl
  • AudioJackControl
  • AudioLFEMuteControl
  • AudioLFEVolumeControl
  • AudioLineLevelControl
  • AudioListenbackControl
  • AudioPhantomPowerControl
  • AudioPhaseInvertControl
  • AudioSelectorControl
  • AudioSoloControl
  • AudioTalkbackControl

If you need one of these, you can either implement it on your side, or submit a pull request.

To implement a new object type on your side, you need to derive Object class and manually implement dynamic dispatch methods (HasProperty, IsPropertySettable, etc.).

To prepare a patch that adds support for a new object type, you also need to derive Object class, but instead of implementing dynamic dispatch by hand, provide a JSON file with properties description. The internal code generation framework (see below) will do the rest of the job for you.

Apple documentation

Reading documentation for underlying Apple interfaces can help when working with libASPL. The following links can be useful.

Official examples:

Headers with comments:

Code generation

To solve the issue with lots of the boilerplate code needed for a plugin, libASPL extensively uses code generation.

There are three code generators:

  • scripts/generate-accessors.py - reads JSON description of object's properties and generates C++ code for dispatching dynamic HAL requests to statically typed getters and setters
  • scripts/generate-bridge.py - reads JSON description of C plugin interface and generates C++ code for dispatching HAL calls to corresponding C++ objects calls
  • scripts/generate-strings.py - reads CoreAudio header files and generates C++ code to convert various identifiers to their string names

All of the generators are created using the excellent Jinja Python module.

See CMake scripts for details on how the generators are invoked. The generated files are added to the repo. Unless you modify the sources, CMake wont regenerate them and there is no need to install Jinja.

Hacking

Install tools for code generation (needed if you modify sources):

brew install python3
pip3 install jinja2

Run release or debug build:

make [release_build|debug_build]

Build and run tests:

make test

Run code generation:

make gen

Remove build results:

make clean

Remove build results and generated files:

make clobber

Format code:

make fmt

Contributions are welcome!

Authors

See here.

License

The library is licensed under MIT.

This library is mostly written from scratch, but is inspired by and borrows some pieces from "SimpleAudio" and "NullAudio" plugins from newer and older Apple documentation.

Apple examples are licensed under "MIT" and "Apple MIT" licenses.