Operational systems course project at UCU (Ukrainian Catholic University).
Warning: instruction lacks practical examples of installation and debugging.
🇺🇦 For Ukrainian translation click here
- Introduction
- macOS tools
- Drivers using DriverKit framework –– about
- Drivers using DriverKit framework –– example
- Additionaly
- Sources/literature
Let's get started and enter the magical world of the drivers creation, and May the Force be with You.
This instruction is mostly based on the official guidelines [3] for writing drivers with DriverKit SDK, DriverKit documentation on Apple Developer website and the sample code [5].
Before diving into more specific tools and examples, let's check some of the definitions and brief details about drivers and their development.
- Dext – driver extension
- Kext – kernel extension
- SDK – Software Development Kit
- HID - Human Interface Device
- API - Application Programming Interface
- I/O - Input / Output
- NIC - Network Interface Controller
- USB - Universal Serial Bus
- IIG - I/O Kit Interface generator
- RTTI - runtime type information
- PCI - Peripheral Component Interconnect
- SIP - System Integrity Protection
- plist - property list
Some notions definitions:
- driver - code/program for control of a hardware device.
- nub – is an object, which represents a communication channel for a device. (more here)
Essentially, a driver is a specific code, which controls a corresponding I/O device, attached to the computer [2]. In other words, drivers can be viewed as a bridge between computer peripherals and the rest of the system. [3] So, it is a mean of communication and control.
Andrew Tanenbaum’s “Modern Operating Systems” [2] provides a good overview of the drivers, which run in the kernel space, but in this tutorial, we will mostly focus on the drivers, which run in the user space.
If You would like to check the I/O Registry (it contains a dynamic "tree" with nubs and drivers) of Your system, You can run the following command in terminal: ioreg.
As told in Amit Singh's "Mac OS X Internals. A Systems Approach" [1], even though usually writing drivers can be considered difficult, the macOS driver architecture is helpful in this regard. One of the most appealing advantages is that it supports user space drivers (the importance of which we will discuss a little later –– in the DriverKit section).
The book, mentioned above provides a great overview of drivers architecture and task of writing them for the macOS systems. Here I will include some of the details, that might be important for understanding, when only starting working with drivers in general and macOS drivers in particular.
Usually, typical Unix systems use device special files (which reside in the /dev/ directory) for the user interface with devices. Newer systems (macOS included) also manage devices more dynamically –– they allow to dynamically create or delete (and automatically assign) these device files. macOS provides device files for storage devices, serial devices, pseudo-terminals, and several pseudo-devices.
In this instruction, we will discuss two possible options, regarding the choice of tools, when writing drivers for macOS systems. The first is the I/O kit –– a collection of frameworks libraries, tools, and other resources for creating device drivers, and the second is the DriverKit –– a modernized replacement of the I/O Kit.
The I/O Kit is a collection of both kernel-level and user-level software, that is used as a simplified driver development mechanism. The I/O Kit also coordinates the use of device drivers. [1]
When writing drivers for macOS using I/O Kit, the drivers is essentially an I/O Kit object, which manages a specific piece of hardware. [1]
If You are interested in I/O Kit, You can check its official documentation.
Information about DriverKit is retrieved from and based on the official presentation of the kit, available by the following link and on the official documentation.
DriverKit is an SDK, which was introduced during the Worldwide Developers Conference in 2019 along with System Extensions. All DriverKit frameworks are based on the I/O Kit, but they were modernized to be more reliable, secure, and comfortable for development. DriverKit is used to build Driver Extensions (dexts) in the user space.
Driver extensions are built using DriverKit, they are now a replacement for I/O Kit device drivers.
They are used to control the following devices:
- USB
- Serial
- NIC (Network Interface Controller)
- HID (Human Interface Device)
As described in the video by the following link there are several advantages to using System Extensions and DriverKit:
- Unlike kexts, System Extensions run in the user space, bugs in them cannot compromise the kernel
- System Extensions have no restrictions on dynamic memory allocation, synchronization, and latency
- Building, testing, and debugging can all be performed on one machine
- Enabled full debugger support
- There is no need to restart the machine if the extension crushes
- Kernel and other applications will not stop running if the extension crushes
- DriverKit provides full modern replacement of the previously discussed I/O kit
- DriverKit drivers run in user space, where, just like System Extensions, they cannot compromise kernel and shutter the security
- These extensions are available to all the users, even for tasks, previously performed only by kexts
- Et cetera
Furthermore, starting with new versions of macOS, kexts will be deprecated. It is officially stated by the Apple developer website, that devices supported on macOS 11 and later require DriverKit instead of I/O Kit. For some details on audio drivers check out this video-tutorial
We will try the newer, more secure, and, perhaps, an easier approach –– DriverKit framework.
In this part we will continue discussing DriverKit, but in more detail. So, in DriverKit Dext (Driver extension) –– is a system extension, which controls hardware and is available to the entire system. There are even some DriverKit drivers, which come with Catalina, so even tho this kit is quite new, it is approved and officially used by Catalina system developers.
As it was already said, dexts work in the userspace, so their workflow is a bit different from the kernel extensions. In short, when a device appears (for which we created or already have a driver extension), I/O Kit matching (which is already prewritten) creates a kernel service to represent Your service. Then system starts a process for the driver (for example, written by You). There will be a new instance of driver for each devices (so a new process is started for each of them).
For more detailed information on that topic check video-presentation [4] of the DriverKit approximately from the 12th minute.
As it was told before, DriverKit API is based on the the I/O Kit API. This new, DriverKit API, is limited, it has no direct access to file system, networking and IPC. Some of it's classes are based on corresponding I/O Kit classes, some are completely new (for more info visit Classes in DriverKit section).
To create a dext project, You can use a template from the Xcode. It will be a great starting point for development of Your driver.
DriverKit uses classes for dext development. So, to start creating driver we should define its class. There is a special file, which holds class definition for a driver. It is the .iig file, interface of the driver is described in it. Such files are processed by the IIG (I/O Kit Interface Generator) tool. This file consists of a class with standard C/C++ types and structure definitions (although it has some new attributes for messaging and dispatch queues that allow communication with separate address spaces), and it is compiled using Clang compiler.
The basic class definition requires You to override following methods (although the most basic .iig file contents from the Xcode template provides only Start method, so this others are not mandatory but are still needed in most cases):
{
init()
Start()
Stop()
free()
}
Example of the class definition:
class ExampleDriver: public IOService
{
public:
virtual bool init() override;
virtual kern_return_t Start(IOService * provider) override;
virtual kern_return_t Stop(IOService * provider) override;
virtual void free() override;
};Depending on the device You are writing driver for, You will want to use different families of drivers and hence You will need to implement more specific and custom methods. For example, if You want to write a driver for a keyboard, You will need to use HIDDriverKit framework and write different methods such as parseKeyboardElement, handleKeyboardReport, etc. They will usually have the LOCALONLY macro, which means that they will run only locally in this dext's process space.
So, different families have different class definitions and implementations.
Moving to implementations: class methods implementations reside in the .cpp file with the same name as Your dext project (and with the same name as the .iig file, too). Here You implement methods from the class definition. But firstly, You will need to define a structure with Instance variable definitions. These will be variables, memory for which will be allocated during the initialization.
Here is a little example how the structure will look like:
struct ExampleDriver_IVars
{
OSBoolean example_variable1;
OSString example_variable2;
};For some kind of a more complicated device (for example, keyboard), You will need variables which will hold keyboard elements in an OSArray, etc. In short, different families –– different required variables.
Now, moving to methods implementations, here is what each method does (not including methods, specific to different devices and families):
- init() – here memory for instance variables must be allocated (so You allocate structure ..._IVars)
- free() - releases instance variables, allocated during init()
- Start() - starts service, validates and matches the provider to the service, configures data structures, prepares hardware, uses other custom methods, etc
- Stop() - undoes everything, which Start() did
To install Your dext, You will also need an App for its activation (more about it in the section below).
Also, to run Your dexts, You will need entitlements (more about them in Entitlements section), even tho for debug You can omit that step. Furthermore, to build system extension You need Developer ID (more about them in Additionally section). For distribution extension must be signed with Your Apps signing certificate and it must be Notarized (more about Notarization here).
You also need to get the Info.plist file ready, because it holds property list and among properties it has a key, which is involved during device matching (more about it in the Info.plist and matching section).
Now the only thing left is to build the App and launch it. After activation the lifecycle of the driver will be managed by the system. You should then be able to find out its process ID using ps -ax | grep -i myuser command and debug it using sudo lldb command.
For information on debug and local development (with turned off SIP) visit Debug section, which is a part of example/case-study part of the instruction.
Each system extension (including dexts, which are our main concern) comes with an App. It belongs to this App's bundle, so the user can install an App in order to install Your custom system extension. So, they are distributed with Apps (that requires Developer ID, more about it in Additionally section).
The activationRequest makes extension available. It can be activated in the App launch, but it is not mandatory. For example, Your app might have some kind of interaction with the user, before install extension (it might ask for user's permission, etc). From the moment of activation the lifecycle of the driver extension will be managed by the system itself. It, for example, means that dext will start the moment it is needed –– when a matching device is connected to the system, etc.
In the example, available in the repository, there is also example of an App, written in Swift.
Also, extension is a separate bundle from You App (even tho it is embedded into it), so it has a separate Info.plist (more about it in the Info.plist and matching section) and it is a separate target in Xcode. Drivers are flat bundles (as You will see, they have no contents folder).
To conclude, to install driver You do not need an instaler or package –– driver will already be a part of Your App bundle.
DriverKit uses different classes for the dext development. There are classes for the drivers themselves, for memory operations, queues, interrupts, timers, etc.
Classes in DriverKit are either substitutes for corresponding classes from I/O Kit, similar to them, or completely new.
Here are some examples of such classes:
- Similar to I/O Kit:
- IOService
- IOMemoryDescriptor
- IOBufferMemoryDescriptor
- and others...
- Replace other classes from I/O Kit:
- IODispatchQueue
- IOInterruptDispatchSource
- IOTimerDispatchSource
- IODispatchQueue::DispatchSync
- IODispatchQueue::DispatchAsync
- and others...
- Completely new classes:
- OSAction
- and others...
More classes here.
A little about some of the classes functionality/meaning:
- IOService is one of the main classes in the kit, it is the base class for Your driver, it has an I/O Kit lifecycle API: Start/Stop/Terminate.
- IODispatchQueue is important, because all the methods are invoked on a queue and drivers control their own queues.
- IODispatchQueue, IOInterruptDispatchSource, IOTimerDispatchSource, IODispatchQueue::DispatchSync, IODispatchQueue::DispatchAsync are also used for event handling.
- OSAction class (for C Function Pointer representation) encapsulates callback from I/O Kit API.
- etc
DriverKit supports different device familes, which provide abstractions of different devices (while IOService represents all devices). So, there are different frameworks for different families, such as:
- NetworkingDriverKit (link)
- HIDDriverKit (link) (used in the example with the keyboard)
- USBDriverKit (link)
- USBSerialDriverKit (link)
- SerialDriverKit (link)
- PCIDriverKit (link)
Considering their functionality, different families have different class definitions and implementations.
As it is described in the documentation C++ is restricted for the I/O Kit (so only a subset of the language can be used in driver development). I/O Kit does not allow:
- exceptions
- multiple inheritance
- templates
- RTTI (runtime type information)
I did not find information on same matters for the DriverKit, but we should consider that it was created on the basis of I/O Kit, so it might have the same restrictions (at least a part of them). On the other hand, dexts run in user space, so they might not have the same restrictions. In the video-presentation [4] it was said, that DriverKit allows dynamic memory allocation (which kernel extensions do not), so there is no such limit for it. It still discusses some restrictions on dexts. For example, dexts must run in a tailored runtime, which isolates them from the rest of the system.
Other limits, which were discussed previosly, are API limits: there is no direct access to file system, networking and IPC.
On a further note, the default language for the DriverKit API is C++17.
Entitlements are required as one of the security measures, they declare capabilities of extensions.
In order for driver to interact with devices and services, You are required to request the entitlement for DriverKit development from Apple.
The system loads only drivers, which that have a valid set of entitlements, that is why You cannot develop a complete product without them. The DriverKit entitlements give your driver permission to run as a driver and define the type of hardware Your driver supports.
To perform installation, your app must have the System Extension entitlement. In short, You require following entitlements:
- com.apple.developer.driverkit (for all drivers)
- transport entitlement, to take control of the device (they are individual for device types, there is example for USB: com.apple.developer.driverkit.transport.usb)
- family entitlement (specific for different families, there is example for HID: com.apple.developer.driverkit.family.hid.device)
To request entitlement:
- Go to https://developer.apple.com/system-extensions/ and follow the link to request an entitlement.
- Apply for the DriverKit entitlement.
- Provide a description of the apps you’ll use.
Entitlements file with .entitlements extensions contains them.
Info.plit (property list) file has usage description of the extension (what it does, why should user use it, etc). Also, the IOKitPersonalities key from the plist file, is used so that the system can understand for which device this driver is suitable. That is, when the system looks for a driver to use for a particular device, it will check whether the information from this key is appropriate for the device, or not. More about that here in "Provide Version and Description Information" and "Specify Criteria for Matching Your Services Against Devices" sections.
Also, for dext update You should change version in the plist, so the system understands, that the driver should be updated.
- Create an App project in Xcode.
- Add a separate target to it, for which use DriverKit Driver template from Xcode.
- Choose which DriverKit family suits You device best.
- Complete class definition in the .iig file.
- Define instance variables and implement methods from the .iig file in the .cpp file.
- Request entitlements for Your dext and add them to the .entitlements file.
- Complete Info.plist file.
- Create You App, and add activationRequest to it.
- Lauch App.
- Debug/use dext.
+ You will need to obtain a Developer ID or another fitting certificate for dext development.
An example, or case-study of DriverKit dext development. Following instruction will provide an example of the keyboard driver creation, will repeat some information from the previos sections for further understanding and will provide some new information.
To start a project, we will create it in the Xcode, which provides a base template for creating DriverKit drivers.
We will add the driver to a pre-existing project, because drivers, created with DriverKit require an app to install and to use them.
Start with creating a new app in the Xcode:
P.S. You can choose another name, which You would prefer, and specify Your organization identifier.
Using Swift for the app (driver itself will be written using C++) like in the official documentation:
Now, we will add driver to the project:
Choose a DriverKit driver:
Choose options:
Now You should be able to see a somewhat similar window:
Congratulation! We are almost done.
Let's try to build a basic keyboard driver using the template we obtained just now and explore it at the same time. For this task we will use parts of code from the official apple documentation [3].
Firstly, what do we have in the template for the driver creation? Take a look at the directory, which is called the same way as Your project itself:
- *.cpp –– is a file with main C++ source file.
- *.iig –– is an IOKit interface generator header file.
- *.entitlements –– is a default entitlements file. We will discuss what are entitlements. and how to populate that file in the next section.
- *.plist –– is a file with specific information to support the loading and installation of the driver.
Let's begin to go through code of the Keyboard device from the documentation. Firstly, let's check out contents of the *.iig file (DriverExample.iig in my case):
#ifndef DriverExample_h
#define DriverExample_h
#include <Availability.h>
#include <DriverKit/IOService.iig>
class DriverExample: public IOService
{
public:
virtual kern_return_t
Start(IOService * provider) override;
};
#endif /* DriverExample_h */It will look just like that. (Note that this information applies on the November 13th of 2021, and there can be slight changes of API in the millennia You currently live in)
Here, IOService –– is a a base class of all the drivers. We can continue working with it, but it would be better to
work with something more specific. There are different families of classes, provided by the DriverKit and for our case
we will choose a class for handling HID events. Why exactly HID (Human Interface Device)? It is because keyboard belongs
to such devices and we want a class to somehow obtain information on what happens with it. The exact class we would
choose instead of the base one is the IOUserHIDEventService.
That is how class in Your *.iig file should look like now (plus the new additional include):
#include <HIDDriverKit/IOUserHIDEventService.iig>
class DriverExample: public IOUserHIDEventService
{
public:
virtual kern_return_t
Start(IOService * provider) override;
};We will also need to implement some init and free methods, so we will now add them to header file, too. That is how the whole file should look like now:
#ifndef DriverExample_h
#define DriverExample_h
#include <Availability.h>
#include <DriverKit/IOService.iig>
#include <HIDDriverKit/IOUserHIDEventService.iig>
class DriverExample: public IOUserHIDEventService
{
public:
virtual bool init() override;
virtual void free() override;
virtual kern_return_t
Start(IOService * provider) override;
};
#endif /* DriverExample_h */Now let's move on to our main source code file –– Your *.cpp file (DriverExample.cpp in my case). It should currently look like that:
#include <os/log.h>
#include <DriverKit/IOUserServer.h>
#include <DriverKit/IOLib.h>
kern_return_t
IMPL(DriverExample, Start)
{
kern_return_t ret;
ret = Start(provider, SUPERDISPATCH);
os_log(OS_LOG_DEFAULT, "Hello World");
return ret;
}To work with a HID service we need some more includes, so let's add them:
#include <DriverKit/OSCollections.h>
#include <HIDDriverKit/HIDDriverKit.h>When the system will instantiate your driver's service class, it will call its init method. Let's add code of this method:
struct DriverExample_IVars
{
OSArray *elements;
struct {
OSArray *elements;
} keyboard;
};
bool DriverExample::init()
{
if (!super::init()) {
return false;
}
ivars = IONewZero(DriverExample_IVars, 1);
if (!ivars) {
return false;
}
exit:
return true;
}(This code goes after the #include "YouProjectName.h" and before the implementation of the start of the service)
Here, during initialization time we allocate space for the driver's variables ––
elements and a keyboard (that contains elements) in our case
(here You can see variables in the DriverExample_IVars structure).
So, following this example, You would need to define a structure with variables, that You driver requires and
and allocate this structure in the init() method.
We allocated instance variables for the keyboard driver, so now we need a method to free the memory from them (further examples of code are from/based on code from [5]):
void DriverExample::free()
{
if (ivars) {
OSSafeReleaseNULL(ivars->elements);
OSSafeReleaseNULL(ivars->keyboard.elements);
}
IOSafeDeleteNULL(ivars, DriverExample_IVars, 1);
super::free();
}This free() method will be called before unloading our service.
Now, let's start organizing our Start method:
kern_return_t
IMPL(DriverExample, Start)
{
kern_return_t ret;
ret = Start(provider, SUPERDISPATCH);
if (ret != kIOReturnSuccess) {
Stop(provider, SUPERDISPATCH);
return ret;
}
//
// Here the code of the startup tasks will go
//
RegisterService();
return ret;
}This method will be called, when the system will be ready to process information from the device.
In this method driver performs all the various start up tasks: variables initializations, changing device settings, allocating memory for data buffers, et cetera.
Now let's add some start up tasks to the code. This code is based on the sample from [5].
kern_return_t
IMPL(DriverExample, Start)
{
kern_return_t ret;
ret = Start(provider, SUPERDISPATCH);
if (ret != kIOReturnSuccess) {
Stop(provider, SUPERDISPATCH);
return ret;
}
os_log(OS_LOG_DEFAULT, "Hello from Your first DriverKit driver!");
ivars->elements = getElements();
if (!ivars->elements) {
os_log(OS_LOG_DEFAULT, "Failed to get elements");
Stop(provider, SUPERDISPATCH);
return kIOReturnError;
}
ivars->elements->retain();
os_log(OS_LOG_DEFAULT, "The startup task is now finished.");
RegisterService();
return ret;
}To actually work with data from the keyboard, You would also need to parse arguments after retaining them. Parsing sample code is also available at [5].
Congratulations! That is actually Your first DriverKit driver! Even though it doesnt really do anything with data from the keyboard (it just retains it) it is, nevertheless, a driver. Yet it is not The End –– in order to run that driver You need to perform some more, less code-oriented, steps.
Based on guidelines and recommendations from [6].
Now that we have our own little driver we might want to test it and use it. In order to do the we first need to install and activate our driver.
The thing is –– all the drivers come with an app, and for DriverKit having an app is a requirement. So we don't just install drivers, we install them from the corresponding app.
For the installation we will use an example driver from the official Apple developer website, which is already approved and is availabe in this repository.
The example in examples/HandlingKeyboardEventsFromAHumanInterfaceDevice provides full code for the Swift app and code for the driver (partly discussed previously).
Lets see which part of the Swift app is related to the driver:
// Activate the driver.
let request = OSSystemExtensionRequest.activationRequest(forExtensionWithIdentifier: driverID, queue: DispatchQueue.main)
request.delegate = self
let extensionManager = OSSystemExtensionManager.shared
extensionManager.submitRequest(request)
Here You can see the activationRequest, which we discussed in previous sections.
This part of code is used to activate Your driver and it can be found in AppDelegate.swift.
Now You can launch Your app and install the driver. From the moment of activation, the system itself will manage driver's lifecycle.
But what if You dont have entitlements from Apple, but still want to install the driver? Visit the following, “debug” section.
If You try to install driver in a “safe mode” (with enabled SIP, should be usual state of Your machine) without entitlements, discussed previously, You will get a following error:
That's why, if You did not receive entitlements and just want to practice and debug Your driver (or system extension, it will work for them, too) You should enter a developer mode and disable SIP. To achieve this, perform following steps.
To enable developer mode in terminal enter the following command:
systemextensionsctl developer on
There might occur the following problem:
It means that You have enabled SIP (System Integrity Protection).
To disable it follow next steps (from article Disabling and Enabling System Integrity Protection):
(Although, before disabling the SIP, i would recommend to create a Time Machine and save in an external storage. I recommend it for security reasons –– You might forget to turn the SIP back on and Your machine will possibly be exposed to malicious code, etc.)
Firstly, enter the Recovery mode. In order to do it, press COMMAND and R just when turning on Your machine. After entering the Recovery mode, go to utilities and choose Terminal. In this Terminal run the following command:
csrutil disable
Now restart the machine, in order for changes to be performed.
Now after entering developer mode You should see the following message:
Now, running the app should not be a problem, and You will be able to debug the driver properly.
Important: Currently in my case a problem occurred with developer team, I am using a private team and I cannot build the driver even in the developer mode, because building driver requires Developer ID, which I don't have. With that said, following tutorial should work well in theory (it is based on the official tutorial for debugging), but it lacks practical examples. For details on getting the Developer ID for such tasks visit "Additionaly" section.
If You would like to run app from the terminal You can try following way (run from the directory, which contains the project, for example this one):
/usr/bin/xcodebuild -target HIDKeyboardApp -configuration Debug
If there is a problem with running x-code like this, You might try the following fix (it will use the Xcode app):
sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
To check list of the extensions (and hopefully see Your driver here) in terminal enter the following command:
systemextensionsctl list
In order to debug Your program (Debugging and Testing System Extensions)
use lldb. To obtain PID of You process use ps. Run lldb from the Terminal, and attach to the process process attach --pid.
Now You should be able to debug Your program.
After You are finished remember to exit the developer mode and enable SIP!
To disable developer mode simply run the following command:
systemextensionsctl developer off
System Integrity Protection is extremely important to ensure that any malicious code doesn't damage Your system, so enter the Recovery mode once again, enter the terminal and enable SIP by running the following command:
csrutil enable
Now restart the machine, in order for changes to be performed.
That is it, now You have both Your app and driver debugged and ready for further adventures.
To be able to build driver (and have more access in macOS development in general), You need to have a Developer ID. Without it You still can develop apps (using Your Apple ID You will have a personal development team), but not the ones, which work with System extensions.
To obtain the Developer ID certificate (more info here) You will need to enroll in the Apple Developer Program (or Apple Developer Enterprise Program). Here is the enrollment link, check which type of enrollment You need/prefer and follow the link in the bottom of the page. Bear in mind, that You still need an Apple ID for that, and the program costs US$99 (price as for January, 2022).
- "MAC OS X Internals: A Systems Approach" by Amit Singh (link)
- Modern Operating Systems, Andrew S. Tanenbaum (mostly chapter 5) (link)
- Creating a Driver Using the DriverKit SDK
- System Extensions and DriverKit video presentation
- Handling Keyboard Events from a Human Interface Device
- Installing System Extensions and Drivers
- DriverKit
- Implementing Drivers, System Extensions, and Kexts
- Introduction to I/O Kit Fundamentals
- Preparing the Development Team
- Non-macOS driver template example (Oracle)
- I/O Kit language choice
Special Thanks go to everyone on the Apple team who created instruments, discussed in this example, documentation for them and code samples, and made everything available on the web.