Skip to content
Draft
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
ab7bcdb
examples/virtio-socket: checkpoint port over from virtio example and …
dreamliner787-9 Mar 30, 2025
1f80fb4
virtio/socket: add virtio-socket device implementation
dreamliner787-9 Apr 1, 2025
2a42530
examples/virtio-socket: add test programs and checkpoint everything w…
dreamliner787-9 Apr 1, 2025
b5d3634
examples/virtio-socket: add missing virtio socket device to dts
dreamliner787-9 Apr 1, 2025
f98ed06
virtio/socket: clean ups + docs
dreamliner787-9 Apr 1, 2025
4f3fe6d
examples/virtio-socket: add odroidc4 support and make sender sleep lo…
dreamliner787-9 Apr 1, 2025
114dd45
virtio/socket: integrate tooling
dreamliner787-9 Apr 4, 2025
d29f72c
examples/virtio-socket: fix dts node name and improve logging of exam…
dreamliner787-9 Apr 4, 2025
77ffd07
examples/virtio-socket: update docs
dreamliner787-9 Apr 4, 2025
2d34827
examples/virtio: download correct kernel
dreamliner787-9 Apr 4, 2025
c192a99
examples/virtio-socket: improve debug prints
dreamliner787-9 Apr 4, 2025
e7f359c
vmm.mk: make indentation consistent
dreamliner787-9 Apr 4, 2025
bdc6b65
examples/virtio: converge with examples/virtio-socket
dreamliner787-9 Apr 4, 2025
c85b778
examples/virtio: fix licences
dreamliner787-9 Apr 4, 2025
dd6c196
virtio/socket: style + trailing spaces
dreamliner787-9 Apr 4, 2025
5d24ad7
examples/virtio: style + trailing whitespaces
dreamliner787-9 Apr 4, 2025
6d549ea
examples/virtio: trailing whitespaces
dreamliner787-9 Apr 4, 2025
0ed5653
examples/virtio: fix readme trailing whitespace
dreamliner787-9 May 6, 2025
4155c85
virtio/socket: fix a spec non-compliance issue where a reset packet m…
dreamliner787-9 May 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions examples/virtio-socket/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
#
# Copyright 2024, UNSW
#
# SPDX-License-Identifier: BSD-2-Clause
#

BUILD_DIR ?= build
export MICROKIT_CONFIG ?= debug

ifeq ($(strip $(MICROKIT_SDK)),)
$(error MICROKIT_SDK must be specified)
endif
export override MICROKIT_SDK:=$(abspath $(MICROKIT_SDK))

# Both QEMU and Maaxboard uses the same guest device tree as the guest does not
# need to access any physical device. The only difference is that QEMU is GICv2 while
# the Maaxboard is GICv3. We apply a GIC overlay for the correct version on the board
# at build time.
# TODO: we should probably determine this better if we have lots of supported platforms.
ifeq ($(strip $(MICROKIT_BOARD)), maaxboard)
export UART_DRIVER := imx
export CPU := cortex-a55
export GIC_DT_OVERLAY := gic_v3_overlay.dts
else ifeq ($(strip $(MICROKIT_BOARD)), odroidc4)
export UART_DRIVER := meson
export CPU := cortex-a55
export GIC_DT_OVERLAY := gic_400_overlay.dts
else ifeq ($(strip $(MICROKIT_BOARD)), qemu_virt_aarch64)
export UART_DRIVER := arm
export CPU := cortex-a53
export GIC_DT_OVERLAY := gic_v2_overlay.dts
QEMU := qemu-system-aarch64
else
$(error Unsupported MICROKIT_BOARD given)
endif

# All platforms use the same Linux and initrd images.
export LINUX := 372b2c56f55f2d461e4570d79332298e0f7ecce4-linux
export INITRD := b6a276df6a0e39f76bc8950e975daa2888ad83df-rootfs.cpio.gz

export BUILD_DIR:=$(abspath $(BUILD_DIR))
export MICROKIT_SDK:=$(abspath $(MICROKIT_SDK))
export VIRTIO_EXAMPLE:=$(abspath .)

export OBJCOPY := llvm-objcopy
export TARGET := aarch64-none-elf
export CC := clang
export CC_USERLEVEL := zig cc
export LD := ld.lld
export AS := llvm-as
export AR := llvm-ar
export DTC := dtc
export RANLIB := llvm-ranlib
export MICROKIT_TOOL ?= $(MICROKIT_SDK)/bin/microkit
export SDDF=$(abspath ../../dep/sddf)
export LIBVMM=$(abspath ../../)

IMAGE_FILE := $(BUILD_DIR)/loader.img
REPORT_FILE := $(BUILD_DIR)/report.txt

all: $(IMAGE_FILE)

qemu $(IMAGE_FILE) $(REPORT_FILE) clean clobber: $(BUILD_DIR)/Makefile FORCE
$(MAKE) -C $(BUILD_DIR) MICROKIT_SDK=$(MICROKIT_SDK) $(notdir $@)

$(BUILD_DIR)/Makefile: virtio-socket.mk
mkdir -p $(BUILD_DIR)
cp virtio-socket.mk $@

FORCE:
140 changes: 140 additions & 0 deletions examples/virtio-socket/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
<!--
Copyright 2025, UNSW
SPDX-License-Identifier: CC-BY-SA-4.0
-->

# Using multiple virtIO devices with a Linux guest

This example demoes libvmm's VirtIO socket device implementation that allow
inter-guests communication without Ethernet or IP protocols.

We have two Linux VMs, a sender and receiver as follows:
```
--------- VM --------- --------- VM ---------
Userland: Userland:
vsock_recv.elf vsock_send.elf
^ |
| v
Kernel: Kernel:
virtio MMIO vsock virtio MMIO vsock
^ |
| v
--------- VMM -------- --------- VMM --------
virtio vsock device /-- virtio vsock device
^ /
| /
rx buffer <----------------/ rx buffer
```

In sender VM, `vsock_send.elf` will send 32k worth of data through virtIO socket
to `vsock_recv.elf` in the receiver VM. Then, the receiver VM will verify that the
data are all correct and both programs will quit.

Each virtIO socket device have a small receive buffer that the peer can write to
to send data. The buffer size is advertised to both the sender and receiver via the
`buf_alloc` field of the packet header for the guest driver to split up packets
as necessary.

Here is a demo of that process happening (receiver is red, sender is green):
![Demo boot log](docs/boot_log.png)

The example currently works on the following platforms:

* QEMU virt AArch64
* Avnet MaaXBoard
* HardKernel OdroidC4

### Metaprogram

Unlike the other examples, this one uses a metaprogram (`meta.py`) with
the [sdfgen](https://github.com/au-ts/microkit_sdf_gen) tooling to generate the
System Description File (SDF) and other necessary artefacts. Previously,
SDFs were written manually, along with C headers for sDDF-specific configurations,
but this approach was tedious and error-prone. Wit this tooling, we can describe
the system at a higher level, automating the generation of system-specific data.

## Dependencies

In addition to the dependencies outlined in the top-level README, the following
dependencies are needed:
* sdfgen (for generating the System Description File with a metaprogram).

### Linux

<!-- TODO bump sdfgen -->
On apt based Linux distributions run the following commands:
```sh
pip3 install sdfgen==0.23.1
```

If you get error: `externally-managed-environment` when installing via pip, instead run:
```sh
pip3 install --break-system-packages sdfgen==0.23.1
```

### macOS

On macOS, you can install the dependencies via Homebrew:
```sh
pip3 install sdfgen==0.23.1
```

If you get error: `externally-managed-environment` when installing via pip, instead run:
```sh
pip3 install --break-system-packages sdfgen==0.23.1
```

### Nix

The top-level `shell.nix` has everything necessary:
```sh
nix-shell ../../shell.nix
```

## Building

```sh
make MICROKIT_BOARD=<BOARD> MICROKIT_SDK=/path/to/sdk
```

Where `<BOARD>` is one of:
* `qemu_virt_aarch64`
* `maaxboard`
* `odroidc4`

Other configuration options can be passed to the Makefile such as `CONFIG`
and `BUILD_DIR`, see the Makefile for details.

By default the build system fetches the Linux kernel and initrd images from
Trustworthy Systems' website. To use your own images, specify `LINUX` and/or
`INITRD`. For example:

```sh
make MICROKIT_BOARD=qemu_virt_aarch64 MICROKIT_SDK=/path/to/sdk LINUX=/path/to/linux INITRD=/path/to/initrd
```

If you would like to simulate the QEMU board you can run the following command:
```sh
make MICROKIT_BOARD=qemu_virt_aarch64 MICROKIT_SDK=/path/to/sdk qemu
```

This will build the example code as well as run the QEMU command to simulate a
system running the whole system.

## Linux kernel notes
If you decide to BYO kernel, you will need to make sure these configuration
values were set when your kernel was built:
```
CONFIG_VSOCKETS=y
CONFIG_VSOCKETS_LOOPBACK=y
CONFIG_VIRTIO_VSOCKETS=y
CONFIG_VIRTIO_VSOCKETS_COMMON=y
```

It is recommended that you use at least Linux v6.13 as previous versions have an
issue where if you try to send more data than the underlying buffer can accomodate,
the kernel driver won't split up your data and the guests will hang forever waiting
for a transfer that will never take place to complete.

## Hardware set up
Since this example does not touch any real hardware, no setup is necessary.
48 changes: 48 additions & 0 deletions examples/virtio-socket/client_vm/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
<!--
Copyright 2025, UNSW
SPDX-License-Identifier: CC-BY-SA-4.0
-->

# Linux kernel image

## Linux kernel

### Details
* Image name: `linux`
* Config name: `linux_config`
* Git remote: https://github.com/torvalds/linux.git
* Tag: v6.13 (commit hash: `ffd294d346d185b70e28b1a28abe367bbfe53c04`)
* Toolchain: `aarch64-none-elf`
* Version: (Arm GNU Toolchain 12.2.Rel1 (Build arm-12.24)) 12.2.1 20221205

You can also get the Linux config used after booting by running the following
command in userspace: `zcat /proc/config.gz`. This is a kernel minimally configured with a
IPv4 network stack + DNS + DHCP and virtIO device drivers.

### Instructions for reproducing
```
git clone --depth 1 --branch v6.13 https://github.com/torvalds/linux.git
cp config linux/.config
make -C linux ARCH=arm64 CROSS_COMPILE=aarch64-none-elf- all -j$(nproc)
```

The path to the image is: `linux/arch/arm64/boot/Image`.

## Buildroot RootFS image

### Details
* Image name: `rootfs.cpio.gz`
* Config name: `buildroot_config`
* Version: 2022.08-rc2

### Instructions for reproducing

```
wget https://buildroot.org/downloads/buildroot-2022.08-rc2.tar.xz
tar xvf buildroot-2022.08-rc2.tar.xz
cp buildroot_config buildroot-2022.08-rc2/.config
make -C buildroot-2022.08-rc2
```

The root filesystem will be located at: `buildroot-2022.08-rc2/output/images/rootfs.cpio.gz` along
with the other buildroot artefacts.
Loading
Loading