Unable to insert kernel modules while running kernel in a separate chroot

[comeillfoo]

I've encountered a broken symbolic link for /lib/modules/$kver if the --root points to a rootfs directory that is not the parent of the directory where the kernel sources are located.
E.g. kernel sources at directory linux and rootfs is sibling directory to sources:

.
|-- linux
`-- rootfs

So, when running vng --root ../rootfs being at linux I have modules at linux/.virtme_mods/lib/modules/0.0.0 and virtme_link_mods will be set to /../linux/.virtme_mods/lib/modules/0.0.0 because of os.path.relpath(kernel.moddir, args.root) (virtme/commands/run.py#L991).

Hence, virtme-init tries to symlink /lib/modules/$kver outside of the provided rootfs.

As a workaround I add a disk image (using --disk) with the missing directory, mount it to /tmp and manually relink /lib/modules/$kver to /tmp/my_disk/.virtme_mods/lib/modules/0.0.0. But are there other solutions besides this?

[jimc]

hey, it seems from your investigations, youre willing to hack some.

vng uses qemu to run the kernel in a vm,
the "--kdir ./" arg to it makes module-loading path relative to $PWD

If this isnt the cause itself, there could be other path-relative issues.

[comeillfoo]

hey, it seems from your investigations, youre willing to hack some.

vng uses qemu to run the kernel in a vm, the "--kdir ./" arg to it makes module-loading path relative to $PWD

If this isnt the cause itself, there could be other path-relative issues.

hi, in fact, I'm willing to test linux kernel on arm64. That's why I have to use not a root folder as rootfs.

I haven't checked it but it seems that varying --kdir won't work for me.

Because for example if kdir is ./linux and root is ./rootfs (via vng --arch arm64 --root ./rootfs --run ./linux ...) then after virtme/commands/run.py#L484 kernel.moddir would be set to ./linux/.virtme_mods/lib/modules/0.0.0. And os.path.relpath between kernel.moddir and args.root would return the same path outside of rootfs - ../linux/.virtme_mods/lib/modules/0.0.0.

[rbmarliere]

With #277, you can do something like:

vng --clean
vng --build
make INSTALL_MOD_PATH=$ROOTFS modules_install
vng --root $ROOTFS --run .

[comeillfoo]

With #277, you can do something like:

vng --clean
vng --build
make INSTALL_MOD_PATH=$ROOTFS modules_install
vng --root $ROOTFS --run .

Hello, thanks a lot. I think I can also pass modules with bind mount:

sudo mount --bind .virtme_mods/lib/modules/0.0.0 "${ROOTFS}/lib/modules/$(make -s kernelrelease)"

or mount kernel sources to guest as 9pfs via:

vng --root "${ROOTFS}" --qemu-opts="-virtfs local,path=.,mount_tag=sometag,security_model=none"
$ mkdir -p /tmp/kernel && mount -t 9p -o trans=virtio sometag /tmp/kernel -oversion=9p2000.L
$ ln -sf /tmp/kernel/.virtme_mods/lib/modules/0.0.0 /lib/modules/$(uname -r)