docker inside firecracker VM

[ashwin31]

Feature Request

I am building dev spaces where developers can code fullstack applications. MicroVM is what I need because of the nature of the workload.

Describe the desired solution

Ability to run docker containers inside MicroVM to support running fullstack applications.

Checks

• Have you searched the Firecracker Issues database for similar requests?
• Have you read all the existing relevant Firecracker documentation?
• Have you read and understood Firecracker's core tenets?

[Manciukic]

Hey,

thanks for opening the issue.
Could I ask you to provide more details in your setup and how to reproduce the issue?

I tried using the CI artifacts and indeed Docker is not working because our microvm kernel and rootfs are stripped down and don't have some of the requirements of Docker. For instance, modules support is disabled in the kernel, and netfilter is not built into it (which is what Docker was complaining about when I tried to run it).

Sep 13 11:31:56 ubuntu-fc-uvm dockerd[8050]: time="2024-09-13T11:31:56.380274421Z" level=info msg="Starting up"
Sep 13 11:31:56 ubuntu-fc-uvm dockerd[8050]: time="2024-09-13T11:31:56.381945081Z" level=warning msg="unable to modify root key limit, number of containers could be limited by this quota: open /proc/sys/kernel/keys/root_maxkeys: no such file or directory"
Sep 13 11:31:56 ubuntu-fc-uvm dockerd[8050]: time="2024-09-13T11:31:56.434845036Z" level=info msg="[graphdriver] using prior storage driver: overlay2"
Sep 13 11:31:56 ubuntu-fc-uvm dockerd[8050]: time="2024-09-13T11:31:56.435134042Z" level=info msg="Loading containers: start."
Sep 13 11:31:56 ubuntu-fc-uvm dockerd[8050]: time="2024-09-13T11:31:56.437903597Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: modprobe: WARNING: Module bridge not found in directory /lib/modules/5.10.209\nmodprobe: WARNING: Module br_netfilter not found in directory /lib/modules/5.10.209\n, error: exit status 1"
Sep 13 11:31:56 ubuntu-fc-uvm dockerd[8050]: time="2024-09-13T11:31:56.440777465Z" level=info msg="unable to detect if iptables supports xlock: 'iptables --wait -L -n': `iptables/1.8.7 Failed to initialize nft: Protocol not supported`" error="exit status 1"
Sep 13 11:31:56 ubuntu-fc-uvm dockerd[8050]: time="2024-09-13T11:31:56.457668100Z" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
Sep 13 11:31:56 ubuntu-fc-uvm dockerd[8050]: failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to register "bridge" driver: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: iptables/1.8.7 Failed to initialize nft: Protocol not supported
Sep 13 11:31:56 ubuntu-fc-uvm dockerd[8050]:  (exit status 1)

However, you should be able to run docker if you're using a kernel that supports all requirements for Docker. For example, you could try pulling the Ubuntu kernel and trying that out. I haven't checked it right now (I'll see if I have time in the next days) but I have done it in the past without issues.

Thanks,
Riccardo

[Manciukic]

Actually, just using iptables-legacy is enough on the latest CI artifacts (I believe since we updated the configs for the 1.9 release). I just had to resize ubuntu-22.04.ext4 to make enough space.

mkdir -p /var/lib/dpkg/
touch /var/lib/dpkg/status

apt update -y
DEBIAN_FRONTEND=noninteractive apt install -y apt-transport-https ca-certificates curl software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" > /etc/apt/sources.list.d/docker.list

apt update -y
apt-cache policy docker-ce
DEBIAN_FRONTEND=noninteractive apt install -y docker-ce

update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

systemctl restart docker
systemctl enable docker

docker run hello-world

Let me know if this completely answers your query or if you're seeing other issues in your setup.

[Manciukic]

Hey,
does this answer your question or do you have any further inquiries?
If there's no further activity on this issue I'm going to close it at the end of next week.
Thanks,
Riccardo