Cannot Run Nydus Images With Docker

[IAmOWh]

Additional Information

The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all.

Version of nydus being used (nydusd --version)

Version: v2.2.5
Git Commit: 4db9fd7
Build Time: 2024-04-25T09:15:09.820565003Z
Profile: release
Rustc: rustc 1.70.0 (90c541806 2023-05-31)

Version of nydus-snapshotter being used (containerd-nydus-grpc --version)

Version: v0.13.12
Revision: ce4848e4439b79723b0635e994b5d9991aa88819
Go version: go1.22.1
Build time: 2024-04-29T20:01:11

Kernel information (uname -r)

command result: uname -r
5.15.0-105-generic

GNU/Linux Distribution, if applicable (cat /etc/os-release)

command result: cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

containerd-nydus-grpc command line used, if applicable (ps aux | grep containerd-nydus-grpc)

root       15802  0.0  0.9 1861748 36788 ?       Ssl  20:02   0:00 /usr/local/bin/containerd-nydus-grpc --config /etc/nydus/config.toml
billyou    25210  0.0  0.0  12120  2872 pts/0    S+   21:10   0:00 grep --color=auto containerd-nydus-grp

client command line used, if applicable (such as: nerdctl, docker, kubectl, ctr)

docker

Screenshots (if applicable)

Details about issue

I followed docker-env-setup.md but had trouble running nydus images(the last step). The convertion was successful and I could create a container based on that nydus image. So what was wrong? Did I miss something that is in other docs?

[IAmOWh]

I can run nydus Images with nerdctl instead of docker.

$ sudo nydusify convert --source ubuntu --target localhost:5000/ubuntu-nydus
$ sudo nerdctl --snapshotter nydus pull localhost:5000/ubuntu-nydus:latest
$ sudo nerdctl --snapshotter nydus run --rm -it localhost:5000/ubuntu-nydus:latest bash

Check storage driver and plugins.

$ docker info
...
Server:
...
 Storage Driver: nydus
  driver-type: io.containerd.snapshotter.v1
...
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog

$ sudo nerdctl info
...
Server:
 Server Version: v1.7.16
 Storage Driver: overlayfs
...
 Plugins:
  Log: fluentd journald json-file syslog
  Storage: aufs native overlayfs nydus
...

[imeoer]

Hi, are you using the docker version specified in the doc?

[IAmOWh]

I am using docker v26.1.0.Is it too new?

$ docker version
Client: Docker Engine - Community
 Version:           26.1.0
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        9714adc
 Built:             Mon Apr 22 17:07:12 2024
 OS/Arch:           linux/amd64
 Context:           default

Server:
 Engine:
  Version:          dev
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       48d769bf2f3bdf18af355d7f319b81009da39736
  Built:            Mon Apr 29 03:35:43 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.7.16
  GitCommit:        83031836b2cf55637d7abf847b17134c51b38e53
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e946
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[imeoer]

@adamqqqplay PTAL, maybe it doesn't work for the latest docker?

[adamqqqplay]

@IAmOWh Hi, have you modified this /etc/docker/daemon.json configuration?

{
  "features": {
    "containerd-snapshotter": true
  },
  "storage-driver": "nydus"
}

[adamqqqplay]

@adamqqqplay PTAL, maybe it doesn't work for the latest docker?

I will try to reproduce this issue.

[IAmOWh]

@IAmOWh Hi, have you modified this /etc/docker/daemon.json configuration?

{
  "features": {
    "containerd-snapshotter": true
  },
  "storage-driver": "nydus"
}

Hi. Yes, my /etc/docker/daemon.json is the same as this one.

[guquanheng]

Have you solved it? I have the same problem.

[beyou923]

Does only v24 docker support Nydus images? I tested other versions of Docker and encountered the same problem. @adamqqqplay

[beyou923]

It seems that this commit in Docker 25 (moby/moby#46502) caused this problem. I removed the init-layer related logic in Docker makes it work. nydus may need to be compatible with init-layer.

[imeoer]

It seems that this commit in Docker 25 (moby/moby#46502) caused this problem. I removed the init-layer related logic in Docker makes it work. nydus may need to be compatible with init-layer.

@beyou923 Thanks for the useful information! are you interested in fixing this?

[IAmOWh]

It seems that this commit in Docker 25 (moby/moby#46502) caused this problem. I removed the init-layer related logic in Docker makes it work. nydus may need to be compatible with init-layer.

Thank you for your answer, I'll try docker v24.