docker: fix hadolint issues (#777)

Signed-off-by: Ric Li <[email protected]>

.dockerignore

@@ -0,0 +1,38 @@
+.dockerignore
+.git/
+.gitignore
+.github/
+.clang-format
+.vscode/
+.cache/
+
+CODEOWNERS
+LICENSE
+README.md
+doc
+
+*.log
+*.patch
+*.diff
+*.yuv
+*.pcapng
+*.pcap
+*.raw
+*.dat
+*.json
+!kahawai.json
+html/
+test.*
+build/
+core*
+out*
+assets/
+*.pkg
+*.jxs
+*.mkv
+*.zip
+*.gz
+*.tar
+mtl_system_status_*
+*.pyc
+*.mp4

.github/linters/.hadolint.yaml

@@ -0,0 +1,6 @@
+---
+##########################
+## Hadolint config file ##
+##########################
+ignored:
+  - DL3008  # Ignore pinned versions check for APT

.github/workflows/linter.yml

@@ -34,7 +34,6 @@ jobs:
           DEFAULT_BRANCH: main
           VALIDATE_CPP: false
           VALIDATE_JSCPD: false
-          VALIDATE_DOCKERFILE_HADOLINT: false
           VALIDATE_JSON: false
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           LOG_LEVEL: WARN

docker/README.md

@@ -2,79 +2,116 @@
 
 Docker guide for Intel® Media Transport Library.
 
-## 1. Build Docker image
+Please note that the Dockerfile provided is intended for development use only. It has been tested for functionality, but not for security. Users are advised to review and modify it as necessary before using it in a production environment.
+
+## 1. DPDK NIC PMD and env setup on host
+
+Follow [run guide](../doc/run.md) to setup the hugepages, driver of NIC PFs, vfio(2110) user group and vfio driver mode for VFs.
+
+### 1.1 Add group to access /dev/vfio/* devices
+
+This section guides you through creating a dedicated group, granting the appropriate permissions, and setting up udev rules to maintain these settings across system reboots and devices re-creations.
+
+Add a new group named `vfio` with GID `2110` to control the VFIO devices. If GID `2110` is in use, consider using a different one.
+
+```bash
+getent group 2110 || sudo groupadd -g 2110 vfio
+```
+
+Create or edit a udev rules file, for example, /etc/udev/rules.d/10-vfio.rules, with your preferred text editor. For instance, using vim:
+
+```bash
+sudo vim /etc/udev/rules.d/10-vfio.rules
+```
+
+Add the following line to set the group ownership to vfio and enable read/write access for the group to any VFIO devices that appear:
+
+```bash
+SUBSYSTEM=="vfio", GROUP="vfio", MODE="0660"
+```
+
+Then reload the udev rules with:
+
+```bash
+sudo udevadm control --reload-rules
+sudo udevadm trigger
+```
+
+## 2. Build Docker image
 
 ```bash
-docker build -t mtl:latest -f ubuntu.dockerfile ./
+docker build -t mtl:latest -f ubuntu.dockerfile ../
 ```
 
 Refer to below build command if you are in a proxy env.
 
 ```bash
 http_proxy=http://proxy.xxx.com:xxx
 https_proxy=https://proxy.xxx.com:xxx
-docker build -t mtl:latest -f ubuntu.dockerfile --build-arg HTTP_PROXY=$http_proxy --build-arg HTTPS_PROXY=$https_proxy ./
+docker build -t mtl:latest -f ubuntu.dockerfile --build-arg HTTP_PROXY=$http_proxy --build-arg HTTPS_PROXY=$https_proxy ../
 ```
 
-## 2. DPDK NIC PMD and env setup on host
+## 3. Run and login into the docker container
 
-Follow [run guide](../doc/run.md) to setup the hugepages, driver of NIC PF, vfio driver mode for VFs.
+### 3.1 Run MTL Manager
 
-## 3. Run and login into the docker container with root user
+Before running any IMTL container, please refer to [MTL Manager](../manager/README.md) to run the Manager daemon server.
 
-The sample usage provided below is enabled with specific privileged settings such as VFIO access, a shared IPC namespace and root user inside the docker.
+For legacy way of running multiple containers without MTL Manager, please add the following arguments to the docker run commands in below sections:
 
-### 3.1 Run the docker container
+```bash
+  -v /tmp/kahawai_lcore.lock:/tmp/kahawai_lcore.lock \
+  -v /dev/null:/dev/null \
+  --ipc=host \
+```
+
+### 3.2 Run the docker container
 
-#### 3.1.1 Run multiple docker container with SHM requirement
+#### 3.2.1 Run with docker command
+
+For DPDK PMD backend, pass the VFIO devices:
 
 ```bash
-touch /tmp/kahawai_lcore.lock
 docker run -it \
   --device /dev/vfio \
   --cap-add SYS_NICE \
   --cap-add IPC_LOCK \
-  --cap-add NET_ADMIN \
-  --cap-add SYS_TIME \
-  --cap-add NET_RAW \
-  -v /tmp/kahawai_lcore.lock:/tmp/kahawai_lcore.lock \
-  -v /dev/null:/dev/null \
-  --ipc=host \
+  -v /var/run/imtl:/var/run/imtl \
+  --ulimit memlock=-1 \
   mtl:latest
 ```
 
-Explanation of Docker arguments:
-
-| Argument | Description |
-| --- | --- |
-| `--device /dev/vfio` | Access the VFIO device |
-| `--cap-add SYS_NICE` | For set_mempolicy |
-| `--cap-add IPC_LOCK` | For DMA mapping |
-| `--cap-add NET_ADMIN` | Optional, for kernel NIC configuration |
-| `--cap-add SYS_TIME` | Optional, for systime adjustment |
-| `--cap-add NET_RAW` | Optional, for AF_XDP socket |
-| `-v /tmp/kahawai_lcore.lock:/tmp/kahawai_lcore.lock` | For multiple instances lcore management |
-| `-v /dev/null:/dev/null` | For multiple instances lcore management |
-| `--ipc=host` | For multiple instances lcore management |
-
-#### 3.1.2 Run single docker container
-
-If you confirm that all IMTL processes will run within a single Docker container, you can disregard the settings related to shared memory. Simply execute the following command:
+For AF_XDP backend, pass the host network interfaces:
 
 ```bash
 docker run -it \
+  --net host \
   --device /dev/vfio \
   --cap-add SYS_NICE \
-  --cap-add IPC_LOCK \
-  --cap-add NET_ADMIN \
-  --cap-add SYS_TIME \
   --cap-add NET_RAW \
+  --cap-add CAP_BPF \
+  -v /var/run/imtl:/var/run/imtl \
+  --ulimit memlock=-1 \
   mtl:latest
 ```
 
-#### 3.1.3 Specify NIC devices for container
+Explanation of `docker run` arguments:
+
+| Argument | Description |
+| --- | --- |
+| `--net host` | For AF_XDP backend to access NICs |
+| `-v /var/run/imtl:/var/run/imtl` | For connection with MTL Manager |
+| `--device /dev/vfio` | For DPDK eal to access the VFIO devices |
+| `--ulimit memlock=-1` | For DPDK PMD to do DMA remapping or AF_XDP backend to create UMEM |
+| `--cap-add SYS_NICE` | For DPDK eal to set NUMA memory policy |
+| `--cap-add IPC_LOCK` | For DPDK PMD to do DMA mapping |
+| `--cap-add NET_RAW` | For AF_XDP backend to create socket |
+| `--cap-add CAP_BPF` | For AF_XDP backend to update xsks_map |
+| `--cap-add SYS_TIME` | For systime adjustment if `--phc2sys` enabled |
+
+#### 3.2.2 Specify VFIO devices for container
 
-If you only need to pass specific NICs to the container, you can use the following command to list the IOMMU group:
+If you only need to pass specific PFs/VFs to the container, you can use the following command to list the IOMMU group:
 
 ```bash
 ../script/nicctl.sh list all
@@ -93,10 +130,12 @@ docker run -it \
   --device /dev/vfio/312 \
   --cap-add SYS_NICE \
   --cap-add IPC_LOCK \
+  -v /var/run/imtl:/var/run/imtl \
+  --ulimit memlock=-1 \
   mtl:latest
 ```
 
-### 3.1.4 Run with docker-compose
+#### 3.2.3 Run with docker-compose
 
 Edit the `docker-compose.yml` file to specify the configuration.
 
@@ -107,20 +146,11 @@ docker-compose run imtl
 # docker compose run imtl
 ```
 
-### 3.2 Switch to the root user inside a Docker container
-
-On the docker bash shell:
-
-```bash
-sudo -s
-```
-
 ## 4. Run RxTXApp
 
 ```bash
-cd Media-Transport-Library/
 # Run below command to generate a fake yuv file or follow "#### 3.3 Prepare source files:" in [run guide](../doc/run.md)
 # dd if=/dev/urandom of=test.yuv count=2160 bs=4800
-# Edit and Run the loop json file
-./build/app/RxTxApp --config_file tests/script/loop_json/1080p60_1v.json
+# Edit and Run the loop json file.
+./app/RxTxApp --config_file tests/script/loop_json/1080p60_1v.json
 ```

docker/ubuntu.dockerfile

@@ -1,58 +1,82 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2023 Intel Corporation
 
-FROM ubuntu@sha256:dfd64a3b4296d8c9b62aa3309984f8620b98d87e47492599ee20739e8eb54fbf
+# NOTE: This Dockerfile is intended for development purposes only.
+# It has been tested for functionality, but not for security.
+# Please review and modify as necessary before using in a production environment.
 
-LABEL maintainer="[email protected]"
+# Ubuntu 22.04, build stage
+FROM ubuntu@sha256:149d67e29f765f4db62aa52161009e99e389544e25a8f43c8c89d4a445a7ca37 AS builder
 
-ENV MTL_REPO=Media-Transport-Library
-ENV DPDK_REPO=dpdk
-ENV DPDK_VER=23.07
-ENV IMTL_USER=imtl
-
-RUN apt-get update -y
-
-# Install dependencies
-RUN apt-get install -y git gcc meson python3 python3-pip pkg-config libnuma-dev libjson-c-dev libpcap-dev libgtest-dev libsdl2-dev libsdl2-ttf-dev libssl-dev
-
-RUN pip install pyelftools ninja
-
-RUN apt-get install -y sudo
-
-# some misc tools
-RUN apt-get install -y vim htop
-
-RUN apt clean all
+LABEL maintainer="[email protected],[email protected]"
 
-# user: imtl
-RUN adduser $IMTL_USER
-RUN usermod -G sudo $IMTL_USER
-RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
-USER $IMTL_USER
+# Install build dependencies and debug tools
+RUN apt-get update -y && \
+    apt-get install -y --no-install-recommends git build-essential meson python3 python3-pyelftools pkg-config libnuma-dev libjson-c-dev libpcap-dev libgtest-dev libsdl2-dev libsdl2-ttf-dev libssl-dev ca-certificates && \
+    apt-get install -y --no-install-recommends m4 clang llvm zlib1g-dev libelf-dev libcap-ng-dev libcap2-bin gcc-multilib && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
 
-WORKDIR /home/$IMTL_USER/
-
-RUN git config --global user.email "[email protected]" && \
-    git config --global user.name "Your Name"
-
-RUN git clone https://github.com/OpenVisualCloud/$MTL_REPO.git
-
-RUN git clone https://github.com/DPDK/$DPDK_REPO.git && \
-    cd $DPDK_REPO && \
-    git checkout v$DPDK_VER && \
-    git switch -c v$DPDK_VER
-
-# build dpdk
-RUN cd $DPDK_REPO && \
-    git am ../Media-Transport-Library/patches/dpdk/$DPDK_VER/*.patch && \
-    meson build && \
-    ninja -C build && \
-    sudo ninja -C build install && \
-    cd ..
-
-# build mtl
-RUN cd $MTL_REPO && \
-    ./build.sh && \
-    cd ..
-
-CMD ["/bin/bash"]
+ENV MTL_REPO=Media-Transport-Library
+ENV DPDK_VER=23.07
+ENV PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/lib64/pkgconfig
+
+COPY . $MTL_REPO
+
+# Clone DPDK and xdp-tools repo
+RUN git clone https://github.com/DPDK/dpdk.git && \
+    git clone --recurse-submodules https://github.com/xdp-project/xdp-tools.git
+
+# Build DPDK with Media-Transport-Library patches
+WORKDIR /dpdk
+RUN git checkout v$DPDK_VER && \
+    git switch -c v$DPDK_VER && \
+    git config --global user.email "[email protected]" && \
+    git config --global user.name "Your Name" && \
+    git am ../$MTL_REPO/patches/dpdk/$DPDK_VER/*.patch && \
+    meson setup build && \
+    meson install -C build && \
+    DESTDIR=/install meson install -C build
+
+# Build the xdp-tools project
+WORKDIR /xdp-tools
+RUN ./configure && make &&\
+    make install && \
+    DESTDIR=/install make install
+WORKDIR /xdp-tools/lib/libbpf/src
+RUN make install && \
+    DESTDIR=/install make install
+
+# Build IMTL
+WORKDIR /$MTL_REPO
+RUN ./build.sh && \
+    DESTDIR=/install meson install -C build && \
+    setcap 'cap_net_raw+ep' ./build/app/RxTxApp
+
+# Ubuntu 22.04, runtime stage
+FROM ubuntu@sha256:149d67e29f765f4db62aa52161009e99e389544e25a8f43c8c89d4a445a7ca37 AS final
+
+LABEL maintainer="[email protected],[email protected]"
+
+# Install runtime dependencies
+RUN apt-get update -y && \
+    apt-get install -y --no-install-recommends libnuma1 libjson-c5 libpcap0.8 libsdl2-2.0-0 libsdl2-ttf-2.0-0 libssl3 zlib1g libelf1 libcap-ng0 libatomic1 && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+# Add user: imtl(1001) with group vfio(2110)
+RUN groupadd -g 2110 vfio && \
+    useradd -m -G vfio -u 1001 imtl
+
+# Copy libraries and binaries
+COPY --chown=imtl --from=builder /install /
+COPY --chown=imtl --from=builder /Media-Transport-Library/build /home/imtl
+
+WORKDIR /home/imtl/
+
+# ldconfig
+RUN ldconfig
+
+USER imtl
+
+CMD ["/bin/bash"]