Variables are being expanded incorrectly in build logs

[flouthoc]

Contributing guidelines and issue reporting guide

• I've read the contributing guidelines and wholeheartedly agree. I've also read the issue reporting guide.

Well-formed report checklist

• I have found a bug that the documentation does not mention anything about my problem
• I have found a bug that there are no open or closed issues that are related to my problem
• I have provided version/information about my environment and done my best to provide a reproducer

Description of bug

Bug description

In build logs RUN instruction expands variable from build-args and does not shows the actual value of variable which was used.

Consider instruction like

ARG CMD=pwd
RUN CMD=ls; echo $CMD

In build logs it prints

#5 [2/2] RUN CMD=ls; echo pwd

But actual command which was executed was echo ls

Consider

Reproduction

Steps to reproduce the problem. If you are using docker build or docker buildx build providing the command as well as any input files will help analysis.

FROM busybox
ARG CMD=pwd
RUN CMD=ls; echo $CMD

docker buildx build --progress=plain --no-cache -t test .
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 144B done
#1 DONE 0.1s

#2 [internal] load metadata for docker.io/library/busybox:latest
#2 DONE 2.2s

#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.1s

#4 [1/2] FROM docker.io/library/busybox:latest@sha256:ab33eacc8251e3807b85bb6dba570e4698c3998eca6f0fc2ccb60575a563ea74
#4 CACHED

#5 [2/2] RUN CMD=ls; echo pwd
#5 0.578 ls
#5 DONE 1.1s

#6 exporting to image
#6 exporting layers
#6 exporting layers 0.2s done
#6 writing image sha256:2d22600e9e7a4d4ffaf2bfc588df7ff44a78d21299757e079c2e81727ec01545
#6 writing image sha256:2d22600e9e7a4d4ffaf2bfc588df7ff44a78d21299757e079c2e81727ec01545 0.0s done
#6 naming to docker.io/library/test 0.0s done
#6 DONE 0.4s

Version information

Client: Docker Engine - Community
 Version:           27.3.1
 API version:       1.47
 Go version:        go1.22.7
 Git commit:        ce12230
 Built:             Fri Sep 20 11:42:53 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.3.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.7
  Git commit:       41ca978
  Built:            Fri Sep 20 11:41:10 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.23
  GitCommit:        57f17b0a6295a39009d861b89e3b3b87b005ca27
 runc:
  Version:          1.1.14
  GitCommit:        v1.1.14-0-g2c9f560
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[tonistiigi]

There is another issue describing this that I couldn't instantly find.

In reality, variables are not expanded in RUN and instead are figured out by your shell. But that makes poor progress output if configuration matrices are used.

We could do some basic shell parsing to detect this example case. For more complicated cases, it would not be precise, though, and also technically, the RUN statement does not need to run in the actual shell at all. It could be SHELL ["/usr/bin/python"] for example.

[flouthoc]

@tonistiigi Thanks for your response.

One confusion which I have is that build logs reflect expansion on the bases of build-args or ARG VAR=value if it is present but when actual run happens it expanded by the shell. Do you have any thoughts on what should build log reflect to end users ?

[thaJeztah]

There is another issue describing this that I couldn't instantly find.

@tonistiigi maybe my ticket?

• Bug: progress output substitutes environment variables where it should not #2415