I am observing the same behavior using the trivial Dockerfile:

FROM hello-world:latest
RUN echo "hello"

My environment:

> nerdctl --version
nerdctl version 2.0.0-rc.0

> nerdctl info
Client:
 Namespace:     default
 Debug Mode:    false

Server:
 Server Version: v1.7.19
 Storage Driver: windows
 Logging Driver: json-file
 Cgroup Driver:
 Cgroup Version:
 Plugins:
  Log: fluentd journald json-file syslog
  Storage: windows-lcow windows
 Security Options:
 Kernel Version:
 Operating System:
 OSType: windows
 Architecture:
 CPUs: 0
 Total Memory: 0B
 Name: MSI
 ID: 8b411fec-0013-40ab-8dcf-140ae06c8ee0

> buildctl debug info
BuildKit: github.com/moby/buildkit v0.15.0-rc2 e83d79a51fb49aeb921d8a2348ae14a58701c98

@slonopotamus try running this with --progress=plain flag. There's a known issue with tty progress. See: #2587 (comment)

🎉 It works with --progress=plain, thanks for the workaround!

@iankingori Traced the problem to the I/O reader returned by StdoutPipe() in nerdctl build.goL78C11. Need to do further investigation if this is a nerdctl issue or a buildkit issue.

It was working in November 2023, but this problem appeared between then and February 2024, per #2587. I don't know if something changed in nerdctl that affected this, or if there was a change in buildkitd. Or maybe both, see the end of #2587 (comment)

But that should help give you somewhere to bisect with (although you'd need to backport the PR if you wanted to test nerdctl before that PR was merged. But you could rule old buildkit in or out easily enough.)

cc @profnandaa

Following the buildctl documentation on using output, I run the buildctl commands for each.

> buildctl --version

buildctl github.com/moby/buildkit v0.17.0 fd61877fa73693dcd4ef64c538f894ec216409a3

# Dockerfile

FROM hello-world:latest
RUN echo "hello"

1. Docker tarball

   > & "C:\Program Files\Buildkit\bin\buildctl.exe" `
       --addr=npipe:////./pipe/buildkitd `
       build `
       --progress=auto `
       --frontend=dockerfile.v0 `
       --local=context=. `
       --output=type=docker,name=docker.io/library/iis-server3:latest `
       --local=dockerfile=C:/Users/xxxxx `
       --opt=filename=Dockerfile
   
   error: invalid output option: output: output file is required for docker exporter. refusing to write to console

   Using nerdctl:

   >   _output/nerdctl --debug build --progress auto -t iis-server3 --output type=docker --file .\Dockerfile .

   

2. OCI tarball

   & "C:\Program Files\Buildkit\bin\buildctl.exe" `
       --addr=npipe:////./pipe/buildkitd `
       build `
       --progress=auto `
       --frontend=dockerfile.v0 `
       --local=context=. `
       --output=type=oci,dest=C:/Users/xxxxx/AppData/Local/Temp/output.tar,name=docker.io/library/iis-server2:latest `
       --local=dockerfile=C:/Users/xxxxx `
       --opt=filename=Dockerfile

   [+] Building 2.4s (7/7) FINISHED
   => [internal] load build definition from Dockerfile                                                                                        0.0s
   => => transferring dockerfile: 78B                                                                                                         0.0s
   => [internal] load metadata for docker.io/library/hello-world:latest                                                                       2.0s
   => [auth] library/hello-world:pull token for registry-1.docker.io                                                                          0.0s
   => [internal] load .dockerignore                                                                                                           0.0s
   => => transferring context: 122B                                                                                                           0.0s
   => [1/2] FROM docker.io/library/hello-world:latest@sha256:d211f485f2dd1dee407a80973c8f129f00d54604d2c90732e8e320e5038a0348                 0.1s
   => => resolve docker.io/library/hello-world:latest@sha256:d211f485f2dd1dee407a80973c8f129f00d54604d2c90732e8e320e5038a0348                 0.1s
   => CACHED [2/2] RUN echo "hello"                                                                                                           0.0s
   => ERROR exporting to oci image format                                                                                                     0.1s
   => => exporting layers                                                                                                                     0.0s
   => => exporting manifest sha256:312b0358bbc84f03b6c40493bba15d0eea0b586cf19c901c8c8e5731e8185daf                                           0.0s
   => => exporting config sha256:0ba713f119e7b225effacd8744c7ee9601c023586f39c71cc2044c3dbf994d57                                             0.0s
   => => sending tarball                                                                                                                      0.0s
   ------
   > exporting to oci image format:
   ------
   error: failed to solve: failed to get reader: content digest sha256:4a74766ac776b275376a07d5357fd928f8b871c9e3d409729ed7e1ff0c1e608c: not found

3. containerd image store

   & "C:\Program Files\Buildkit\bin\buildctl.exe" `
       --addr=npipe:////./pipe/buildkitd `
       build `
       --progress=auto `
       --frontend=dockerfile.v0 `
       --local=context=. `
       --output=type=image,name=docker.io/library/iis-server3:latest `
       --local=dockerfile=C:/Users/xxxxx `
       --opt=filename=Dockerfile

   [+] Building 2.3s (7/7) FINISHED
   => [internal] load build definition from Dockerfile                                                                                 0.0s
   => => transferring dockerfile: 78B                                                                                                  0.0s
   => [internal] load metadata for docker.io/library/hello-world:latest                                                                1.9s
   => [auth] library/hello-world:pull token for registry-1.docker.io                                                                   0.0s
   => [internal] load .dockerignore                                                                                                    0.0s
   => => transferring context: 122B                                                                                                    0.0s
   => [1/2] FROM docker.io/library/hello-world:latest@sha256:d211f485f2dd1dee407a80973c8f129f00d54604d2c90732e8e320e5038a0348          0.1s
   => => resolve docker.io/library/hello-world:latest@sha256:d211f485f2dd1dee407a80973c8f129f00d54604d2c90732e8e320e5038a0348          0.1s
   => CACHED [2/2] RUN echo "hello"                                                                                                    0.0s
   => exporting to image                                                                                                               0.0s
   => => exporting layers                                                                                                              0.0s
   => => exporting manifest sha256:967e0c72299274e2770111f97d1d05c9b139417e755d8464b9e336a55276b543                                    0.0s
   => => exporting config sha256:0ba713f119e7b225effacd8744c7ee9601c023586f39c71cc2044c3dbf994d57                                      0.0s
   => => naming to docker.io/library/iis-server3:latest
   

   > ctr --namespace=buildkit images ls
   
   REF                                  TYPE                                                 DIGEST                                                                  SIZE      PLATFORMS     LABELS
   docker.io/library/iis-server3:latest application/vnd.docker.distribution.manifest.v2+json sha256:967e0c72299274e2770111f97d1d05c9b139417e755d8464b9e336a55276b543 115.0 MiB windows/amd64 -
   

If I understand the last post correctly, the first case is the one that is happening underneath the failing nerdctl build command, is that right?

error: invalid output option: output: output file is required for docker exporter. refusing to write to console

This error comes from buildctl, but only happens if buildctl is directly connected to a terminal, and I expect the same failure on Linux, or with --progress=plain.

So try redirecting buildctl's stdout and stderr to separate files, which should emulate what nerdctl is doing. If that works, then buildctl is fine and the problem does lie in nerdctl's setup. (Based on the initial error, nerdctl closed the fd connected to buildkit's stdout too early? Weird that --progress=plain affects that though. I haven't actually checked the code though, so maybe that's not weird for current implementation reasons.)

error: failed to solve: failed to get reader: content digest sha256:4a74766ac776b275376a07d5357fd928f8b871c9e3d409729ed7e1ff0c1e608c: not found

That second case's failure looks like a bug in buildkit; a quick check shows that digest is the Windows nanoserver 10.0.20348.2762 base image, so it should be in the content store already. Might be worth raising as an issue in the buildkit project, and if possible, confirming whether we have the same problem on Linux. (I assume it's not fixed by --progress=plain)

This last one might be an issue with exporting "foreign"/"non-distributable" layers; that layer is not marked non-distributable anymore, but I recall there was a bug in buildkit or containerd where the "foreign"/"non-distibutable" flags were being conflated with the presence of URLs where the layer could be fetched by HTTP, and I suspect the Windows base images still have the URL metadata even though the layers are no longer flagged foreign/non-distributable... moby/buildkit#2561 (comment) is what I'm thinking of, and it looks like this conflation still exists. I didn't break that comment out into a separate bug report at the time, sadly. (There might also be an issue somewhere else in either buildkit or containerd that this issue is triggering; but we'd have to track that down via buildkit first anyway.)

Anyway, see if adding prefer-nondist-layers=true to the output options fixes that last one, or has any effect at all, even.

@TBBle Everything works fine on Linux. The problem is with Windows using PowerShell/ cmd.

Test 1 in #3629 (comment) also shows things working correctly on Windows. However, to test the failure case under nerctl build, stdout and stderr should be redirected to a file.

Seems like a buildkit issue. Filing a bug in buildkit

The bug's actually in https://github.com/containerd/console: containerd/console#83