Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build fails on multi-arch images with "docker exporter does not currently support exporting manifest lists" #85

Open
pjmpsu opened this issue Mar 2, 2022 · 11 comments

Comments

@pjmpsu
Copy link

pjmpsu commented Mar 2, 2022

I am able to successfully build linux/arm or linux/amd64 images. When attempting to pass in multiple params to IMAGE_PLATFORM like so IMAGE_PLATFORM: linux/amd64,linux/arm64. The result is as follows:

#23 exporting to oci image format
#23 ERROR: docker exporter does not currently support exporting manifest lists
------
 > exporting to oci image format:
------
error: failed to solve: docker exporter does not currently support exporting manifest lists
FATA[0046] failed to build: build: exit status 1        
FATA[0046] failed to run task: exit status 1
@cheeseball212
Copy link

This is a great idea! Adding multi arch support would be very beneficial and streamline processes. We have employees using new mac systems with Apple's M1 processor that have reported their docker containers no longer work. We have since been looking for a way to support multiple platforms in the same image and this could certainly help in achieving that.

@joaosa
Copy link

joaosa commented Apr 4, 2022

I am also very interested in this, but it seems like an upstream issue. Look here and here

@holgerstolzenberg
Copy link

holgerstolzenberg commented May 3, 2022

This is indeed something that becomes more and more necessary. With the emerging ARM technology introduce by Apple M1 chipset, more and more of our developers are in need of a multi-arch build for local development. Using hard coded architecture bound tags my-image:1.2.3-linux-amd64 ist not feasible, as this changes from dev env to dev env.

Currently we are using a variant of the docker-image resource (ljfranklin/docker-buildx-resource) that supports buildx, but like the docker-image resource this one should also be regarded as deprecated and everyone is encouraged to migrate to oci-build-task and registry-image resource. The resource itself is not maintained that well (no offence here).

Also - we lastly experienced some problems related to the lists.manifest.json in Artifactory using this resource, but could not clarify upon now if it is related to the resource or Artifactory (being our artifact storage system).

We really would want to migrate to oci-build-task and registry-image resource for various reasons:

  • documentation
  • stability
  • performance
  • caching
  • maintenance
  • being official Concourse resources

...but not being able to do multi-arch builds is a showstopper for us.

As already mentioned, the build already runs for multiple architectures, it is only a matter of the exporter to support it.

Would be great to see that coming soon. ✌️

@dackroyd
Copy link
Contributor

Got further with this by running a custom build of the oci-build-task, where its --output is set to "type=oci,dest="+imagePath. The image builds successfully with buildctl, however the final steps of the oci-build-task fail because it requires a manifest.json to be present in the tarball, which the oci format does not include.

Having then bypassed that section as well, and attempting to use the resulting tarball with the registry-image-resource, the same manifest.json failure pops up. Managed to dig up an old open issue from there pointing to the problem: concourse/registry-image-resource#49 (comment)

[...] would it be accurate to say the current oci format (supported by in), while having Docker-specific manifest.json in it, is still valid OCI at least? i.e. it's a superset?

I suspect that being able to properly produce the oci output via this oci-build-task, and the registry-image-resource also supporting the oci format could resolve the issue of multi-arch support?

@dackroyd
Copy link
Contributor

I believe this is resolved by #93 and concourse/registry-image-resource#321. The later appears to have made it into registry-image-resource release v1.6.0: concourse/registry-image-resource@v1.5.1...v1.6.0, while oci-build-task has not yet cut a release since the corresponding change went in.

@brunellie
Copy link

brunellie commented Oct 9, 2023

To fix this just add OUTPUT_OCI: true in the task params.

@scottwsides
Copy link

To fix this just add OUTPUT_OCI: true in the task params.

Could you be more specific? Where are these 'task params'? What does your final docker buildx build.... command look like

@meezaan
Copy link

meezaan commented May 3, 2024

@brunellie this worked for me.

@scottwsides see screenshot.

Screenshot 2024-05-03 at 10 39 47 AM

@kolserdav
Copy link

I use --output type=image instead of --output type=docker and it works for me:

docker buildx build --platform="linux/arm64,linux/amd64"  --output="type=image"

@brunellie
Copy link

@scottwsides This is an issue on the Concourse oci-build-task. I was mentioning the params field in the generic Concourse task configuration. I'm using the task to build multi-arch so there's no command to show

@baritarek
Copy link

baritarek commented Jun 28, 2024

I am using the M3 and this command worked for me
docker buildx build --platform linux/amd64,linux/arm64 -t <dockerimage>:multi-arch --output type=image,dest=./<dockerimage>.tar .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants