Add wanda ray image builds for Docker Hub

[andrew-anyscale]

This adds wanda-based builds for ray Docker Hub images.

Changes:

• Add ray-image-cpu/cuda.wanda.yaml for ray images
• Add ray-extra-image-cpu/cuda.wanda.yaml for extra dependencies
• Add ray-llm-image-cuda.wanda.yaml and ray-llm-extra variant
• Add push_ray_image.py for pushing to Docker Hub via crane
• Update build.rayci.yml with wanda image build and push steps

Topic: ray-image
Relative: ray-wheel
Labels: draft

[andrew-anyscale]

Reviews in this chain:
└#59935 Migrate wheel builds to wanda (x86_64)
　└#59969 Add wanda cpp wheel builds (x86_64)
　　└#59936 Add wanda ray image builds for Docker Hub
　　　└#59937 Add wanda anyscale image builds for release tests

[andrew-anyscale]

#	head	base	diff	date	summary
0	02a8921d	f104bf59	diff	Jan 7 7:57 AM	9 files changed, 593 insertions(+), 49 deletions(-)
1	fe687866	288cadb6	diff	Jan 7 7:58 AM	0 files changed
2	46cc8625	288cadb6	diff	Jan 7 8:38 AM	4 files changed, 440 insertions(+), 31 deletions(-)
3	70df7561	164e2e4c	rebase	Jan 7 11:29 AM	0 files changed
4	ae32d45a	b4fb3e06	rebase	Jan 7 12:21 PM	0 files changed
5	04cfd26a	cd792539	rebase	Jan 7 17:29 PM	0 files changed
6	55d1473f	7d257e47	rebase	Jan 7 17:31 PM	0 files changed
7	84d74797	e93a821d	diff	Jan 8 8:27 AM	1 file changed, 6 insertions(+), 14 deletions(-)
8	c275fe24	8a4bdf5b	rebase	Jan 8 8:32 AM	0 files changed
9	7c78fa2d	a0b03e94	rebase	Jan 8 12:01 PM	0 files changed
10	29cf5abd	32347fa5	rebase	Jan 8 12:51 PM	0 files changed
11	d566c8a8	8d1057a9	diff	Jan 8 13:16 PM	0 files changed
12	91900297	d8901f73	rebase	Jan 8 13:33 PM	0 files changed
13	73822f53	9e438258	rebase	Jan 8 14:30 PM	0 files changed
14	94805e95	6801f72f	rebase	Jan 9 8:45 AM	0 files changed
15	fc79d67d	9834ea38	rebase	Jan 9 10:01 AM	0 files changed
16	db94f935	25e0d03c	rebase	Jan 9 10:59 AM	0 files changed
17	05c06707	2a361721	rebase	Jan 9 12:36 PM	0 files changed
18	a6718508	b81d4cb4	rebase	Jan 9 13:32 PM	0 files changed

[gemini-code-assist]

Code Review

This pull request successfully migrates the Ray Docker image builds to use wanda, which simplifies the CI configuration. The new wanda.yaml files and the shared Dockerfile are well-structured. However, I've identified a few areas for improvement, primarily in the new push_ray_image.py script. There's a critical issue where the script only supports pushing the main ray images, which is a regression from the previous system that also handled ray-extra and ray-llm variants. Additionally, the script duplicates significant portions of the image tagging logic from another file, which will create maintenance challenges. I've also included a suggestion to reduce duplication in the .buildkite/build.rayci.yml file using YAML anchors.

[gemini-code-assist]

The logic for generating image tags in _generate_image_tags and its helper functions (_format_python_version_tag, _format_platform_tag) duplicates logic that already exists in ci/ray_ci/docker_container.py.

This duplication makes the codebase harder to maintain. If the tagging scheme ever changes, it will need to be updated in two separate places, which is error-prone.

I recommend refactoring the tag generation logic from ci/ray_ci/docker_container.py into a shared utility module that both this script and docker_container.py can import. This will create a single source of truth for image tagging.

[andrew-anyscale]

Since ci/ray_ci/docker_container.py is being deprecated by this flow, I just ported the logic over here instead of refactoring to a single shared library. Attempting to untangle them was a bit messy, but I can do so if it would make sense for consistency

[gemini-code-assist]

This matrix definition is repeated for ray_images_cuda_push (lines 246-261) and ray-extra-image-cuda-build (lines 290-305). To improve maintainability and reduce duplication, you can use YAML anchors to define these lists once and reuse them.

You can define the anchors here, and then use *python_versions and *cuda_versions in the other matrix definitions.

For example, in the ray_images_cuda_push step, you would change the matrix to:

    matrix:
      setup:
        python: *python_versions
        cuda: *cuda_versions

    matrix:
      setup:
        python: &python_versions
          - "3.10"
          - "3.11"
          - "3.12"
        cuda: &cuda_versions
          - "11.7.1-cudnn8"
          - "11.8.0-cudnn8"
          - "12.1.1-cudnn8"
          - "12.3.2-cudnn9"
          - "12.4.1-cudnn"
          - "12.5.1-cudnn"
          - "12.6.3-cudnn"
          - "12.8.1-cudnn"
          - "12.9.1-cudnn"

[gemini-code-assist]

The GPU_PLATFORM constant is hardcoded here and is also defined in ci/ray_ci/docker_container.py. Duplicating constants can lead to inconsistencies if one is updated and the other is not.

This constant should be defined in a single, shared location and imported where needed. This could be done as part of the larger refactoring of the tag generation logic I mentioned in another comment.