NEW: Prototype Docker image for Windows

[carterbox]

Having a Docker image for the Windows build environment is a first step toward reproducible builds offline using the build-locally.py script.

This PR adds a windows docker image based on the one first posted by @xhochy.

The image starts with the Windows Server Core LTS 2019 image (which is cached on azure-pipelines) then adds conda and some Visual Studio build tools.

The base image is approx 5.7 GB, the conda install is 1.1 GB, and Visual Studio is 4 GB (with components below).

One nice things about the VS installer used in this image is that it is configured using a file that lists exactly which components are included. I think this is key to easily reducing the VS components to the bare minimum for conda-forge. Unfortunately, I am not familiar enough with the conda compiler toolchain to know which components are already available as conda-packages and which must be included in the image.

Related

conda-forge/conda-forge.github.io#928
https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B#Internal_version_numbering

Support/Lifecycle dates for visual studio releases

https://docs.microsoft.com/en-us/lifecycle/products/visual-studio-2019
https://docs.microsoft.com/en-us/lifecycle/products/visual-studio-2017

Download links for archived versions of visual studio installer

You can switch between visual studio years by clicking on the version dropdown in the upper left menu.
https://docs.microsoft.com/en-us/visualstudio/install/use-command-line-parameters-to-install-visual-studio?view=vs-2022
https://docs.microsoft.com/en-us/visualstudio/install/use-command-line-parameters-to-install-visual-studio?view=vs-2017

Description of workloads and components

These components workloads change from year release to year release and are not available before the 2017 installer. This docker image tries to install the minimum required for the C++ workload. Switch between year releases in the upper left menu.
https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2017

[carterbox]

Conda-forge related feedstocks

https://github.com/conda-forge/vc-feedstock/blob/main/recipe/conda_build_config.yaml
https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml

[carterbox]

[carterbox]

I tried building the h5py-feedstock in this container, but at the very end of the build after overlink checking I get the following error:

INFO :: Time taken to mark (prefix)
        0 replacements in 0 files was 0.06 seconds
Traceback (most recent call last):
  File "C:\Users\ContainerAdministrator\miniconda3\Scripts\conda-build-script.py", line 10, in <module>
    sys.exit(main())
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_build\cli\main_build.py", line 488, in main
    execute(sys.argv[1:])
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_build\cli\main_build.py", line 477, in execute
    outputs = api.build(args.recipe, post=args.post, test_run_post=args.test_run_post,
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_build\api.py", line 186, in build
    return build_tree(
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_build\build.py", line 3088, in build_tree
    packages_from_this = build(metadata, stats,
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_build\build.py", line 2371, in build
    newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_build\build.py", line 1692, in bundle_conda
    conda_package_handling.api.create(metadata.config.host_prefix, files,
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_package_handling\api.py", line 88, in create
    raise err
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_package_handling\api.py", line 82, in create
    out = SUPPORTED_EXTENSIONS[ext].create(prefix, file_list, out_fn, out_folder, **kw)
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_package_handling\tarball.py", line 160, in create
    out_file = create_compressed_tarball(prefix, file_list, out_folder,
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_package_handling\tarball.py", line 88, in create_compressed_tar
ball
    _create_libarchive(fullpath, files, compression_filter, filter_opts)
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_package_handling\tarball.py", line 74, in _create_libarchive
    archive_utils.create_archive(fullpath, files, compression_filter, filter_opts)
  File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\conda_package_handling\archive_utils.py", line 29, in create_archive
    raise ArchiveCreationError(message)
conda_package_handling.exceptions.ArchiveCreationError: Failed to clean up compressor

so the good news is that I seem to have the correct compiler tooling installed, but something isn't as expected for packaging.

[jakirkham]

cc @xhochy (in case this is of interest 🙂)

[carterbox]

Two more things that need to be addressed:

• What is the legality of creating a docker container with Visual Studio / MSVC pre-installed?

I believe that the general license for the Visual Studio build tools does not allow public redistribution.

You may not [...] share, publish, rent or lease the software, or provide the software as a stand-alone hosted as solution for others to use, or transfer the software or this agreement to any third party.

However, it does allow installing onto a container for internal/private use. So we could make this container then only distribute it to conda-forge members via private github/azure docker container registry. That's probably not what we want though, so special license from Microsoft is probably wanted.

• Is the image small enough? Could possibly use server-core or nanoserver image, but that would require installing visual studio / conda to a folder on the image building host then using ADD instead of installing inside the container Cannot use nanoserver because it only supports 64bit applications and conda depends on some win32 components.

[jaimergp]

The cleanup error seems to come from conda_package_handling and its reliance on libarchive. Maybe it's just a permissions error on some directories of the mounted volumes?

[carterbox]

OK. With conda_package_handling >=2, there is no longer an archive creation error. I built h5py and libavif successfully inside this container (with tar archives; not the new conda format).

[carterbox]

Is the image small enough? Could possibly use server-core or nanoserver image, but that would require installing visual studio / conda to a folder on the image building host then using ADD instead of installing inside the container Cannot use nanoserver because it only supports 64bit applications and conda depends on some win32 components.

Maybe nanoserver is an option again because Anaconda has dropped support for win32?
https://www.anaconda.com/blog/new-release-anaconda-distribution-now-supporting-m1