Very initial support for CY2023 / Rocky 8 containers #166
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a first attempt at building CY2023 compliant containers based on the nvidia/cuda:11.8.0-devel-rockylinux8 containers. There are currently a number of limitations which will be addressed later in this PR. - only built common and base so far - no attempt at releases or Conan-related functionality - no cudagl container for now, so likely no OpenGL - still some hardcoded references to centos7 in a few places - some centos7 packages no longer make sense / available (libdb4, openjpeg 1) - no need for special PATH for git-218 - haven't updated all the packages in the README.md - probably want a more elegant mechanism to determine the base distribution - not sure how to set the ABI to C++11 as required by CY2023 Draft only for now. Signed-off-by: Jean-Francois Panisset <[email protected]>
Qt 5.15.2 has issues building under Rocky 8 / gcc 11, Qt 5.15.8
open source is now available, move up to that.
- scripts/base/build_qt.sh
Starting with Qt 5.15.3, Qt tarball has been renamed qt-everywhere-opensource-src-...
- python/aswfdocker/tests/test_builder.py
- expect new symbols for the base container
- expect 3-clang14 as new version for common image
- python/aswfdocker/tests/test_migrater.py
- 5 valid years for openexr packages (2019-2023)
- ci-*/README.md
Updated by `aswfdocker dockergen` with new Qt
Signed-off-by: Jean-Francois Panisset <[email protected]>
- scripts/common/install_yumpackages.sh Qt 5.15.8 wants wayland-devel to build on Rocky 8 - python/aswfdocker/builder.py - python/aswfdocker/tests/test_builder.py Use generic ci-baseos-gl-conan labels Signed-off-by: Jean-Francois Panisset <[email protected]>
- python/aswfdocker/data/versions.yaml - ci-*/README.md PySide bumped to 5.15.8 to match Qt for CY2023 - packages/base[1-3]/Dockerfile Get rid of distro-specific label in intermediate containers Signed-off-by: Jean-Francois Panisset <[email protected]>
- .github/workflows/docker-builds.yml - .github/workflows/release.yml Bump Docker buildx plugin to much more recent v0.10.3 - packages/conan/recipes/ninja/conandata.yml Checkum for Ninja 1.11.1 Signed-off-by: Jean-Francois Panisset <[email protected]>
Signed-off-by: Jean-Francois Panisset <[email protected]>
This one was breaking the intermediate build of packages, causing Docker to try to look for a remote container. Signed-off-by: Jean-Francois Panisset <[email protected]>
aloysbaillet
reviewed
Feb 7, 2023
- packages/conan/settings/* Needed to define conan profiles for 2023 with correct gcc and ABI - packages/base3/Dockerfile Python is now a conan-only package, no more container to pull from. This doesn't quite work yet. - README.md Added links to 2023 containers Signed-off-by: Jean-Francois Panisset <[email protected]>
- In RHEL8 and derivatives, devtoolset now called gcc-toolset, deal with different installation location - rename symbols for base OS - document changes in CHANGELOG.md Signed-off-by: Jean-Francois Panisset <[email protected]>
- propagate ASWF_DTS_PREFIX everywhere to have correct PATH to gcc - fetch and apply KDE patch bundle to allow Qt 5.15.8 to build with gcc 11 Signed-off-by: Jean-Francois Panisset <[email protected]>
- scripts/base/build_pyside.sh - patch for PySide 5.15.8 to build against Python 3.10.x - patch for Pyside 5.15.8 to build against NumPy 1.23.x - packages/conan/recipes/python/conanfile.py sysconfigdata gets baked at compile time and surfaces the Conan build directory, which can confuse older CMake versions looking for INCLUDEPY. This replaces build prefix with /usr/local, which is not great since we don't know for sure that we'll be installing Python there, but it's better than the current situation. - packages/base3/Dockerfile You need to know the NUMPY_VERSION to build PySide - packages/vfx1/Dockerfile - packages/vfx2/Dockerfile tbb is now a Conan-only package Signed-off-by: Jean-Francois Panisset <[email protected]>
Closed
lgritz
reviewed
Mar 1, 2023
lgritz
reviewed
Mar 1, 2023
lgritz
reviewed
Mar 1, 2023
lgritz
reviewed
Mar 1, 2023
* CHANELOG.md
* README.md
* python/aswfdocker/data/versions.yaml
* python/aswfdocker/tests/test_builder.py
* ci-*/README.md
* OpenEXR 3.1.6, Imath 3.1.7
* HDF5 1.8.23
* OSL 1.12.10.0
* common and some vfx packages now have a clang 15.0(.7) build
* Pipfile
* Pipfile.lock
* One more instance of Python 3.9 removed
* python/aswfdocker/data/ci-image-dockerfile.jinja2
* ci-*/Dockerfile
* packages/conan/recipes/*/conanfile.py
* We need a deploy() method in conan recipes, need to call conan
install explicitly on a reference for it to get called
* Move to dockerfile:1.4 syntax
* packages/conan/recipes/python/conanfile.py
* a better workaround for the issue with temporary build paths
embeded in python sysconfig: edit at deploy() time when we know
the installation directory
* scripts/common/install_yumpackages.sh
* clean up non-ASCII characters
* packages/*/Dockerfile
* Move to dockerfile:1.4 syntax
* call `conan install` on references to make sure deploy() is called
* packages/conan/settings/profiles/vfx2023
* vfx2023 profile uses Python 3.10
Signed-off-by: Jean-Francois Panisset <[email protected]>
Had gotten pipenv environment in a weird state Signed-off-by: Jean-Francois Panisset <[email protected]>
Signed-off-by: Jean-Francois Panisset <[email protected]>
- OpenEXR to 3.1.7 - MaterialX to 1.38.7 - install_conanpackages.sh script needed to for execution of deploy() method in Conan recipes, needed to relocate some packages at install time Signed-off-by: Jean-Francois Panisset <[email protected]>
Need at least one entry in img-common-1 Signed-off-by: Jean-Francois Panisset <[email protected]>
Some -devel packages initially missing in Rocky 8 now seem to be available. Small build documentation fix. Signed-off-by: Jean-Francois Panisset <[email protected]>
QT 5.15.9 Open Source is now out, update to that, and get the build patch from persistent patch on linuxfromscratch Signed-off-by: Jean-Francois Panisset <[email protected]>
- bump PySide2 to 5.15.9 to match Qt - download shenanigans for PySide2, one less patch needed to build with gcc 11.2.1 - use bind mount to avoid having to copy local conan builds into containers Signed-off-by: Jean-Francois Panisset <[email protected]>
Build against latest USD 23.05 with patch to build against MaterialX 1.38.7 Signed-off-by: Jean-Francois Panisset <[email protected]>
- ci_common images are "year independent", test accordingly to use the new deployment menthod - update CONTRIBUTING.md to show current year Signed-off-by: Jean-Francois Panisset <[email protected]>
Signed-off-by: Jean-Francois Panisset <[email protected]>
conanfile.py for pybind11 and boost were calling for (latest) python conan package, that wasn't being found, use explicit ASWF_PYTHON_VERSION Instead. Signed-off-by: Jean-Francois Panisset <[email protected]>
install_opengl.sh adds the OpenGL runtime and development environment on top of the cuda base container to reproduce what the (no longer maintained) cudagl containers used to provide. Signed-off-by: Jean-Francois Panisset <[email protected]>
Signed-off-by: Jean-Francois Panisset <[email protected]>
Signed-off-by: Jean-Francois Panisset <[email protected]>
Signed-off-by: Jean-Francois Panisset <[email protected]>
If you try to run pyinstaller on an installed copy of Conan (via pip), you will miss some modules, for instance conan.tools.system and recipes will fail to build. Instead install Conan from source and use the provided pyinstaller.py to generate the standalone binary. Signed-off-by: Jean-Francois Panisset <[email protected]>
Signed-off-by: Jean-Francois Panisset <[email protected]>
- set the default config for Conan just before calling it to avoid breaking pre 2023 builds - update imath to 3.1.8 Signed-off-by: Jean-Francois Panisset <[email protected]>
Issue in upstream Python patch library used by Conan, tools.patch() silently ignores file renames in patch, have to do it manually in conanfile.py Signed-off-by: Jean-Francois Panisset <[email protected]>
- bump imath to 3.1.9 - Qt 5.15.9 should be built as C++17 to match VFX platform and avoid link error (template instantiation issue) - ugly workarounds for inclusion of qtsan_impl.h from KDE5 patch set Signed-off-by: Jean-Francois Panisset <[email protected]>
Use the build-in patch mechanism to build PySide as a Conan package for VFX 2023 Signed-off-by: Jean-Francois Panisset <[email protected]>
Signed-off-by: Jean-Francois Panisset <[email protected]>
Update to OpenEXR 3.1.8 Signed-off-by: Jean-Francois Panisset <[email protected]>
Reflect currently included versions Signed-off-by: Jean-Francois Panisset <[email protected]>
No such thing as Clang 14.1, we're using 14.0.6 Signed-off-by: Jean-Francois Panisset <[email protected]>
aloysbaillet
reviewed
Jun 8, 2023
| @@ -29,6 +29,8 @@ jinja2 = "*" | |||
| requests = "*" | |||
| importlib-resources = "*" | |||
| conan = "==1.47" | |||
| pytest = "*" | |||
There was a problem hiding this comment.
pytest and mypy are already listed in the dev-packages section, I don't think we need them here?
There was a problem hiding this comment.
I have to admit I'm not super clear on how Pipfile works, all I know is that at some point I ran pipenv lock which updated Pipfile and Pipfile.lock. I'll go back and revisit. I'm also not clear on how conan==1.47 in the (global?) Pipfile interacts with the VFX-year specific version (currently at 1.58).
aloysbaillet
approved these changes
Jun 8, 2023
lgritz
reviewed
Jun 8, 2023
| @@ -372,7 +372,7 @@ versions: | |||
| ASWF_OCIO_CONFIGS_VERSION: "1.0_r2" | |||
| ASWF_OCIO_VERSION: "2.2.1" | |||
| ASWF_OIIO_VERSION: "2.4.9.0" | |||
| ASWF_OPENEXR_VERSION: "3.1.7" | |||
| ASWF_OPENEXR_VERSION: "3.1.8" | |||
There was a problem hiding this comment.
Crap, never mind, I was thinking of imath. Apologies.
|
Issue #170 has follow up issues to be cleaned up after this gets merged. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a first attempt at building CY2023 compliant containers based on the nvidia/cuda:11.8.0-devel-rockylinux8 containers. There are currently a number of limitations which will be addressed later in this PR.
Interested in any feedback as to whether this is going in the right direction or not.
Draft only for now, will be updating to resolve some of the issues listed above.
Signed-off-by: Jean-Francois Panisset [email protected]