release: v1.9.0

[claytonparnell]

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[claytonparnell]

Staleness Report: 1.9.0(gpu)

Package	Current Version in the Distribution image	Latest Relevant Version in Upstream
numpy	1.26.4	1.26.4
jinja2	3.1.4	3.1.4
${\color{red}pandas}$	2.1.4	2.2.2
altair	5.3.0	5.3.0
${\color{red}boto3}$	1.34.106	1.34.132
ipython	8.25.0	8.25.0
jupyter-lsp	2.2.5	2.2.5
${\color{red}jupyterlab}$	4.1.6	4.2.2
amazon-q-developer-jupyterlab-ext	3.1.0	3.1.0
${\color{red}langchain}$	0.1.9	0.2.5
jupyter-ai	2.18.1	2.18.1
amazon-sagemaker-jupyter-ai-q-developer	1.0.4	1.0.4
jupyter-scheduler	2.7.1	2.7.1
amazon-sagemaker-jupyter-scheduler	3.1.0	3.1.0
amazon-sagemaker-sql-magic	0.1.3	0.1.3
${\color{red}jupyterlab-lsp}$	5.0.3	5.1.0
amazon_sagemaker_sql_editor	0.1.9	0.1.9
${\color{red}scipy}$	1.11.4	1.14.0
${\color{red}scikit-learn}$	1.4.2	1.5.0
pip	23.3.2	23.3.2
${\color{red}torchvision}$	0.15.2	0.18.1
autogluon	0.8.3	0.8.3
ipywidgets	8.1.3	8.1.3
${\color{red}notebook}$	7.1.3	7.2.1
aws-glue-sessions	1.0.6	1.0.6
conda	23.11.0	23.11.0
${\color{red}fastapi}$	0.110.3	0.111.0
${\color{red}jupyter-collaboration}$	1.1.0	1.2.0
jupyter-dash	0.4.2	0.4.2
jupyter-server-proxy	4.2.0	4.2.0
jupyterlab-git	0.50.0	0.50.0
${\color{red}keras}$	2.14.0	2.15.0
langchain-aws	0.1.6	0.1.6
matplotlib	3.8.4	3.8.4
nodejs	18.20.2	18.20.2
py-xgboost-gpu	1.7.6	1.7.6
thrift_sasl	0.4.3	0.4.3
pyhive	0.7.0	0.7.0
python-gssapi	1.8.3	1.8.3
python-lsp-server	1.11.0	1.11.0
${\color{red}pytorch-gpu}$	2.0.0	2.3.1
sagemaker-code-editor	1.1.0	1.1.0
sagemaker-headless-execution-driver	0.0.12	0.0.12
sagemaker-jupyterlab-emr-extension	0.1.9	0.1.9
sagemaker-jupyterlab-extension	0.3.2	0.3.2
sagemaker-kernel-wrapper	0.0.2	0.0.2
sagemaker-python-sdk	2.224.1	2.224.1
sagemaker-studio-analytics-extension	0.0.21	0.0.21
sasl	0.3.1	0.3.1
supervisor	4.2.5	4.2.5
${\color{red}tensorflow-gpu}$	2.14.0	2.16.1
uvicorn	0.30.1	0.30.1

Staleness Report: 1.9.0(cpu)

Package	Current Version in the Distribution image	Latest Relevant Version in Upstream
numpy	1.26.4	1.26.4
jinja2	3.1.4	3.1.4
${\color{red}pytorch}$	2.0.0	2.3.1
${\color{red}pandas}$	2.1.4	2.2.2
altair	5.3.0	5.3.0
${\color{red}boto3}$	1.34.106	1.34.132
ipython	8.25.0	8.25.0
jupyter-lsp	2.2.5	2.2.5
${\color{red}jupyterlab}$	4.1.6	4.2.2
amazon-q-developer-jupyterlab-ext	3.1.0	3.1.0
${\color{red}langchain}$	0.1.9	0.2.5
jupyter-ai	2.18.1	2.18.1
amazon-sagemaker-jupyter-ai-q-developer	1.0.4	1.0.4
jupyter-scheduler	2.7.1	2.7.1
amazon-sagemaker-jupyter-scheduler	3.1.0	3.1.0
amazon-sagemaker-sql-magic	0.1.3	0.1.3
${\color{red}jupyterlab-lsp}$	5.0.3	5.1.0
amazon_sagemaker_sql_editor	0.1.9	0.1.9
${\color{red}scipy}$	1.11.4	1.14.0
${\color{red}scikit-learn}$	1.4.2	1.5.0
pip	23.3.2	23.3.2
${\color{red}torchvision}$	0.15.2	0.18.1
autogluon	0.8.3	0.8.3
ipywidgets	8.1.3	8.1.3
${\color{red}notebook}$	7.1.3	7.2.1
aws-glue-sessions	1.0.6	1.0.6
conda	23.11.0	23.11.0
${\color{red}fastapi}$	0.110.3	0.111.0
${\color{red}jupyter-collaboration}$	1.1.0	1.2.0
jupyter-dash	0.4.2	0.4.2
jupyter-server-proxy	4.2.0	4.2.0
jupyterlab-git	0.50.0	0.50.0
keras	2.15.0	2.15.0
langchain-aws	0.1.6	0.1.6
matplotlib	3.8.4	3.8.4
nodejs	18.20.2	18.20.2
py-xgboost-cpu	1.7.6	1.7.6
thrift_sasl	0.4.3	0.4.3
pyhive	0.7.0	0.7.0
python-gssapi	1.8.3	1.8.3
python-lsp-server	1.11.0	1.11.0
sagemaker-code-editor	1.1.0	1.1.0
sagemaker-headless-execution-driver	0.0.12	0.0.12
sagemaker-jupyterlab-emr-extension	0.1.9	0.1.9
sagemaker-jupyterlab-extension	0.3.2	0.3.2
sagemaker-kernel-wrapper	0.0.2	0.0.2
sagemaker-python-sdk	2.224.1	2.224.1
sagemaker-studio-analytics-extension	0.0.21	0.0.21
sasl	0.3.1	0.3.1
supervisor	4.2.5	4.2.5
${\color{red}tensorflow}$	2.15.0	2.16.1
uvicorn	0.30.1	0.30.1

[claytonparnell]

Testing

FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_cpu[keras.test.Dockerfile-required_packages0] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_cpu[scipy.test.Dockerfile-required_packages4] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_cpu[pandas.test.Dockerfile-required_packages7] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_cpu[sm-python-sdk.test.Dockerfile-required_packages8] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_cpu[pytorch.examples.Dockerfile-required_packages9] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_gpu[keras.test.Dockerfile-required_packages0] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_gpu[scipy.test.Dockerfile-required_packages4] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_gpu[sm-python-sdk.test.Dockerfile-required_packages8] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_gpu[pytorch.examples.Dockerfile-required_packages9] - assert 1 == 0

• keras: Documented here. we pin to an old commit in test which works with 2.12, but not with 2.14; ran test locally without pin and it succeeds. Will fix in followup to dynamically get correct tests.
• scipy: Documented here: Tests fail with E TypeError: 'dia_matrix' object is not subscriptable or E TypeError: 'coo_matrix' object is not subscriptable
• pytorch: Several failures all related to file not found, running locally found that we're missing sudo apt install libsdl2-dev libsdl2-2.0-0 -y which is now needed for tests. After adding this tests succeed.
• pandas: Documented here: Just one test case fails in CPU because of use of deprecated pyarrow function in the test.
• sm-python-sdk`: Documented here: Each test failed with either
  • FileNotFoundError: [Errno 2] No such file or directory: 'docker' - expected; we don't have docker client in testing environment
  • botocore.exceptions.NoCredentialsError: Unable to locate credentials - also expected; we don't have creds in testing environment

[balajisankar15]

Why did we make this change ?
Is tensorflow-gpu officially maintained ?

[claytonparnell]

Yes it's built from the same feedstock as tensorflow (tensorflow-feedstock) and holds gpu-specific builds

[balajisankar15]

GPU and CPU variants have different versions of tensorflow. We can keep them consistent at 2.14.0

[claytonparnell]

We calculate the closure separately so we can get the highest versions, given the constraints. GPU has additional constraint but I don't think we need to dock CPU because of that. Customers are used to 2.15 and we should keep it wherever possible.