Skip to content

Commit

Permalink
Fix issues to support latest version of TensorFlow. (#2216)
Browse files Browse the repository at this point in the history 
* Change _captures to captures.
* Update code to avoid keras version error.
* Add new versions of tf in CI pipeline.
* Update tflite ops definitions.
* Refactor the tests in CI pipeline.
---------

Signed-off-by: Jay Zhang <[email protected]>
  • Loading branch information
@fatcat-z
fatcat-z authored Aug 18, 2023
1 parent 0152029 commit a0bd3e5
Show file tree
Hide file tree
Showing 30 changed files with 513 additions and 349 deletions.
12 changes: 11 additions & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
.coverage*
*.pyc
.idea
build
Expand All @@ -18,3 +17,14 @@ tests/tfhub/*/*.onnx
tests/tfhub/*/*.tar.gz
tests/tfhub/*/*.tflite
tests/tfhub/*/**

# Unit test / coverage reports
.coverage*
.cache
.hypothesis/
.pytest_cache/
test-output.xml

# VSCode
.vscode/
!.vscode/extensions.json
6 changes: 3 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ The common issues we run into we try to document here [Troubleshooting Guide](Tr

| Build Type | OS | Python | TensorFlow | ONNX opset | Status |
| --- | --- | --- | --- | --- | --- |
| Unit Test - Basic | Linux, MacOS<sup>\*</sup>, Windows<sup>\*</sup> | 3.7-3.10 | 1.15, 2.1-2.11 | 14-18 | [![Build Status](https://dev.azure.com/tensorflow-onnx/tensorflow-onnx/_apis/build/status/unit_test?branchName=main)](https://dev.azure.com/tensorflow-onnx/tensorflow-onnx/_build/latest?definitionId=16&branchName=main) |
| Unit Test - Full | Linux, MacOS, Windows | 3.7-3.10 | 1.15, 2.1-2.11 | 14-18 | [![Build Status](https://dev.azure.com/tensorflow-onnx/tensorflow-onnx/_apis/build/status/unit_test-matrix?branchName=main)](https://dev.azure.com/tensorflow-onnx/tensorflow-onnx/_build/latest?definitionId=18&branchName=main) | |
| Unit Test - Basic | Linux, Windows | 3.7-3.10 | 1.15, 2.8-2.13 | 14-18 | [![Build Status](https://dev.azure.com/tensorflow-onnx/tensorflow-onnx/_apis/build/status/unit_test?branchName=main)](https://dev.azure.com/tensorflow-onnx/tensorflow-onnx/_build/latest?definitionId=16&branchName=main) |
| Unit Test - Full | Linux, Windows | 3.7-3.10 | 1.15, 2.8-2.13 | 14-18 | [![Build Status](https://dev.azure.com/tensorflow-onnx/tensorflow-onnx/_apis/build/status/unit_test-matrix?branchName=main)](https://dev.azure.com/tensorflow-onnx/tensorflow-onnx/_build/latest?definitionId=18&branchName=main) | |
<br/>

## Supported Versions
Expand All @@ -34,7 +34,7 @@ If you want the graph to be generated with a specific opset, use ```--opset``` i

### TensorFlow

We support ```tf-1.x graphs``` and ```tf-2.x```. To keep our test matrix manageable we test tf2onnx running on top of ```tf-1.13 or better```.
We support ```tf-1.x graphs``` and ```tf-2.x```. To keep our test matrix manageable we test tf2onnx running on top of ```tf-1.15 or better```.

When running under tf-2.x tf2onnx will use the tensorflow V2 controlflow.

Expand Down
100 changes: 62 additions & 38 deletions ci_build/azure_pipelines/keras2onnx_application_tests.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,55 +8,67 @@ jobs:
vmImage: 'ubuntu-latest'
strategy:
matrix:
Python37-onnx1.11-tf1.15:
python.version: '3.7'
ONNX_PATH: onnx==1.11.0
Python37-tf-1.15:
python.version: '3.7' # Max version that supports tf 1.15
ONNX_PATH: onnx==1.14.1
INSTALL_KERAS: pip install keras==2.3.1
UNINSTALL_KERAS:
INSTALL_TENSORFLOW: pip install tensorflow==1.15.0
INSTALL_ORT: pip install onnxruntime==1.11.0
INSTALL_ORT: pip install onnxruntime==1.14.1 # Max version that supports python 3.7
INSTALL_KERAS_RESNET: pip install keras-resnet
INSTALL_TRANSFORMERS:
INSTALL_NUMPY: pip install numpy==1.19.0
INSTALL_LEGACY: pip install h5py==2.9.0
NIGHTLY_BUILD_TEST: python run_all_v2.py --exclude "test_keras_applications_v2.py"

Python37-onnx1.11-tf2.5:
python.version: '3.7'
ONNX_PATH: onnx==1.11.0
Python39-tf2-2.12:
python.version: '3.9'
ONNX_PATH: onnx==1.14.1
INSTALL_KERAS:
UNINSTALL_KERAS: pip uninstall keras -y
INSTALL_TENSORFLOW: pip install tensorflow==2.5.0
INSTALL_ORT: pip install onnxruntime==1.11.0
INSTALL_TENSORFLOW: pip install tensorflow==2.12.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_KERAS_RESNET: pip install keras-resnet
INSTALL_TRANSFORMERS: pip install transformers==3.4.0
INSTALL_NUMPY: pip install numpy==1.19.0
NIGHTLY_BUILD_TEST: python run_all_v2.py

Python39-onnx1.12-tf2.8:
python.version: '3.9'
ONNX_PATH: onnx==1.12.0
Python310-tf2-2.11:
python.version: '3.10'
ONNX_PATH: onnx==1.14.1
INSTALL_KERAS:
UNINSTALL_KERAS:
INSTALL_TENSORFLOW: pip install tensorflow==2.8.0
INSTALL_ORT: pip install onnxruntime==1.12.0
INSTALL_TENSORFLOW: pip install tensorflow==2.11.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_KERAS_RESNET: pip install keras-resnet
INSTALL_TRANSFORMERS: pip install transformers==3.4.0
INSTALL_NUMPY:
NIGHTLY_BUILD_TEST: python run_all_v2.py

Python310-onnx1.12-tf2.9:
python.version: '3.10'
ONNX_PATH: onnx==1.12.0
Python38-tf2-2.10:
python.version: '3.8'
ONNX_PATH: onnx==1.14.1
INSTALL_KERAS:
UNINSTALL_KERAS:
INSTALL_TENSORFLOW: pip install tensorflow==2.9.0
INSTALL_ORT: pip install onnxruntime==1.13.1
INSTALL_TENSORFLOW: pip install tensorflow==2.10.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_KERAS_RESNET: pip install keras-resnet
INSTALL_TRANSFORMERS: pip install transformers==4.12.0
INSTALL_NUMPY:
NIGHTLY_BUILD_TEST: python run_all_v2.py

Python310-tf2-oldest:
python.version: '3.10'
ONNX_PATH: onnx==1.14.1
INSTALL_KERAS:
UNINSTALL_KERAS:
INSTALL_TENSORFLOW: pip install tensorflow==2.8.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_KERAS_RESNET: pip install keras-resnet
INSTALL_TRANSFORMERS: pip install transformers==4.2.0
INSTALL_NUMPY:
NIGHTLY_BUILD_TEST: python run_all_v2.py

steps:
- template: 'templates/keras2onnx_application_tests.yml'
parameters:
Expand All @@ -68,55 +80,67 @@ jobs:
vmImage: 'windows-2019'
strategy:
matrix:
Python37-onnx1.11-tf1.15:
python.version: '3.7'
ONNX_PATH: onnx==1.11.0
Python37-tf-1.15:
python.version: '3.7' # Max version that supports tf 1.15
ONNX_PATH: onnx==1.14.1
INSTALL_KERAS: pip install keras==2.3.1
UNINSTALL_KERAS:
INSTALL_TENSORFLOW: pip install tensorflow==1.15.0
INSTALL_ORT: pip install onnxruntime==1.11.0
INSTALL_ORT: pip install onnxruntime==1.14.1 # Max version that supports python 3.7
INSTALL_KERAS_RESNET: pip install keras-resnet
INSTALL_TRANSFORMERS:
INSTALL_NUMPY: pip install numpy==1.19.0
INSTALL_LEGACY: pip install h5py==2.9.0
NIGHTLY_BUILD_TEST: python run_all_v2.py --exclude "test_keras_applications_v2.py"

Python37-onnx1.11-tf2.5:
python.version: '3.7'
ONNX_PATH: onnx==1.11.0
Python39-tf2-2.12:
python.version: '3.9'
ONNX_PATH: onnx==1.14.1
INSTALL_KERAS:
UNINSTALL_KERAS: pip uninstall keras -y
INSTALL_TENSORFLOW: pip install tensorflow==2.5.0
INSTALL_ORT: pip install onnxruntime==1.11.0
INSTALL_TENSORFLOW: pip install tensorflow==2.12.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_KERAS_RESNET: pip install keras-resnet
INSTALL_TRANSFORMERS: pip install transformers==3.4.0
INSTALL_NUMPY: pip install numpy==1.19.0
NIGHTLY_BUILD_TEST: python run_all_v2.py

Python39-onnx1.12-tf2.8:
python.version: '3.9'
ONNX_PATH: onnx==1.12.0
Python310-tf2-2.11:
python.version: '3.10'
ONNX_PATH: onnx==1.14.1
INSTALL_KERAS:
UNINSTALL_KERAS:
INSTALL_TENSORFLOW: pip install tensorflow==2.8.0
INSTALL_ORT: pip install onnxruntime==1.12.0
INSTALL_TENSORFLOW: pip install tensorflow==2.11.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_KERAS_RESNET: pip install keras-resnet
INSTALL_TRANSFORMERS: pip install transformers==3.4.0
INSTALL_NUMPY:
NIGHTLY_BUILD_TEST: python run_all_v2.py

Python310-onnx1.12-tf2.9:
python.version: '3.10'
ONNX_PATH: onnx==1.12.0
Python38-tf2-2.10:
python.version: '3.8'
ONNX_PATH: onnx==1.14.1
INSTALL_KERAS:
UNINSTALL_KERAS:
INSTALL_TENSORFLOW: pip install tensorflow==2.9.0
INSTALL_ORT: pip install onnxruntime==1.13.1
INSTALL_TENSORFLOW: pip install tensorflow==2.10.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_KERAS_RESNET: pip install keras-resnet
INSTALL_TRANSFORMERS: pip install transformers==4.12.0
INSTALL_NUMPY:
NIGHTLY_BUILD_TEST: python run_all_v2.py

Python310-tf2-oldest:
python.version: '3.10'
ONNX_PATH: onnx==1.14.1
INSTALL_KERAS:
UNINSTALL_KERAS:
INSTALL_TENSORFLOW: pip install tensorflow==2.8.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_KERAS_RESNET: pip install keras-resnet
INSTALL_TRANSFORMERS: pip install transformers==4.2.0
INSTALL_NUMPY:
NIGHTLY_BUILD_TEST: python run_all_v2.py

steps:
- template: 'templates/keras2onnx_application_tests.yml'
parameters:
Expand Down
128 changes: 70 additions & 58 deletions ci_build/azure_pipelines/keras2onnx_unit_test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,49 +8,55 @@ jobs:
strategy:
matrix:
############ TF Keras Unit Tests ############
Python38-tf2.5:
python.version: '3.8'
ONNX_PATH: onnx==1.11.0
TENSORFLOW_PATH: tensorflow==2.5.0
INSTALL_ORT: pip install onnxruntime==1.11.0
INSTALL_NUMPY: pip install numpy==1.19.0
Python310-tf2-2.13:
python.version: '3.10'
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.13.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

Python37-tf2.8:
python.version: '3.7'
ONNX_PATH: onnx==1.12.0
TENSORFLOW_PATH: tensorflow==2.8.0
INSTALL_ORT: pip install onnxruntime==1.12.0
Python38-tf2-2.12:
python.version: '3.8'
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.12.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

Python39-tf2.9:
Python39-tf2-2.11:
python.version: '3.9'
ONNX_PATH: onnx==1.12.0
TENSORFLOW_PATH: tensorflow==2.9.0
INSTALL_ORT: pip install onnxruntime==1.12.0
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.11.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

Python310-tf2.10:
Python310-tf2-2.10:
python.version: '3.10'
ONNX_PATH: onnx==1.12.0
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.10.0
INSTALL_ORT: pip install onnxruntime==1.13.1
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

Python38-tf2-2.9:
python.version: '3.8'
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.9.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

Python310-tf2-oldest:
python.version: '3.10'
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.8.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

############ Pure Keras Unit Tests ############
Keras-Py37-tf1.15.0:
python.version: '3.7'
ONNX_PATH: onnx==1.11.0
python.version: '3.7' # Max version that supports tf 1.15
ONNX_PATH: onnx==1.14.1
KERAS: keras==2.4.3
TENSORFLOW_PATH: tensorflow==1.15.0
INSTALL_ORT: pip install onnxruntime==1.11.0
INSTALL_NUMPY: pip install numpy==1.19.0

Keras-Py38-tf2.2.0:
python.version: '3.8'
ONNX_PATH: -i onnx==1.11.0
KERAS: keras==2.4.3
TENSORFLOW_PATH: tensorflow==2.2.0
INSTALL_ORT: pip install onnxruntime==1.11.0
INSTALL_ORT: pip install onnxruntime==1.14.1 # Max version that supports python 3.7
INSTALL_NUMPY: pip install numpy==1.19.0

steps:
Expand All @@ -64,49 +70,55 @@ jobs:
strategy:
matrix:
############ TF Keras Unit Tests ############
Python38-tf2.5:
python.version: '3.8'
ONNX_PATH: onnx==1.11.0
TENSORFLOW_PATH: tensorflow==2.5.0
INSTALL_ORT: pip install onnxruntime==1.11.0
INSTALL_NUMPY: pip install numpy==1.19.0
Python310-tf2-2.13:
python.version: '3.10'
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.13.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

Python37-tf2.8:
python.version: '3.7'
ONNX_PATH: onnx==1.12.0
TENSORFLOW_PATH: tensorflow==2.8.0
INSTALL_ORT: pip install onnxruntime==1.12.0
Python38-tf2-2.12:
python.version: '3.8'
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.12.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

Python39-tf2.9:
Python39-tf2-2.11:
python.version: '3.9'
ONNX_PATH: onnx==1.12.0
TENSORFLOW_PATH: tensorflow==2.9.0
INSTALL_ORT: pip install onnxruntime==1.12.0
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.11.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

Python310-tf2.10:
Python310-tf2-2.10:
python.version: '3.10'
ONNX_PATH: onnx==1.12.0
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.10.0
INSTALL_ORT: pip install onnxruntime==1.13.1
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

Python38-tf2-2.9:
python.version: '3.8'
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.9.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

Python310-tf2-oldest:
python.version: '3.10'
ONNX_PATH: onnx==1.14.1
TENSORFLOW_PATH: tensorflow==2.8.0
INSTALL_ORT: pip install onnxruntime==1.15.0
INSTALL_NUMPY:

############ Pure Keras Unit Tests ############
Keras-Py37-tf1.15.0:
python.version: '3.7'
ONNX_PATH: onnx==1.11.0
python.version: '3.7' # Max version that supports tf 1.15
ONNX_PATH: onnx==1.14.1
KERAS: keras==2.4.3
TENSORFLOW_PATH: tensorflow==1.15.0
INSTALL_ORT: pip install onnxruntime==1.11.0
INSTALL_NUMPY: pip install numpy==1.19.0

Keras-Py38-tf2.2.0:
python.version: '3.8'
ONNX_PATH: -i onnx==1.11.0
KERAS: keras==2.4.3
TENSORFLOW_PATH: tensorflow==2.2.0
INSTALL_ORT: pip install onnxruntime==1.11.0
INSTALL_ORT: pip install onnxruntime==1.14.1 # Max version that supports python 3.7
INSTALL_NUMPY: pip install numpy==1.19.0

steps:
Expand Down
Loading

0 comments on commit a0bd3e5

Please sign in to comment.