Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[do not merge] Running regression tests for python upgrades #824

Closed
wants to merge 131 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
131 commits
Select commit Hold shift + click to select a range
e6c2106
added prepare_for_eval, eval only if is_time_remaining
Niccolo-Ajroldi Sep 12, 2024
8bad99d
added prepare_for_eval to all submissions
Niccolo-Ajroldi Sep 14, 2024
1c7d51c
fix formatting
Niccolo-Ajroldi Sep 15, 2024
21a580b
fix formatting
Niccolo-Ajroldi Sep 15, 2024
420b583
updated documentation
Niccolo-Ajroldi Sep 15, 2024
d9c4ee9
add prepare_for_eval to spec.py
Niccolo-Ajroldi Oct 18, 2024
9caedc5
make prepare_for_eval backward compatible
Niccolo-Ajroldi Oct 21, 2024
4d74d2c
optional prepare_for_eval arg
Niccolo-Ajroldi Oct 21, 2024
364ce41
Merge branch 'dev' into prepare_for_eval
Niccolo-Ajroldi Oct 31, 2024
8cc4f4a
default dropout rates for workloads are added
init-22 Oct 31, 2024
a6fc879
adding the dropout info in fixed workload section
init-22 Oct 31, 2024
1983899
removing bold headings
init-22 Oct 31, 2024
e16ebe0
fix: changing the dtype in random_utils to uint32
init-22 Nov 14, 2024
42da4fd
feat: package updates with python 3.11
init-22 Nov 14, 2024
1005776
fix: reverting the python311 changes
init-22 Nov 14, 2024
ce99901
feat: package updates with python311
init-22 Nov 14, 2024
21fb3f9
fix: absl package version change
init-22 Nov 14, 2024
67b9f15
fix: pytorch version change
init-22 Nov 14, 2024
78df36f
fix: tf version to use numpy < 2
init-22 Nov 14, 2024
76b084b
fix: removed cifar10 and mnist
init-22 Nov 14, 2024
b5ad298
fix: changing PRNGkey in random_utils to key
init-22 Nov 15, 2024
2584416
fix: librispeech requirement of tf-text rolled back to v2.17
init-22 Nov 15, 2024
d603ce9
fix: using the main repo and branch for testing
init-22 Nov 15, 2024
9819868
fix: changing the range of MAX_UINT32
init-22 Nov 16, 2024
4b2e64e
bringing back PRNGKey instead of key, till the python311 branch is me…
init-22 Nov 16, 2024
be68f8c
fix: overflow error resolved and PRNGKey to key
init-22 Nov 16, 2024
f72028f
fix: ran yapf for passing the checks
init-22 Nov 19, 2024
2b8b771
fix: ran yapf for passing the checks
init-22 Nov 19, 2024
e890c89
fix: minor changes in docs
init-22 Nov 20, 2024
d8f07b7
fix: triggering the checks again
init-22 Nov 22, 2024
ff176d7
fix: triggering the checks again
init-22 Nov 22, 2024
579a485
fix pytorch_default_init()
EIFY Nov 26, 2024
1bc2a7b
fix: changing the python versions in workflow to pass the tests
init-22 Nov 30, 2024
7a0fee3
fix: changing numpy compatible version
init-22 Nov 30, 2024
7cdea16
adding key_data to check the CI tests
init-22 Nov 30, 2024
7264c3f
fix: updated packge of sacrebleu changed the way it used to work, hen…
init-22 Dec 1, 2024
abbdc82
fix: temporarily commenting tfa
init-22 Dec 1, 2024
86029a7
fix: explicitly using mask kwarg to use MultiHeadDotProductAttention …
init-22 Dec 2, 2024
aca45a2
fix: using flax.core.pop instead of variables.pop, better way to upda…
init-22 Dec 2, 2024
2618c5e
fix: changing the traindiffs_tests branch to main again
init-22 Dec 2, 2024
8c90625
fix: unfreeze() in test_param_shapes expect FrozenDict also added fla…
init-22 Dec 2, 2024
1b587b7
fix: formatting changes with yapf
init-22 Dec 3, 2024
c65d93e
fix: running yapf again with 0.32, earlier using 0.43
init-22 Dec 3, 2024
3afd1df
fix: running yapf again with 0.32, earlier using 0.43
init-22 Dec 3, 2024
6ff2010
fix: latest versions of typing dont support Text instead str is recom…
init-22 Dec 3, 2024
55bacbd
fix: minor yapf
init-22 Dec 3, 2024
9d37d3e
Merge pull request #806 from init-22/feat_default_dropout_in_doc
priyakasimbeg Dec 4, 2024
ea66793
Merge pull request #789 from Niccolo-Ajroldi/prepare_for_eval
priyakasimbeg Dec 5, 2024
cfd5a00
Merge branch 'python_upgrades' into python311
priyakasimbeg Dec 6, 2024
5eac985
fix: going back to sacrebleu v1.3.1
init-22 Dec 7, 2024
90959e1
Merge pull request #810 from init-22/fix-unit32
priyakasimbeg Dec 12, 2024
fe90379
Merge pull request #819 from EIFY/torch-init-fix
priyakasimbeg Dec 12, 2024
7867711
feat: custom tf_addons support in TF2.18
init-22 Dec 17, 2024
d6dd2e8
fix: resolving pylint issues in custom_tf_addons
init-22 Dec 17, 2024
a0b587a
resolved pyline and changed the pylint version to current version of …
init-22 Dec 17, 2024
9393145
fix: removing tensorflow addons from setup cfg
init-22 Dec 18, 2024
53eff1d
fix: adding absolute paths for custom_tf_addons in randaugment
init-22 Dec 19, 2024
9d1c957
Merge pull request #811 from init-22/python311
priyakasimbeg Dec 20, 2024
d21d820
fix: changes jax.tree_map to jax.tree.map
init-22 Dec 22, 2024
785d82b
fix: MultiHeadDotProductAttention and optax ctc_loss changes
init-22 Dec 22, 2024
d4aa90a
fix: removed the sacrebleu dependency
init-22 Dec 22, 2024
5e348e4
fix: resolving pylint errors
init-22 Dec 22, 2024
21a3d03
Merge pull request #828 from init-22/custom_bleu
priyakasimbeg Jan 9, 2025
1b88a2e
Merge pull request #827 from init-22/resolve_deprecations
priyakasimbeg Jan 9, 2025
426e6ee
Merge branch 'python_test_env_upgrade' into python_upgrades
priyakasimbeg Jan 9, 2025
b769e6c
fix startup script for python version upgrade
priyakasimbeg Jan 9, 2025
b65157e
fix: getargspec is not supported in python311, using getfullargspec i…
init-22 Jan 14, 2025
8327283
Create equivalent pyproject toml
fsschneider Jan 15, 2025
c8dc704
yapf requires toml
fsschneider Jan 15, 2025
50658bc
Revert to auto-finding packages (includes `tests/`)
fsschneider Jan 15, 2025
616a0f4
Match version to GH
fsschneider Jan 15, 2025
bad76f5
Let `setuptools_scm` handle versioning.
fsschneider Jan 15, 2025
ff4a457
Fix version test
fsschneider Jan 15, 2025
f97c880
Match file name of version test to the other tests
fsschneider Jan 15, 2025
f98b554
Fix linting
fsschneider Jan 15, 2025
8171a32
Update version test to only check major and minor elements, excluding…
fsschneider Jan 15, 2025
96cc471
Rename job in regression tests workflow from `criteo_resnet_pytorch` …
fsschneider Jan 15, 2025
230bf84
Fix some markdown linting issues.
fsschneider Jan 15, 2025
ce44582
Add trailing new line
fsschneider Jan 15, 2025
37f556d
Rename package from `algorithmic-efficiency` to `algoperf`.
fsschneider Jan 15, 2025
bc666a7
Fix linting (due to shorter package name in imports)
fsschneider Jan 15, 2025
3d8e606
Merge branch 'dev' into python_upgrades
priyakasimbeg Jan 16, 2025
d9f13ab
upgrade_jax
priyakasimbeg Jan 16, 2025
1e62f15
Merge remote-tracking branch 'refs/remotes/origin/python_upgrades' in…
priyakasimbeg Jan 16, 2025
01eb881
change jax version
priyakasimbeg Jan 16, 2025
c9b6411
change jax python version
priyakasimbeg Jan 16, 2025
fc526a4
modify limits of ints
priyakasimbeg Jan 18, 2025
6c8fd56
Merge pull request #834 from mlcommons/random_utils
priyakasimbeg Jan 18, 2025
e713053
Merge branch 'dev' into python_upgrades
priyakasimbeg Jan 18, 2025
7d580f1
fix: using jax.random.key_data only when the workload is jax
init-22 Jan 18, 2025
5715618
revert to use PRNGKey
priyakasimbeg Jan 22, 2025
d57dec3
Merge remote-tracking branch 'refs/remotes/origin/python_upgrades' in…
priyakasimbeg Jan 22, 2025
3fb722d
revert changes to submission runner for prng key
priyakasimbeg Jan 22, 2025
9b7cee4
remove extracting key_data
priyakasimbeg Jan 22, 2025
5775ed1
cast np.int32 as int for random.Random arg
priyakasimbeg Jan 23, 2025
1352e70
fix: vim installation
init-22 Jan 26, 2025
d7eebf8
use inductor backend to compile deepspeech instead of eager
priyakasimbeg Jan 30, 2025
58159c5
adding mem_fraction 0.80 for jax workfloads to resolve OOM of certain…
init-22 Feb 2, 2025
81bc93d
mem fraction typo
init-22 Feb 3, 2025
f6ca2bc
env variable for conformer set at the top
init-22 Feb 3, 2025
59126ae
Update documentation with new targets
fsschneider Feb 3, 2025
ff0086c
Use 1.5 instead of 3x the budget for self-tuning
fsschneider Feb 3, 2025
03bc79e
Update max allowed runtimes for each workload
fsschneider Feb 3, 2025
16eb8d6
Clarify in comment that its the old budgets
fsschneider Feb 3, 2025
d3f788d
Adapt step hint as well
fsschneider Feb 3, 2025
b3dec67
Clarify `step_hint`
fsschneider Feb 3, 2025
b4ed6cc
set env variables for pytorch before initializing w ddp.
priyakasimbeg Feb 4, 2025
c1f182e
Merge pull request #830 from fsschneider/pyproject.toml
priyakasimbeg Feb 5, 2025
def1a43
Merge pull request #831 from fsschneider/version_bump
priyakasimbeg Feb 5, 2025
ebf0341
set jax to 0.4.26
priyakasimbeg Feb 5, 2025
1ce6dea
set jax versions
priyakasimbeg Feb 7, 2025
2a0f1c9
switch to pyproject toml
priyakasimbeg Feb 8, 2025
082be03
fix pytorch version
priyakasimbeg Feb 8, 2025
39bb876
fix jax versions
priyakasimbeg Feb 8, 2025
b45a69b
fix: adding wandb under 'full' section
init-22 Feb 8, 2025
b719a6e
fix: wandb version upgrade
init-22 Feb 10, 2025
1ce3e62
remove wandb from full
priyakasimbeg Feb 10, 2025
7261e49
Merge branch 'dev' into minor_nits
fsschneider Feb 10, 2025
b356844
Merge pull request #832 from fsschneider/minor_nits
priyakasimbeg Feb 10, 2025
9ee2d80
Merge branch 'dev' into rename_algoperf
fsschneider Feb 10, 2025
5969f1d
Merge pull request #833 from fsschneider/rename_algoperf
priyakasimbeg Feb 10, 2025
b4f6cb1
Merge branch 'dev' into update_budgets
fsschneider Feb 10, 2025
738658c
Merge pull request #838 from fsschneider/update_budgets
priyakasimbeg Feb 10, 2025
5be969b
fix isort version in test
priyakasimbeg Feb 10, 2025
cabcc59
Merge pull request #839 from mlcommons/lint_fix
priyakasimbeg Feb 10, 2025
c8e2546
Merge branch 'dev' into python_upgrades
priyakasimbeg Feb 11, 2025
a12733a
revert isort version
priyakasimbeg Feb 11, 2025
bee0e3f
revert import order changes
priyakasimbeg Feb 11, 2025
1f72cb3
remove temporary testing for upgrades
priyakasimbeg Feb 11, 2025
4dadef0
update import path in randaugment.py
priyakasimbeg Feb 11, 2025
f375099
isort changes
priyakasimbeg Feb 11, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 25 additions & 25 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install Modules and Run
Expand All @@ -25,10 +25,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install Modules and Run
Expand All @@ -42,10 +42,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install Modules and Run
Expand All @@ -59,10 +59,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install Modules and Run
Expand All @@ -77,10 +77,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install Modules and Run
Expand All @@ -96,10 +96,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install Modules and Run
Expand All @@ -113,10 +113,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install Modules and Run
Expand All @@ -130,10 +130,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install Modules and Run
Expand All @@ -148,10 +148,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install Modules and Run
Expand All @@ -166,10 +166,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install Modules and Run
Expand All @@ -184,10 +184,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install pytest
Expand All @@ -199,7 +199,7 @@ jobs:
pip install .[pytorch_cpu]
- name: Run pytest tests
run: |
pytest -vx tests/version_test.py
pytest -vx tests/test_version.py
pytest -vx tests/test_num_params.py
pytest -vx tests/test_param_shapes.py
pytest -vx tests/test_param_types.py
Expand All @@ -208,10 +208,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v4
with:
python-version: 3.9
python-version: 3.11.10
cache: 'pip' # Cache pip dependencies\.
cache-dependency-path: '**/setup.py'
- name: Install pytest
Expand Down
18 changes: 9 additions & 9 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,17 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v2
with:
python-version: 3.9
python-version: 3.11.10
- name: Install pylint
run: |
python -m pip install --upgrade pip
pip install pylint==2.16.1
- name: Run pylint
run: |
pylint algorithmic_efficiency
pylint algoperf
pylint reference_algorithms
pylint prize_qualification_baselines
pylint submission_runner.py
Expand All @@ -27,14 +27,14 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v2
with:
python-version: 3.9
python-version: 3.11.10
- name: Install isort
run: |
python -m pip install --upgrade pip
pip install isort
pip install isort==5.12.0
- name: Run isort
run: |
isort . --check --diff
Expand All @@ -43,14 +43,14 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.9
- name: Set up Python 3.11.10
uses: actions/setup-python@v2
with:
python-version: 3.9
python-version: 3.11.10
- name: Install yapf
run: |
python -m pip install --upgrade pip
pip install yapf==0.32
pip install yapf==0.32 toml
- name: Run yapf
run: |
yapf . --diff --recursive
Loading
Loading