From c736241b0af62025d603343e184b0041384d011b Mon Sep 17 00:00:00 2001 From: Simon Zhao Date: Tue, 20 Feb 2024 01:39:09 +0800 Subject: [PATCH] Resolve issue #2018 (#2022) * Issue with TF Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Comment out the PR gate affected tests with the upgrade to TF>2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Comment out the nightly builds affected tests with the upgrade to TF>2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :bug: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Comment out the nightly builds affected tests with the upgrade to TF>2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * revert the breaking tests with TF 2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * temporary pin to TF=2.8.4 Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Update security tests Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Try to resolve #2018 Signed-off-by: Simon Zhao * Exclude tensorflow versions that are not supported Signed-off-by: Simon Zhao * Correct version comparison using packaging.version.Version Signed-off-by: Simon Zhao * Capture importerror Signed-off-by: Simon Zhao * Restrict tensorflow < 2.13 Signed-off-by: Simon Zhao * Set tensorflow < 2.12 Signed-off-by: Simon Zhao * Not triggering unit tests on Draft PR (#2033) * Not triggering unit tests on Draft PR Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Change a PR-triggering file to test Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> --------- Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> Signed-off-by: Simon Zhao * Refactor ranking metric `map` to be the same as Spark's (#2004) * Announcement LF Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update email Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update README.md Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * security Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * license and contribution notice Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * update author link Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Add new code of conduct from LF Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Replacing references GRU4Rec to GRU Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Replacing references GRU4Rec to GRU Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Replacing references GRU4Rec in config files Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update references Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Delete conda.md Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * refactor map_at_k and map to be the same as Spark's Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * list of test failing to fix Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update readme LF feedback @wutaomsft Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update NEWS.md Co-authored-by: Andreas Argyriou Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update README.md Co-authored-by: Andreas Argyriou Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Fix test errors, Refactor column check utils to be simpler Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Rename ranking tests to be _at_k suffixed Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Change test names in the test group Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * add comment to mocked fn in a test Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * :memo: Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * remove unused input Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * :memo: Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * no need to output the logs twice Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * packages Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * skipping flaky test Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Issue with TF Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Comment out the PR gate affected tests with the upgrade to TF>2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Comment out the nightly builds affected tests with the upgrade to TF>2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * :bug: Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Comment out the nightly builds affected tests with the upgrade to TF>2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * revert the breaking tests with TF 2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * temporary pin to TF=2.8.4 Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update security tests Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update expected values to not use fixture Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * list of test failing to fix Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Fix missing fixture error Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> --------- Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> Co-authored-by: miguelgfierro Co-authored-by: Andreas Argyriou Co-authored-by: Miguel Fierro <3491412+miguelgfierro@users.noreply.github.com> Signed-off-by: Simon Zhao * Add missing kernelspec language Signed-off-by: Simon Zhao * Remove scrapbook and papermill deps Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * notebook utils programmatic execution Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Test notebook programmatic Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Added test notebook for utils Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * data notebooks Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Replace papermill and scrapbook for new internal function Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Replace papermill and scrapbook for new internal function Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Update new programmatic execution code Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Update new programmatic execution code Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Update notebooks with new utility Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :bug: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Issue with xDeepFM WIP Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :bug: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :bug: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Document the tests in programmatic notebook Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * WIP Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * WIP Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Import missing store_metadata Signed-off-by: Simon Zhao * Correct pattern matching and substitution Signed-off-by: Simon Zhao * Merge multiline parameters into one line Signed-off-by: Simon Zhao * Increase timeout Signed-off-by: Simon Zhao * Fix nightly test errors (#2045) * Revert tests tolerance * Fix notebook parameter parsing * Add notebook utils tests to test groups * Fix notebooks * Fix notebook unit tests * Update evaluation metrics name map. Handle None for exp_var * Fix smoke tests * cleanup * Fix functional test errors * make notebook parameter update function to be private * Fix benchmark notebook bug * fix remaining bugs --------- Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> Signed-off-by: Simon Zhao * Fix benchmarks last cell to store value, not [value] Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: remove papermill and scrapbook references Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: remove papermill and scrapbook references Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: remove papermill and scrapbook references Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: remove papermill and scrapbook references Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: remove papermill and scrapbook references Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Updated PR template Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Updated contributing Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Updated PR template and contributing Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Updated contributing Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * [Fix] correct MIND data construction of user behavior history Signed-off-by: Simon Zhao * change path hybrid Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Update hybrid to CF Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * change path hybrid Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * change path hybrid Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Replace LayerRNNCell with AbstractRNNCell Signed-off-by: Simon Zhao * Stop testing for deeprec Signed-off-by: Simon Zhao * Refactor ranking metric `map` to be the same as Spark's (#2004) * Announcement LF Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update email Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update README.md Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * security Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * license and contribution notice Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * update author link Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Add new code of conduct from LF Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Replacing references GRU4Rec to GRU Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Replacing references GRU4Rec to GRU Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Replacing references GRU4Rec in config files Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update references Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Delete conda.md Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * refactor map_at_k and map to be the same as Spark's Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * list of test failing to fix Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update readme LF feedback @wutaomsft Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update NEWS.md Co-authored-by: Andreas Argyriou Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update README.md Co-authored-by: Andreas Argyriou Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Fix test errors, Refactor column check utils to be simpler Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Rename ranking tests to be _at_k suffixed Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Change test names in the test group Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * add comment to mocked fn in a test Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * :memo: Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * remove unused input Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * :memo: Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * no need to output the logs twice Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * packages Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * skipping flaky test Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Issue with TF Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Comment out the PR gate affected tests with the upgrade to TF>2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Comment out the nightly builds affected tests with the upgrade to TF>2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * :bug: Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Comment out the nightly builds affected tests with the upgrade to TF>2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * revert the breaking tests with TF 2.10.1 Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * temporary pin to TF=2.8.4 Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update security tests Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Update expected values to not use fixture Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * list of test failing to fix Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> * Fix missing fixture error Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> --------- Signed-off-by: miguelgfierro Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> Co-authored-by: miguelgfierro Co-authored-by: Andreas Argyriou Co-authored-by: Miguel Fierro <3491412+miguelgfierro@users.noreply.github.com> * notebook utils programmatic execution Signed-off-by: miguelgfierro * Test notebook programmatic Signed-off-by: miguelgfierro * Added test notebook for utils Signed-off-by: miguelgfierro * Replace papermill and scrapbook for new internal function Signed-off-by: miguelgfierro * Replace papermill and scrapbook for new internal function Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Update new programmatic execution code Signed-off-by: miguelgfierro * Update new programmatic execution code Signed-off-by: miguelgfierro * Update notebooks with new utility Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :bug: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Issue with xDeepFM WIP Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :bug: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :bug: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Document the tests in programmatic notebook Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * WIP Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * WIP Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Import missing store_metadata Signed-off-by: Simon Zhao * Correct pattern matching and substitution Signed-off-by: Simon Zhao * Increase timeout Signed-off-by: Simon Zhao * Fix nightly test errors (#2045) * Revert tests tolerance * Fix notebook parameter parsing * Add notebook utils tests to test groups * Fix notebooks * Fix notebook unit tests * Update evaluation metrics name map. Handle None for exp_var * Fix smoke tests * cleanup * Fix functional test errors * make notebook parameter update function to be private * Fix benchmark notebook bug * fix remaining bugs --------- Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> Signed-off-by: Simon Zhao * Fix benchmarks last cell to store value, not [value] Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: remove papermill and scrapbook references Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: remove papermill and scrapbook references Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: remove papermill and scrapbook references Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Updated PR template Signed-off-by: miguelgfierro * Updated contributing Signed-off-by: miguelgfierro * Updated PR template and contributing Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Updated contributing Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * change path hybrid Signed-off-by: miguelgfierro * change path hybrid Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Creating a jupyter book Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Creating documentation Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * WIP Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Added rst files Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * license Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Weird warning with a link in the docstrings Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Fix docstring errors and replace .. note:: with Note: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Automatic build of documentation Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Automatic build of documentation dev Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Automatic build of documentation deps Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Automatic build of documentation deps Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Automatic build of documentation deps Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Delete workflow and try via UI Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Added again the workflow Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * git add * -rf Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * git add * -f Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * add git info Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * actions to automatically update documentation Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * actions to automatically update documentation Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * actions to automatically update documentation :bug: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * actions to automatically update documentation :bug: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * trying github token Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * trying github token Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * trying github token and pull before pushing Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * pull rebase Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * pull rebase and -Xtheirs Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * clean Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Update documentation badge Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * install all deps Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * try adding other sphinx extensions Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Refact model rst Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * comment geoimc and rlrmc docs until issue is fixed Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * :memo: Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Adding init and other special members Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Adding init and other special members Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Reviewing other rst Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Change sphinx version Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Change sphinx version and jupyter book Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Change the way we compile the documentation Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Using the latest JB release Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Documentation working Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Update docs/_config.yml Co-authored-by: Simon Zhao Signed-off-by: Simon Zhao * Update docs/requirements-doc.txt Co-authored-by: Simon Zhao Signed-off-by: Simon Zhao * Update docs/_config.yml Co-authored-by: Simon Zhao Signed-off-by: Simon Zhao * Added comments by @SimonYansenZhao Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao * Upgrade versions of GitHub Actions See https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/ Signed-off-by: Simon Zhao * Update setup.py Co-authored-by: Miguel Fierro <3491412+miguelgfierro@users.noreply.github.com> Signed-off-by: Simon Zhao * Try to disable sum and sum_component only Signed-off-by: Simon Zhao * Upgrade AzureML docker image Signed-off-by: Simon Zhao * Correct variable names Signed-off-by: Simon Zhao * Install git in the Conda env Signed-off-by: Simon Zhao * Disable test_xdeepfm_component_definition Signed-off-by: Simon Zhao * Use latest CUDA Signed-off-by: Simon Zhao * Correct GPU selection Signed-off-by: Simon Zhao * Remove leading whitespaces in Dockerfile Signed-off-by: Simon Zhao * Simplify azureml-test/action.yml Signed-off-by: Simon Zhao * Install wget in Docker image Signed-off-by: Simon Zhao * Update Signed-off-by: Simon Zhao --------- Signed-off-by: miguelgfierro Signed-off-by: Simon Zhao Signed-off-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> Co-authored-by: miguelgfierro Co-authored-by: Jun Ki Min <42475935+loomlike@users.noreply.github.com> Co-authored-by: Andreas Argyriou Co-authored-by: Miguel Fierro <3491412+miguelgfierro@users.noreply.github.com> Co-authored-by: thaiminhpv --- .github/actions/azureml-test/action.yml | 49 +++++++--------- .github/workflows/azureml-cpu-nightly.yml | 4 +- .github/workflows/azureml-gpu-nightly.yml | 4 +- .../workflows/azureml-release-pipeline.yml | 4 +- .github/workflows/azureml-spark-nightly.yml | 4 +- .github/workflows/azureml-unit-tests.yml | 4 +- .github/workflows/sarplus.yml | 12 ++-- .github/workflows/update_documentation.yml | 4 +- .../models/sequential/rnn_cell_implement.py | 4 +- setup.py | 3 +- .../submit_groupwise_azureml_pytest.py | 56 +++++++++++-------- tests/ci/azureml_tests/test_groups.py | 20 +++---- tests/security/test_dependency_security.py | 12 ++-- 13 files changed, 92 insertions(+), 88 deletions(-) diff --git a/.github/actions/azureml-test/action.yml b/.github/actions/azureml-test/action.yml index 3770298d2..d74b88160 100644 --- a/.github/actions/azureml-test/action.yml +++ b/.github/actions/azureml-test/action.yml @@ -69,7 +69,7 @@ runs: using: "composite" steps: - name: Setup python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: "3.8" - name: Install azureml-core and azure-cli on a GitHub hosted server @@ -82,43 +82,32 @@ runs: - name: Install wheel package shell: bash run: pip install --quiet wheel - - name: Submit CPU tests to AzureML + - name: Submit tests to AzureML shell: bash - if: contains(inputs.TEST_GROUP, 'cpu') run: >- - python tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py --clustername ${{inputs.CPU_CLUSTER_NAME}} - --subid ${{inputs.AZUREML_TEST_SUBID}} --reponame "recommenders" --branch ${{ github.ref }} - --rg ${{inputs.RG}} --wsname ${{inputs.WS}} --expname ${{inputs.EXP_NAME}}_${{inputs.TEST_GROUP}} - --testlogs ${{inputs.TEST_LOGS_PATH}} --testkind ${{inputs.TEST_KIND}} - --conda_pkg_python ${{inputs.PYTHON_VERSION}} --testgroup ${{inputs.TEST_GROUP}} - --disable-warnings --sha "${GITHUB_SHA}" - - name: Submit GPU tests to AzureML - shell: bash - if: contains(inputs.TEST_GROUP, 'gpu') - run: >- - python tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py --clustername ${{inputs.GPU_CLUSTER_NAME}} - --subid ${{inputs.AZUREML_TEST_SUBID}} --reponame "recommenders" --branch ${{ github.ref }} - --rg ${{inputs.RG}} --wsname ${{inputs.WS}} --expname ${{inputs.EXP_NAME}}_${{inputs.TEST_GROUP}} - --testlogs ${{inputs.TEST_LOGS_PATH}} --add_gpu_dependencies --testkind ${{inputs.TEST_KIND}} - --conda_pkg_python ${{inputs.PYTHON_VERSION}} --testgroup ${{inputs.TEST_GROUP}} - --disable-warnings --sha "${GITHUB_SHA}" - - name: Submit PySpark tests to AzureML - shell: bash - if: contains(inputs.TEST_GROUP, 'spark') - run: >- - python tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py --clustername ${{inputs.CPU_CLUSTER_NAME}} - --subid ${{inputs.AZUREML_TEST_SUBID}} --reponame "recommenders" --branch ${{ github.ref }} - --rg ${{inputs.RG}} --wsname ${{inputs.WS}} --expname ${{inputs.EXP_NAME}}_${{inputs.TEST_GROUP}} - --testlogs ${{inputs.TEST_LOGS_PATH}} --add_spark_dependencies --testkind ${{inputs.TEST_KIND}} - --conda_pkg_python ${{inputs.PYTHON_VERSION}} --testgroup ${{inputs.TEST_GROUP}} - --disable-warnings --sha "${GITHUB_SHA}" + python tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py \ + --subid ${{inputs.AZUREML_TEST_SUBID}} \ + --reponame "recommenders" \ + --branch ${{ github.ref }} \ + --rg ${{inputs.RG}} \ + --wsname ${{inputs.WS}} \ + --expname ${{inputs.EXP_NAME}}_${{inputs.TEST_GROUP}} \ + --testlogs ${{inputs.TEST_LOGS_PATH}} \ + --testkind ${{inputs.TEST_KIND}} \ + --conda_pkg_python ${{inputs.PYTHON_VERSION}} \ + --testgroup ${{inputs.TEST_GROUP}} \ + --disable-warnings \ + --sha "${GITHUB_SHA}" \ + --clustername $(if [[ ${{inputs.TEST_GROUP}} =~ "gpu" ]]; then echo "${{inputs.GPU_CLUSTER_NAME}}"; else echo "${{inputs.CPU_CLUSTER_NAME}}"; fi) \ + $(if [[ ${{inputs.TEST_GROUP}} =~ "gpu" ]]; then echo "--add_gpu_dependencies"; fi) \ + $(if [[ ${{inputs.TEST_GROUP}} =~ "spark" ]]; then echo "--add_spark_dependencies"; fi) - name: Get exit status shell: bash id: exit_status run: echo "code=$(cat ${{inputs.PYTEST_EXIT_CODE}})" >> $GITHUB_OUTPUT - name: Check Success/Failure if: ${{ steps.exit_status.outputs.code != 0 }} - uses: actions/github-script@v3 + uses: actions/github-script@v7 with: script: | core.setFailed('All tests did not pass!') diff --git a/.github/workflows/azureml-cpu-nightly.yml b/.github/workflows/azureml-cpu-nightly.yml index e2142a231..72bb700cf 100644 --- a/.github/workflows/azureml-cpu-nightly.yml +++ b/.github/workflows/azureml-cpu-nightly.yml @@ -47,7 +47,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Check out repository code - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Get test group names id: get_test_groups uses: ./.github/actions/get-test-groups @@ -71,7 +71,7 @@ jobs: test-group: ${{ fromJSON(needs.get-test-groups.outputs.test_groups) }} steps: - name: Check out repository code - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Execute tests uses: ./.github/actions/azureml-test id: execute_tests diff --git a/.github/workflows/azureml-gpu-nightly.yml b/.github/workflows/azureml-gpu-nightly.yml index dcf0bc102..efac48774 100644 --- a/.github/workflows/azureml-gpu-nightly.yml +++ b/.github/workflows/azureml-gpu-nightly.yml @@ -47,7 +47,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Check out repository code - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Get test group names id: get_test_groups uses: ./.github/actions/get-test-groups @@ -71,7 +71,7 @@ jobs: test-group: ${{ fromJSON(needs.get-test-groups.outputs.test_groups) }} steps: - name: Check out repository code - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Execute tests uses: ./.github/actions/azureml-test id: execute_tests diff --git a/.github/workflows/azureml-release-pipeline.yml b/.github/workflows/azureml-release-pipeline.yml index 8475a9a2f..d9899658e 100644 --- a/.github/workflows/azureml-release-pipeline.yml +++ b/.github/workflows/azureml-release-pipeline.yml @@ -33,9 +33,9 @@ jobs: needs: [unit-test-workflow, cpu-nightly-workflow, gpu-nightly-workflow, spark-nightly-workflow] steps: - name: Check out repository code - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Setup python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: "3.8" - name: Install wheel package diff --git a/.github/workflows/azureml-spark-nightly.yml b/.github/workflows/azureml-spark-nightly.yml index 434929c2e..b3a76f9ea 100644 --- a/.github/workflows/azureml-spark-nightly.yml +++ b/.github/workflows/azureml-spark-nightly.yml @@ -46,7 +46,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Check out repository code - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Get test group names id: get_test_groups uses: ./.github/actions/get-test-groups @@ -70,7 +70,7 @@ jobs: test-group: ${{ fromJSON(needs.get-test-groups.outputs.test_groups) }} steps: - name: Check out repository code - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Execute tests uses: ./.github/actions/azureml-test id: execute_tests diff --git a/.github/workflows/azureml-unit-tests.yml b/.github/workflows/azureml-unit-tests.yml index a175b7247..13ed56005 100644 --- a/.github/workflows/azureml-unit-tests.yml +++ b/.github/workflows/azureml-unit-tests.yml @@ -36,7 +36,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Check out repository code - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Get test group names id: get_test_groups uses: ./.github/actions/get-test-groups @@ -60,7 +60,7 @@ jobs: test-group: ${{ fromJSON(needs.get-test-groups.outputs.test_groups) }} steps: - name: Check out repository code - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Execute tests uses: ./.github/actions/azureml-test id: execute_tests diff --git a/.github/workflows/sarplus.yml b/.github/workflows/sarplus.yml index 6df1c6c9b..766b31645 100644 --- a/.github/workflows/sarplus.yml +++ b/.github/workflows/sarplus.yml @@ -41,10 +41,10 @@ jobs: matrix: python-version: ["3.8", "3.9"] steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Set up Python ${{ matrix.python-version }} - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} @@ -96,7 +96,7 @@ jobs: - name: Upload Python wheel as GitHub artifact when merged into main # Upload the whl file of the specific python version if: github.ref == 'refs/heads/main' - uses: actions/upload-artifact@v2 + uses: actions/upload-artifact@v4 with: name: pysarplus-${{ env.sarplus_version }}-cp${{ matrix.python-version }}-wheel path: ${{ env.PYTHON_ROOT }}/dist/*.whl @@ -104,7 +104,7 @@ jobs: - name: Upload Python source as GitHub artifact when merged into main # Only one pysarplus source tar file is needed if: github.ref == 'refs/heads/main' && matrix.python-version == '3.10' - uses: actions/upload-artifact@v2 + uses: actions/upload-artifact@v4 with: name: pysarplus-${{ env.sarplus_version }}-source path: ${{ env.PYTHON_ROOT }}/dist/*.tar.gz @@ -131,7 +131,7 @@ jobs: hadoop-version: "3.3.1" steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Test run: | @@ -180,7 +180,7 @@ jobs: - name: Upload Scala bundle as GitHub artifact when merged into main if: github.ref == 'refs/heads/main' - uses: actions/upload-artifact@v2 + uses: actions/upload-artifact@v4 with: name: sarplus-${{ env.sarplus_version }}-bundle_2.12-spark-${{ matrix.spark-version }}-jar path: ${{ env.SCALA_ROOT }}/target/scala-2.12/*bundle*.jar diff --git a/.github/workflows/update_documentation.yml b/.github/workflows/update_documentation.yml index a6627dae1..30e2eadf1 100644 --- a/.github/workflows/update_documentation.yml +++ b/.github/workflows/update_documentation.yml @@ -16,10 +16,10 @@ jobs: steps: - name: Checkout repository - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Set up Python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: "3.10" diff --git a/recommenders/models/deeprec/models/sequential/rnn_cell_implement.py b/recommenders/models/deeprec/models/sequential/rnn_cell_implement.py index 8d8f4c782..e48aaafb0 100644 --- a/recommenders/models/deeprec/models/sequential/rnn_cell_implement.py +++ b/recommenders/models/deeprec/models/sequential/rnn_cell_implement.py @@ -601,9 +601,9 @@ def __init__( ): self._build_bias = build_bias - if args is None or (nest.is_sequence(args) and not args): + if args is None or (nest.is_nested(args) and not args): raise ValueError("`args` must be specified") - if not nest.is_sequence(args): + if not nest.is_nested(args): args = [args] self._is_sequence = False else: diff --git a/setup.py b/setup.py index 13e06d06d..758765690 100644 --- a/setup.py +++ b/setup.py @@ -59,7 +59,7 @@ extras_require = { "gpu": [ "nvidia-ml-py3>=7.352.0", - "tensorflow==2.8.4", # FIXME: Temporarily pinned due to issue with TF version > 2.10.1 See #2018 + "tensorflow>=2.8.4,!=2.9.0.*,!=2.9.1,!=2.9.2,!=2.10.0.*,<3", "tf-slim>=1.1.0", "torch>=1.13.1", # for CUDA 11 support "fastai>=1.0.46,<2", @@ -73,6 +73,7 @@ "pytest>=3.6.4", "pytest-cov>=2.12.1", "pytest-mock>=3.6.1", # for access to mock fixtures in pytest + "packaging>=20.9", # for version comparison in test_dependency_security.py ], } # For the brave of heart diff --git a/tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py b/tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py index 4fe1e5f8e..86d0c80ab 100644 --- a/tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py +++ b/tests/ci/azureml_tests/submit_groupwise_azureml_pytest.py @@ -37,7 +37,6 @@ """ import argparse import logging -import glob from azureml.core.authentication import AzureCliAuthentication from azureml.core import Workspace @@ -146,7 +145,6 @@ def setup_persistent_compute_target(workspace, cluster_name, vm_size, max_nodes) def create_run_config( cpu_cluster, - docker_proc_type, add_gpu_dependencies, add_spark_dependencies, conda_pkg_jdk, @@ -165,7 +163,6 @@ def create_run_config( the following: - Reco_cpu_test - Reco_gpu_test - docker_proc_type (str) : processor type, cpu or gpu add_gpu_dependencies (bool) : True if gpu packages should be added to the conda environment, else False add_spark_dependencies (bool) : True if PySpark packages should be @@ -179,7 +176,39 @@ def create_run_config( run_azuremlcompute = RunConfiguration() run_azuremlcompute.target = cpu_cluster run_azuremlcompute.environment.docker.enabled = True - run_azuremlcompute.environment.docker.base_image = docker_proc_type + if not add_gpu_dependencies: + # https://github.com/Azure/AzureML-Containers/blob/master/base/cpu/openmpi4.1.0-ubuntu22.04 + run_azuremlcompute.environment.docker.base_image = "mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04" + else: + run_azuremlcompute.environment.docker.base_image = None + # Use the latest CUDA + # See + # * https://learn.microsoft.com/en-us/azure/machine-learning/how-to-train-with-custom-image?view=azureml-api-1#use-a-custom-dockerfile-optional + # * https://github.com/Azure/AzureML-Containers/blob/master/base/gpu/openmpi4.1.0-cuda11.8-cudnn8-ubuntu22.04 + run_azuremlcompute.environment.docker.base_dockerfile = r""" +FROM nvcr.io/nvidia/cuda:12.3.1-devel-ubuntu22.04 +USER root:root +ENV NVIDIA_VISIBLE_DEVICES all +ENV NVIDIA_DRIVER_CAPABILITIES compute,utility +ENV LANG=C.UTF-8 LC_ALL=C.UTF-8 +ENV DEBIAN_FRONTEND noninteractive +RUN apt-get update && \ + apt-get install -y wget git-all && \ + apt-get clean -y && \ + rm -rf /var/lib/apt/lists/* +# Conda Environment +ENV MINICONDA_VERSION py38_23.3.1-0 +ENV PATH /opt/miniconda/bin:$PATH +ENV CONDA_PACKAGE 23.5.0 +RUN wget -qO /tmp/miniconda.sh https://repo.anaconda.com/miniconda/Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh && \ + bash /tmp/miniconda.sh -bf -p /opt/miniconda && \ + conda install conda=${CONDA_PACKAGE} -y && \ + conda update --all -c conda-forge -y && \ + conda clean -ay && \ + rm -rf /opt/miniconda/pkgs && \ + rm /tmp/miniconda.sh && \ + find / -type d -name __pycache__ | xargs rm -rf +""" # Use conda_dependencies.yml to create a conda environment in # the Docker image for execution @@ -195,6 +224,7 @@ def create_run_config( # install recommenders reco_extras = "dev" + conda_dep.add_conda_package("anaconda::git") if add_gpu_dependencies and add_spark_dependencies: conda_dep.add_channel("conda-forge") conda_dep.add_conda_package(conda_pkg_jdk) @@ -326,13 +356,6 @@ def create_arg_parser(): default="STANDARD_D3_V2", help="Set the size of the VM either STANDARD_D3_V2", ) - # cpu or gpu - parser.add_argument( - "--dockerproc", - action="store", - default="cpu", - help="Base image used in docker container", - ) # Azure subscription id, when used in a pipeline, it is stored in keyvault parser.add_argument( "--subid", action="store", default="123456", help="Azure Subscription ID" @@ -421,16 +444,6 @@ def create_arg_parser(): logger = logging.getLogger("submit_groupwise_azureml_pytest.py") args = create_arg_parser() - - if args.dockerproc == "cpu": - from azureml.core.runconfig import DEFAULT_CPU_IMAGE - - docker_proc_type = DEFAULT_CPU_IMAGE - else: - from azureml.core.runconfig import DEFAULT_GPU_IMAGE - - docker_proc_type = DEFAULT_GPU_IMAGE - cli_auth = AzureCliAuthentication() workspace = setup_workspace( @@ -450,7 +463,6 @@ def create_arg_parser(): run_config = create_run_config( cpu_cluster=cpu_cluster, - docker_proc_type=docker_proc_type, add_gpu_dependencies=args.add_gpu_dependencies, add_spark_dependencies=args.add_spark_dependencies, conda_pkg_jdk=args.conda_pkg_jdk, diff --git a/tests/ci/azureml_tests/test_groups.py b/tests/ci/azureml_tests/test_groups.py index 6c44411fe..f05e27a9f 100644 --- a/tests/ci/azureml_tests/test_groups.py +++ b/tests/ci/azureml_tests/test_groups.py @@ -69,23 +69,23 @@ "tests/smoke/recommenders/recommender/test_deeprec_model.py::test_FFM_iterator", # 0.74s "tests/smoke/recommenders/recommender/test_newsrec_utils.py::test_news_iterator", # 3.04s # - "tests/smoke/recommenders/recommender/test_deeprec_model.py::test_model_lightgcn", # 6.03s # FIXME: Issue with TF version > 2.10.1 See #2018 - "tests/functional/examples/test_notebooks_gpu.py::test_lightgcn_deep_dive_functional", # 19.45s # FIXME: Issue with TF version > 2.10.1 See #2018 + "tests/smoke/recommenders/recommender/test_deeprec_model.py::test_model_lightgcn", # 6.03s + "tests/functional/examples/test_notebooks_gpu.py::test_lightgcn_deep_dive_functional", # 19.45s # - "tests/smoke/recommenders/recommender/test_deeprec_model.py::test_model_sum", # 27.23s # FIXME: Issue with TF version > 2.10.1 See #2018 + # "tests/smoke/recommenders/recommender/test_deeprec_model.py::test_model_sum", # 27.23s # FIXME: Disabled due to the issue with TF version > 2.10.1 See #2018 # "tests/smoke/recommenders/recommender/test_deeprec_model.py::test_model_dkn", # 187.20s "tests/functional/examples/test_notebooks_gpu.py::test_dkn_quickstart_functional", # 1167.93s # - "tests/functional/examples/test_notebooks_gpu.py::test_slirec_quickstart_functional", # 175.00s # FIXME: Issue with TF version > 2.10.1 See #2018 - "tests/smoke/recommenders/recommender/test_deeprec_model.py::test_model_slirec", # 346.72s # FIXME: Issue with TF version > 2.10.1 See #2018 + "tests/functional/examples/test_notebooks_gpu.py::test_slirec_quickstart_functional", # 175.00s + "tests/smoke/recommenders/recommender/test_deeprec_model.py::test_model_slirec", # 346.72s ], "group_gpu_002": [ # Total group time: 1896.76s "tests/unit/examples/test_notebooks_gpu.py::test_gpu_vm", # 0.76s (Always the first test to check the GPU works) "tests/smoke/recommenders/recommender/test_deeprec_model.py::test_model_xdeepfm", # 3.10s # FIXME: https://github.com/microsoft/recommenders/issues/1883 # "tests/smoke/examples/test_notebooks_gpu.py::test_xdeepfm_smoke", # 77.93s - "tests/functional/examples/test_notebooks_gpu.py::test_xdeepfm_functional", # FIXME: Issue with TF version > 2.10.1 See #2018 + "tests/functional/examples/test_notebooks_gpu.py::test_xdeepfm_functional", # "tests/smoke/examples/test_notebooks_gpu.py::test_cornac_bivae_smoke", # 67.84s "tests/functional/examples/test_notebooks_gpu.py::test_cornac_bivae_functional", # 453.21s @@ -426,12 +426,12 @@ ], "group_gpu_002": [ # Total group time: "tests/unit/examples/test_notebooks_gpu.py::test_gpu_vm", # 0.76s (Always the first test to check the GPU works) - "tests/unit/recommenders/models/test_deeprec_model.py::test_xdeepfm_component_definition", # FIXME: Issue with TF version > 2.10.1 See #2018 + # "tests/unit/recommenders/models/test_deeprec_model.py::test_xdeepfm_component_definition", # FIXME: Disabled due to the issue with TF version > 2.10.1 See #2018 "tests/unit/recommenders/models/test_deeprec_model.py::test_dkn_component_definition", "tests/unit/recommenders/models/test_deeprec_model.py::test_dkn_item2item_component_definition", - "tests/unit/recommenders/models/test_deeprec_model.py::test_slirec_component_definition", # FIXME: Issue with TF version > 2.10.1 See #2018 + "tests/unit/recommenders/models/test_deeprec_model.py::test_slirec_component_definition", "tests/unit/recommenders/models/test_deeprec_model.py::test_nextitnet_component_definition", - "tests/unit/recommenders/models/test_deeprec_model.py::test_sum_component_definition", # FIXME: Issue with TF version > 2.10.1 See #2018 + # "tests/unit/recommenders/models/test_deeprec_model.py::test_sum_component_definition", # FIXME: Disabled due to the issue with TF version > 2.10.1 See #2018 "tests/unit/recommenders/models/test_deeprec_model.py::test_lightgcn_component_definition", "tests/unit/recommenders/models/test_deeprec_utils.py::test_prepare_hparams", "tests/unit/recommenders/models/test_deeprec_utils.py::test_load_yaml_file", @@ -449,7 +449,7 @@ "group_notebooks_gpu_002": [ # Total group time: 241.15s "tests/unit/examples/test_notebooks_gpu.py::test_gpu_vm", # 0.76s (Always the first test to check the GPU works) "tests/unit/examples/test_notebooks_gpu.py::test_wide_deep", - "tests/unit/examples/test_notebooks_gpu.py::test_xdeepfm", # FIXME: Issue with TF version > 2.10.1 See #2018 + "tests/unit/examples/test_notebooks_gpu.py::test_xdeepfm", "tests/unit/examples/test_notebooks_gpu.py::test_gpu_vm", ], } diff --git a/tests/security/test_dependency_security.py b/tests/security/test_dependency_security.py index 82a4f0596..fccb0b22b 100644 --- a/tests/security/test_dependency_security.py +++ b/tests/security/test_dependency_security.py @@ -7,6 +7,8 @@ import numpy as np import pandas as pd +from packaging.version import Version + try: import tensorflow as tf import torch @@ -16,17 +18,17 @@ def test_requests(): # Security issue: https://github.com/psf/requests/releases/tag/v2.31.0 - assert requests.__version__ >= "2.31.0" + assert Version(requests.__version__) >= Version("2.31.0") def test_numpy(): # Security issue: https://github.com/advisories/GHSA-frgw-fgh6-9g52 - assert np.__version__ >= "1.13.3" + assert Version(np.__version__) >= Version("1.13.3") def test_pandas(): # Security issue: https://github.com/advisories/GHSA-cmm9-mgm5-9r42 - assert pd.__version__ >= "1.0.3" + assert Version(pd.__version__) >= Version("1.0.3") @pytest.mark.gpu @@ -34,10 +36,10 @@ def test_tensorflow(): # Security issue: https://github.com/advisories/GHSA-w5gh-2wr2-pm6g # Security issue: https://github.com/advisories/GHSA-r6jx-9g48-2r5r # Security issue: https://github.com/advisories/GHSA-xxcj-rhqg-m46g - assert tf.__version__ >= "2.8.4" + assert Version(tf.__version__) >= Version("2.8.4") @pytest.mark.gpu def test_torch(): # Security issue: https://github.com/advisories/GHSA-47fc-vmwq-366v - assert torch.__version__ >= "1.13.1" + assert Version(torch.__version__) >= Version("1.13.1")