Improve conditional imports

[whophil]

Purpose

This PR improves the conditional imports.

• Logic for conditional imports is unified across all optimizers
• Instead of totally overriding sys.path before attempting import, this prepends to sys.path
  • This makes more sense to me than the current behavior. Any external system which builds/installs an importable extension module should put this into a standard place - i.e., somewhere on the default sys.path. I regret suggesting PYOPTSPARSE_IMPORT_SNOPT_FROM, but we can keep it for now.
• Errors encountered by try_import_compiled_module_from are returned so that they can be re-raised later with a meaningful traceback.

Expected time until merged

1 week

Type of change

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (non-backwards-compatible fix or feature)
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no API changes)
• Documentation update
• Maintenance update
• Other (please describe)

This is technically breaking if you considered pyOpt_utils.try_import_module_from_path to be part of the public API.

Testing

Checklist

• I have run ruff check and ruff format to make sure the Python code adheres to PEP-8 and is consistently formatted
• I have formatted the Fortran code with fprettify or C/C++ code with clang-format as applicable
• I have run unit and regression tests which pass locally with my changes
• I have added new tests that prove my fix is effective or that my feature works
• I have added necessary documentation

[codecov]

Codecov Report

❌ Patch coverage is 89.23077% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.97%. Comparing base (bdea96c) to head (e7c46ac).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
pyoptsparse/pyOpt_utils.py	91.30%	2 Missing ⚠️
pyoptsparse/pyCONMIN/pyCONMIN.py	75.00%	1 Missing ⚠️
pyoptsparse/pyIPOPT/pyIPOPT.py	75.00%	1 Missing ⚠️
pyoptsparse/pyNSGA2/pyNSGA2.py	75.00%	1 Missing ⚠️
pyoptsparse/pyPSQP/pyPSQP.py	75.00%	1 Missing ⚠️
pyoptsparse/pySLSQP/pySLSQP.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #458       +/-   ##
===========================================
- Coverage   86.22%   74.97%   -11.25%     
===========================================
  Files          24       24               
  Lines        3419     3429       +10     
===========================================
- Hits         2948     2571      -377     
- Misses        471      858      +387     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kanekosh]

Thanks! Just a minor comment from me.

Linting and formatting checks are failing because we updated the Azure setup to switch to ruff and pre-commit, but we haven't merged the code update PR #455 yet. So please don't worry about it.

[kanekosh]

I believe this should be import_module("snopt", ["/some/path"]), otherwise import_module loops over each letter of /some/path.
Maybe it'd be nice to add an explicit check on the path type in the import_module function?

[ewu63]

Left minor comments, looks good to me.

[ewu63]

This is to handle an empty str set to the var?

[ewu63]

This PR isn't testing all the optimizers since it came from a public fork, can someone in the lab test this out? Maybe @A-CGray ?

[A-CGray]

This PR isn't testing all the optimizers since it came from a public fork, can someone in the lab test this out? Maybe @A-CGray ?

Tested this on our latest private image with all optimisers, everything passes

[ewu63]

Looks good to me, thanks Phil! Just looks like ruff wanted an extra line in one file.