CloudCatalog, pyRFU, SWxSOC forced to use boto3==1.37.1

[sapols]

The pipeline detected a dependency conflict between Cloudcatalog and pySPEDAS on Jan 30, 2025. The offending package was botocore: pySPEDAS wanted botocore>=1.36.0,<1.36.2 while Cloudcatalog wanted botocore>=1.36.9,<1.37.0.

Upon further inspection, the conflict was resolvable and really a result of my pipeline's single-package dependency resolution; you could call it faulty logic, or at least lacking. The problem ultimately came from Cloudcatalog's boto3; both Cloudcatalog and pySPEDAS accept any version of boto3, but the way my script ran, Cloudcatalog was pulling in the latest boto3==1.36.9. This is too high for the downstream pySPEDAS dependencies to support.

So, in this commit, I added a custom script so Cloudcatalog would pull in boto3==1.36.1.

This fixed the reported dependency conflict and allowed the pipeline to succeed once more. But we should revisit this decision if it ever causes problems.

[sapols]

Well in this commit I bumped the version to 1.37.1, because pySPEDAS's requirement changed to botocore>=1.37.0,<1.37.2. This might be a game of cat and mouse I play for a while if pySPEDAS's requirements keep changing.

[sapols]

And well in this commit I gave pyRFU the same treatment as Cloudcatalog for the same reason.

All 3 of these packages have fairly loose boto3 requirements, but now whatever range pySPEDAS pulls in is effectively leading the pack. Guess it will be a game of cat and mouse for a while after all.

[sapols]

We're now using get-dep-tree-for-package-w-boto.sh instead of get-dep-tree-for-package.sh for packages with this problem (currently cloudcatalog, pyrfu and swxsoc).

[sapols]

Note that in this commit I bumped the boto3/botocore versions from 1.37.1 to 1.38.23 because pySPEDAS's range changed to >=1.38.23,<1.38.28 which conflicted with the packages using get-dep-tree-for-package-w-boto.sh (>=1.37.1,<1.38.0).

[sapols]

And in this commit I bumped the boto3/botocore versions from 1.38.23 to 1.39.9 because pySPEDA's range changed to >=1.39.9,<1.39.12. I also bumped s3transfer==0.13.0 because 0.11.1 wasn't compatible anymore.