We currently rely on a patchwork of python dependency handling in development and actual deployments with some installation going through pip with:
pip install -r requirements.txt
pip install -r recommended.txt
pip install -r local-requirements.txt
and some handled with explicit versioned package installation as in
pip install 'paramiko==VERSION'
pip install 'openstack==VERSION'
...
in the Dockerfiles in docker-migrid.

We also have overlap in requirements.txt and recommended.txt, which only make things more complex.

We decided to restructure the existing requirements files into a single main one with all mandatory requirements needed for all migrid installs, plus individual requirements-FEATURE.txt extensions that only include requirements to enable a particular FEATURE like sftp, jupyter, cloud, etc.
That will allow us to generally use
pip install -r requirements.txt
followed by
pip install -r requirements-FEATURE.txt
for each enabled FEATURE and hopefully reduce the requirements files overlap to a minimum.

We will want to then gradually move to use that structure both in the development/testing use e.g. in containerized unit tests and linting as well as in the docker-migrid Dockerfiles.

Please note that we will stick to installing various non-python library dependencies (openssl, cracklib, etc.) as well as the python dependencies we can avoid maintaining ourselves through the Linux distro channels.
As long as our requirements*.txt files don't directly require versions conflicting with those OS package versions, that will work fine and leave the OS-installed version alone. In other words we just have to avoid requirements*.txt specifying that package X must be a version bigger than what the OS provides for those packages we intend the OS package to provide.