python3-pip is not installed automatically even though Python is activated for the workspace

[moooeeeep]

python3-pip is not installed automatically when a new workspace is bootstrapped and updated, even though Python is enabled.

When I set up a new workspace on a fresh Ubuntu (I used 18.04), the package python3-pip is not added as a dependency and is not installed. Thus the setup won't be able to install pip osdeps, and executing install/bin/pip will fail with an error (/usr/bin/python3: No module named pip).

We need to add the package to the layout explicitly to make sure it is actually available:

package_sets:
    - github: rock-core/package_set
    - github: rock-core/rock-package_set
layout:
    - python3-pip

I think it should be installed automatically.

[g-arjones]

I have to disagree on this one. If you don't have a python package in your workspace there's absolutely no reason to install pip.

The current behavior is by design.

[g-arjones]

The same goes for activating python by the way, people that don't use python shouldn't be bothered by python-related prompts.

[moooeeeep]

At the point where this problem occurs, the user has already specified that they would like to use Python. Autoproj adds the shims to install/bin, but doesn't install the (at that point more or less mandatory) dependency to python3-pip (you could install Python packages, but no pip osdeps).

I acknowledge your argument against pulling in Python support for everyone in response to this issue though.

[g-arjones]

At the point where this problem occurs, the user has already specified that they would like to use Python. Autoproj adds the shims to install/bin

Is autoproj asking whether the user would like to use python when there are no python packages in the workspace? That's not supposed to happen.

but doesn't install the (at that point more or less mandatory) dependency to python3-pip

Again, autoproj should only install pip if necessary (i.e you have a python source package that has a setup.py in it or you need an osdeps that is managed by PipManager).

[moooeeeep]

Is autoproj asking whether the user would like to use python when there are no python packages in the workspace? That's not supposed to happen.

No, but the rock.core package set does, which is imported directly or indirectly in most buildconfs or package sets that I have come across so far anyways.

autoproj should only install pip if necessary

My point is, it doesn't do this (correctly). On Ubuntu 18.04 it installs python-pip, even though the user most likely needs python3-pip.

Example buildconf: https://github.com/moooeeeep/my_buildconf

The relevant output of what happens:

[...]
  checked out my_python_pkg
  building initial autoproj import log, this may take a while
  installing OS packages: python-pip
  Do you want to activate python? [no] yes
  Select the path to the python executable [/usr/bin/python3] 
  installing/updating Python dependencies: pillow
Command failed
autoproj: failed in osdeps phase
    'pip install --user pillow' returned status 1
    see /home/dfki.uni-bremen.de/fmueller/ws/install/log/autoproj-osdeps.log for details
    last 1 lines are:

    /usr/bin/python3: No module named pip

Here, pip is installed for the wrong Python version (because of this spec probably) and therefore not available to actually install the osdep.

To reproduce this, you might need to uninstall python3-pip (it is not installed on a fresh Ubuntu installation).

[moooeeeep]

Also note this error, which also occurs with that setup: rock-core/autobuild#106

[moooeeeep]

And when you add a dependency to opencv-python, you can also reproduce this issue: #338

[g-arjones]

but the rock.core package set does

I thought we were talking about autoproj only (I don't use rock), sorry about that.

My point is, it doesn't do this (correctly)

To be honest, I really believe that the inevitable long term solution is to deprecate python2 once and for all (this is actually a point I made when the whole python activation code was added to autoproj). Those folks using python2 can always override the osdeps in their buildconfs/package sets (which you can do to fix your issue as well, just override pip to be python3-pip instead of python-pip in your configuration).

[moooeeeep]

I thought we were talking about autoproj only (I don't use rock), sorry about that.

Apparantly I assumed these two are coupled more tightly than they actually are (again). Sorry for the confusion.

To be honest, I really believe that the inevitable long term solution is to deprecate python2 once and for all (this is actually a point I made when the whole python activation code was added to autoproj).

I'd second this.

And thanks for the suggestion with the override, maybe that will do for the time being.

[moooeeeep]

The problem exists only on Ubuntu 18.04, where only Python 2 stuff is installed although Python 3 support is selected.

This causes errors with installations of pip dependencies and autobuild packages via setuptools.

But as Ubuntu 18.04 support is losing relevance, this can probably be ignored by now.