distutils issues on macOS Monterey + Python 3.8

[pauldmccarthy]

• I read the conda-forge documentation and could not find the solution for my problem there.

Issue:

Howdy, I'm a bit out of my depth here; a colleague has reported having distutils-related issues with a conda-forge Python 3.8 conda installation, that does not occur with Python 3.9 or 3.10, nor with python 3.8/3.9/3.10 installed from defaults.

The problem is that attempts to access distutils.util result in AttributeError: module 'distutils' has no attribute 'util'. This can be reliably reproduced by my colleague on macOS Monterey, but I am unable to reproduce it on macOS Catalina. Steps to reproduce are as follows:

Start with a fresh environment and Miniconda installation:

eval $(env|sort|grep PYTHON|cut -d '=' -f 1|xargs -n 1 echo "unset ")
eval $(env|sort|grep CONDA| cut -d '=' -f 1|xargs -n 1 echo "unset ")
curl -L -O https://repo.anaconda.com/miniconda/Miniconda3-py39_4.10.3-MacOSX-x86_64.sh
sh Miniconda3-py39_4.10.3-MacOSX-x86_64.sh  -b -p  ./fresh_miniconda

Create a Python 3.8 environment:

./fresh_miniconda/bin/conda create -y -c conda-forge -p ./py38 python=3.8

Collecting package metadata (current_repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.10.3
  latest version: 4.11.0

Please update conda by running

    $ conda update -n base -c defaults conda



## Package Plan ##

  environment location: ${HOME}/py38

  added / updated specs:
    - python=3.8


The following NEW packages will be INSTALLED:

  ca-certificates    conda-forge/osx-64::ca-certificates-2021.10.8-h033912b_0
  libcxx             conda-forge/osx-64::libcxx-12.0.1-habf9029_1
  libffi             conda-forge/osx-64::libffi-3.4.2-h0d85af4_5
  libzlib            conda-forge/osx-64::libzlib-1.2.11-h9173be1_1013
  ncurses            conda-forge/osx-64::ncurses-6.2-h2e338ed_4
  openssl            conda-forge/osx-64::openssl-3.0.0-h0d85af4_2
  pip                conda-forge/noarch::pip-21.3.1-pyhd8ed1ab_0
  python             conda-forge/osx-64::python-3.8.12-h43ca1e7_2_cpython
  python_abi         conda-forge/osx-64::python_abi-3.8-2_cp38
  readline           conda-forge/osx-64::readline-8.1-h05e3726_0
  setuptools         conda-forge/osx-64::setuptools-60.2.0-py38h50d1736_0
  sqlite             conda-forge/osx-64::sqlite-3.37.0-h23a322b_0
  tk                 conda-forge/osx-64::tk-8.6.11-h5dbffcc_1
  wheel              conda-forge/noarch::wheel-0.37.1-pyhd8ed1ab_0
  xz                 conda-forge/osx-64::xz-5.2.5-haf1e3a3_1
  zlib               conda-forge/osx-64::zlib-1.2.11-h9173be1_1013


Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate ${HOME}/py38
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Then try to use distutils.util:

$ ./py38/bin/python -c "import distutils; print(distutils)"
<module 'distutils' from '${HOME}/py38/lib/python3.8/distutils/__init__.py'>

$ ./py38/bin/python -c "import distutils.util; print(distutils.util)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: module 'distutils' has no attribute 'util'

Repeating the above process with a Python 3.9 environment:

./fresh_miniconda/bin/conda create -y -c conda-forge -p ./py39 python=3.9

Collecting package metadata (current_repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.10.3
  latest version: 4.11.0

Please update conda by running

    $ conda update -n base -c defaults conda



## Package Plan ##

  environment location: ${HOME}/py39

  added / updated specs:
    - python=3.9


The following NEW packages will be INSTALLED:

  bzip2              conda-forge/osx-64::bzip2-1.0.8-h0d85af4_4
  ca-certificates    conda-forge/osx-64::ca-certificates-2021.10.8-h033912b_0
  libffi             conda-forge/osx-64::libffi-3.4.2-h0d85af4_5
  libzlib            conda-forge/osx-64::libzlib-1.2.11-h9173be1_1013
  ncurses            conda-forge/osx-64::ncurses-6.2-h2e338ed_4
  openssl            conda-forge/osx-64::openssl-3.0.0-h0d85af4_2
  pip                conda-forge/noarch::pip-21.3.1-pyhd8ed1ab_0
  python             conda-forge/osx-64::python-3.9.9-h38b4d05_0_cpython
  python_abi         conda-forge/osx-64::python_abi-3.9-2_cp39
  readline           conda-forge/osx-64::readline-8.1-h05e3726_0
  setuptools         conda-forge/osx-64::setuptools-60.2.0-py39h6e9494a_0
  sqlite             conda-forge/osx-64::sqlite-3.37.0-h23a322b_0
  tk                 conda-forge/osx-64::tk-8.6.11-h5dbffcc_1
  tzdata             conda-forge/noarch::tzdata-2021e-he74cb21_0
  wheel              conda-forge/noarch::wheel-0.37.1-pyhd8ed1ab_0
  xz                 conda-forge/osx-64::xz-5.2.5-haf1e3a3_1
  zlib               conda-forge/osx-64::zlib-1.2.11-h9173be1_1013


Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate ${HOME}/py39
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Accessing distutils.util now works:

$ ./py39/bin/python -c "import distutils; print(distutils)"
<module 'distutils' (<_distutils_hack.DistutilsMetaFinder.spec_for_distutils.<locals>.DistutilsLoader object at 0x102893d30>)>

$ ./py39/bin/python -c "import distutils.util; print(distutils.util)"
<module 'distutils.util' from '${HOME}/py39/lib/python3.9/site-packages/setuptools/_distutils/util.py'>

So it looks like the _distutils_hack (which I know absolutely nothing about) is not being found in Python 3.8 environmments on Monterey for some reason.

Any thoughts? Thanks!

./fresh_miniconda/bin/conda info

     active environment : None
       user config file : ${HOME}/.condarc
 populated config files : ${HOME}/.condarc
          conda version : 4.10.3
    conda-build version : not installed
         python version : 3.9.5.final.0
       virtual packages : __osx=10.16=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : ${HOME}/fresh_miniconda  (writable)
      conda av data dir : ${HOME}/fresh_miniconda/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/intel/osx-64
                          https://conda.anaconda.org/intel/noarch
                          https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : ${HOME}/fresh_miniconda/pkgs
                          ${HOME}/.conda/pkgs
       envs directories : ${HOME}/fresh_miniconda/envs
                          ${HOME}/.conda/envs
               platform : osx-64
             user-agent : conda/4.10.3 requests/2.26.0 CPython/3.9.5 Darwin/21.2.0 OSX/10.16
                UID:GID : 501:20
             netrc file : None
           offline mode : False

[wtclarke]

Hi I'm the colleague that @pauldmccarthy mentioned.

To add a little more information to this. Pinning the setuptools version to that which comes with python 3.8 in the default channel (setuptools-58.0.4) results in successful import of distutils.util.

./fresh_miniconda/bin/conda create -y -c conda-forge -p ./py38_cf_st58 python=3.8 setuptools=58
./py38_cf_st58/bin/python -c "import distutils.util; print(distutils.util)"
<module 'distutils.util' from '${HOME}/py38_cf_st58/lib/python3.8/distutils/util.py'>

This is in contrast to setuptools-60.2.0 which is downloaded with the conda-forge channel specified.

[isuruf]

I can't reproduce it, so there's no way to help you unless you can provide access to a machine where this can be reproduced.