Can't import en_core_web_trf: AttributeError: module 'spacy.lang.en' has no attribute '__all__'

[rjurney]

How to reproduce the behaviour

I am using what loads at this time as spacy-nightly[transformers,cuda100], on Google Colab's Python3.

The following code was run:

!pip install snorkel spacy-nightly[transformers,cuda100] --pre

import spacy
import spacy_transformers

from spacy.cli import download
download("en_core_web_trf")

nlp = spacy.load("en_core_web_trf")

The following exception occurs:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-19-911d29e313e9> in <module>()
----> 1 nlp = spacy.load("en_core_web_trf")

7 frames
/usr/local/lib/python3.6/dist-packages/spacy/util.py in get_lang_class(lang)

AttributeError: module 'spacy.lang.en' has no attribute '__all__'

Your Environment

• Operating System: (Colab) Linux 5a8332f5cac4 4.19.112+ #1 SMP Thu Jul 23 08:00:38 PDT 2020 x86_64 x86_64 x86_64 GNU/Linux
• Python Version Used: Python 3.6.9
• spaCy Version Used: spacy-nightly==3.0.0rc5
• Environment Information: Rain.

[ines]

Hi! This is a super strange error and I've never seen this before 🤔 I just tried it in Colab and I can't reproduce it, it all runs fine for me and loads as expected? Maybe something got corrupted when Colab installed spacy-nightly on top of the spaCy version it comes pre-installed with?

If you still have your Colab env open, what happens if you import spacy.lang.en (which is the module containing the language setup that spaCy lazy-loads when you load the model)?

[honnibal]

This is a long shot, but did you try restarting the kernel after doing the pip install?

[rjurney]

You were correct. I had installed spacy-nightly without uninstalling the included spacy first. Then I installed spacy-nightly and removed the original spacy, which messed things up.

The solution to install this in Colab is:

!yes | pip uninstall spacy
!pip install spacy-nightly[cuda100,transformers] --pre

After this the following code runs fine:

import spacy
import spacy_transformers
from spacy.cli import download


download("en_core_web_trf")
nlp = spacy.load("en_core_web_trf")

However, download now seems superfluous according to the debug output, since load can download.

@honnibal is there a relevant place in the documentation to add this? I would like to make my first PR if there is :)

[ines]

Glad it worked! 👍

spacy.load only loads pipelines from a package name or local path – it doesn't download anything. That only happens in the download function/command. Under the hood, it delegates to pip install and installs the downloaded package in your Python environment. If the same package and version is already installed, the download will be skipped.

[rjurney]

@ines could I update the install instructions for Colab?