SpaCy model won't import in AWS Glue job (python shell)

[SalmanZafar-DataScience]

How to reproduce the problem

I can't import 2 spacy models en_core_web_sm and de_core_news_sm into an AWS Glue job that I created on python shell.

1. Previously, I imported spacy and all other packages by defining them in setup.py by doing this:

from setuptools import setup
setup(name=spacy,
packages=[ ],
version=2.3.1,
install_requires=['spacy']
)

2. Running the following command python setup.py bdist_egg creates an .egg file which is then uploaded in a S3 bucket.
3. Path of that .egg file in S3 Bucket is then mentioned in the Glue job. This way all the packages are imported without any issues.

The same process when gets applied to the spacy models en_core_web_sm/de_core_news_sm won't work and throws an error:

_Can't find model 'en_core_web_sm'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory._

Your Environment

• Operating System: macOS Catalina
• Python Version Used: 3.7.6
• spaCy Version Used: 2.3.2

[svlandeg]

You have to install the models separately - you don't get them automatically when installing spaCy.
You can either do

python -m spacy download en_core_web_sm 

or install via pip by using the direct URL (have a peek at the compatibility table to adjust the version as needed):

pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.1/en_core_web_sm-2.3.1.tar.gz

I haven't worked with Glue myself yet - but hopefully this helps to set up your installation?