Error when trying to download 'fr_dep_news_trf' #7361
-
How to reproduce the problemI was upgrading spacy to 3.0 in a new environment and followed these commands English and german worked fine but for french I got an error related to sentencepiece from what I understand. Building wheels for collected packages: sentencepiece
Building wheel for sentencepiece (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\Tristan\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Tristan\\AppData\\Local\\Temp\\pip-install-zfv2xj07\\sentencepiece_6b32d9bf018241f9ac2adf4b5f2a8a52\\setup.py'"'"'; __file__='"'"'C:\\Users\\Tristan\\AppData\\Local\\Temp\\pip-install-zfv2xj07\\sentencepiece_6b32d9bf018241f9ac2adf4b5f2a8a52\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\Tristan\AppData\Local\Temp\pip-wheel-etrnpchd'
cwd: C:\Users\Tristan\AppData\Local\Temp\pip-install-zfv2xj07\sentencepiece_6b32d9bf018241f9ac2adf4b5f2a8a52\
Complete output (15 lines):
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
copying sentencepiece.py -> build\lib.win-amd64-3.9
running build_ext
building '_sentencepiece' extension
creating build\temp.win-amd64-3.9
creating build\temp.win-amd64-3.9\Release
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.27.29110\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0\include -IC:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.27.29110\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt /EHsc /Tpsentencepiece_wrap.cxx /Fobuild\temp.win-amd64-3.9\Release\sentencepiece_wrap.obj /MT /I..\build\root\include
clÿ: Ligne de commande warning D9025ÿ: substitution de '/MD' par '/MT'
sentencepiece_wrap.cxx
sentencepiece_wrap.cxx(2777): fatal error C1083: Impossible d'ouvrir le fichier includeÿ: 'sentencepiece_processor.h'ÿ: No such file or directory
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
----------------------------------------
ERROR: Failed building wheel for sentencepiece
Running setup.py clean for sentencepiece
Failed to build sentencepiece
Installing collected packages: sentencepiece, protobuf, fr-dep-news-trf
Attempting uninstall: sentencepiece
Found existing installation: sentencepiece 0.1.95
Uninstalling sentencepiece-0.1.95:
Successfully uninstalled sentencepiece-0.1.95
Running setup.py install for sentencepiece ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\Tristan\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Tristan\\AppData\\Local\\Temp\\pip-install-zfv2xj07\\sentencepiece_6b32d9bf018241f9ac2adf4b5f2a8a52\\setup.py'"'"'; __file__='"'"'C:\\Users\\Tristan\\AppData\\Local\\Temp\\pip-install-zfv2xj07\\sentencepiece_6b32d9bf018241f9ac2adf4b5f2a8a52\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\Tristan\AppData\Local\Temp\pip-record-d6jde4x3\install-record.txt' --single-version-externally-managed --user --prefix= --compile --install-headers 'C:\Users\Tristan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\Include\sentencepiece'
cwd: C:\Users\Tristan\AppData\Local\Temp\pip-install-zfv2xj07\sentencepiece_6b32d9bf018241f9ac2adf4b5f2a8a52\
Complete output (15 lines):
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
copying sentencepiece.py -> build\lib.win-amd64-3.9
running build_ext
building '_sentencepiece' extension
creating build\temp.win-amd64-3.9
creating build\temp.win-amd64-3.9\Release
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.27.29110\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0\include -IC:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0\include -IC:\Program Files (x86)\Microsoft Visual
Studio\2019\BuildTools\VC\Tools\MSVC\14.27.29110\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt /EHsc /Tpsentencepiece_wrap.cxx /Fobuild\temp.win-amd64-3.9\Release\sentencepiece_wrap.obj /MT /I..\build\root\include
clÿ: Ligne de commande warning D9025ÿ: substitution de '/MD' par '/MT'
sentencepiece_wrap.cxx
sentencepiece_wrap.cxx(2777): fatal error C1083: Impossible d'ouvrir le fichier includeÿ: 'sentencepiece_processor.h'ÿ: No such file or directory
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
----------------------------------------
Rolling back uninstall of sentencepiece
Moving to c:\users\tristan\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages\sentencepiece-0.1.95.dist-info\
from C:\Users\Tristan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\~entencepiece-0.1.95.dist-info
Moving to c:\users\tristan\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages\sentencepiece\
from C:\Users\Tristan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\~entencepiece
ERROR: Command errored out with exit status 1: 'C:\Users\Tristan\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Tristan\\AppData\\Local\\Temp\\pip-install-zfv2xj07\\sentencepiece_6b32d9bf018241f9ac2adf4b5f2a8a52\\setup.py'"'"'; __file__='"'"'C:\\Users\\Tristan\\AppData\\Local\\Temp\\pip-install-zfv2xj07\\sentencepiece_6b32d9bf018241f9ac2adf4b5f2a8a52\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\Tristan\AppData\Local\Temp\pip-record-d6jde4x3\install-record.txt' --single-version-externally-managed --user --prefix= --compile --install-headers 'C:\Users\Tristan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\Include\sentencepiece' Check the logs for full command output. Your Environment
|
Beta Was this translation helpful? Give feedback.
Replies: 3 comments
-
Hi, this error is that it's failing to compile Alternatively, if you'd like to continue using python 3.9 and you're comfortable compiling things, you can clone the github repo (https://github.com/google/sentencepiece), check out Whatever you do, be sure that you end up with |
Beta Was this translation helpful? Give feedback.
-
Indeed this seemed to be the issue, thank you! However I do realize that this new french pipeline doesn't support NER, is this a planned feature for the future? |
Beta Was this translation helpful? Give feedback.
-
No, not unless we have a corpus that contains all the required types of annotation on the same documents. It doesn't work well to train a transformer pipeline with documents with "partial" annotation, where you have some docs just with tags+parses and some docs just with NER. |
Beta Was this translation helpful? Give feedback.
Hi, this error is that it's failing to compile
sentencepiece
, because the sdist for the required version is broken (it doesn't include the necessary source code). The required version,v0.1.91
, was released before python 3.9 came out, so it doesn't have precompiled binary wheels throughpip
. I think the easiest solution is use python 3.8 instead, where there should be a binary wheel available.Alternatively, if you'd like to continue using python 3.9 and you're comfortable compiling things, you can clone the github repo (https://github.com/google/sentencepiece), check out
v0.1.91
and follow the instructions in the README: https://github.com/google/sentencepiece/tree/v0.1.91#installation. …