Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GPT-3 Docker encountered an build error #5

Open
jndinesh opened this issue Dec 17, 2023 · 2 comments
Open

GPT-3 Docker encountered an build error #5

jndinesh opened this issue Dec 17, 2023 · 2 comments

Comments

@jndinesh
Copy link

Encountered failures while trying to docker build GPT3 using Docker. Please refer to the build output for more information.


WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: python -m pip install --upgrade pip
Removing intermediate container b9082a92f080
 ---> 237996c866f4
Step 18/33 : ARG NEMO_REVISION=bd0b818ac2eb420b1cea19eb8bcc6215c87b06d0
 ---> Running in 5d981d2408c0
Removing intermediate container 5d981d2408c0
 ---> 5e409f983c81
Step 19/33 : ENV CUSTOM_NEMO_REVISION ${NEMO_REVISION}
 ---> Running in 9396427db2bc
Removing intermediate container 9396427db2bc
 ---> 1921a4a2de6f
Step 20/33 : ARG NEMO_BASE_VERSION=r1.20.0
 ---> Running in af63b5e5c910
Removing intermediate container af63b5e5c910
 ---> 8130644c5d1e
Step 21/33 : ENV CUSTOM_NEMO_BASE_VERSION ${NEMO_BASE_VERSION}
 ---> Running in a70505e89166
Removing intermediate container a70505e89166
 ---> 52cfb9c0061b
Step 22/33 : RUN if [ "${NEMO_REVISION}" == SKIP ]; then       if [ -d /opt/bignlp/NeMo ]; then         ln -s /opt/bignlp/NeMo       ; else         echo "Error: NEMO_REVISION=SKIP but there is no BigNLP NeMo installation in base image." &&         exit 1       ; fi     ; else       git clone https://github.com/NVIDIA/NeMo.git &&       cd NeMo &&       echo NEMO_REVISION=${NEMO_REVISION} &&       git checkout ${NEMO_REVISION} &&       pip uninstall -y nemo-toolkit &&       pip install "cython<3.0.0" &&       pip install --no-build-isolation -e ".[nlp]"     ; fi
 ---> Running in cf031a6a6c08
Cloning into 'NeMo'...
NEMO_REVISION=bd0b818ac2eb420b1cea19eb8bcc6215c87b06d0
Note: switching to 'bd0b818ac2eb420b1cea19eb8bcc6215c87b06d0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at bd0b818ac Broadcast loss only when using pipeline parallelism and within the pipeline parallel domain (#7576)
WARNING: Skipping nemo-toolkit as it is not installed.
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting cython<3.0.0
  Obtaining dependency information for cython<3.0.0 from https://files.pythonhosted.org/packages/f8/26/ca0f1bb049b83c25cafa39f3fa5287c826a6ab36e665c906209e07f4deac/Cython-0.29.36-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata
  Downloading Cython-0.29.36-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (3.1 kB)
Downloading Cython-0.29.36-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (1.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.9/1.9 MB 13.5 MB/s eta 0:00:00
Installing collected packages: cython
  Attempting uninstall: cython
    Found existing installation: Cython 3.0.2
    Uninstalling Cython-3.0.2:
      Successfully uninstalled Cython-3.0.2
Successfully installed cython-0.29.36
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: python -m pip install --upgrade pip
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Obtaining file:///workspace/NeMo
  Checking if build backend supports build_editable: started
  Checking if build backend supports build_editable: finished with status 'done'
  Preparing editable metadata (pyproject.toml): started
  Preparing editable metadata (pyproject.toml): finished with status 'done'
Collecting huggingface-hub (from nemo-toolkit==1.20.0)
  Obtaining dependency information for huggingface-hub from https://files.pythonhosted.org/packages/05/09/1945ca6ba3ad8ad6e2872ba682ce8d68c5e63c8e55458ed8ab4885709f1d/huggingface_hub-0.19.4-py3-none-any.whl.metadata
  Downloading huggingface_hub-0.19.4-py3-none-any.whl.metadata (14 kB)
Requirement already satisfied: numba in /usr/local/lib/python3.10/dist-packages (from nemo-toolkit==1.20.0) (0.57.1+1.g5fba9aa8f)
Requirement already satisfied: numpy<1.24,>=1.22 in /usr/local/lib/python3.10/dist-packages (from nemo-toolkit==1.20.0) (1.22.2)
Requirement already satisfied: onnx>=1.7.0 in /usr/local/lib/python3.10/dist-packages (from nemo-toolkit==1.20.0) (1.14.0)
Requirement already satisfied: python-dateutil in /usr/local/lib/python3.10/dist-packages (from nemo-toolkit==1.20.0) (2.8.2)
Collecting ruamel.yaml (from nemo-toolkit==1.20.0)
  Obtaining dependency information for ruamel.yaml from https://files.pythonhosted.org/packages/57/db/4b74a29f5fd175aea3ff0d95f8230d9bb8a54e38d963c6f96065b9e2eef3/ruamel.yaml-0.18.5-py3-none-any.whl.metadata
  Downloading ruamel.yaml-0.18.5-py3-none-any.whl.metadata (23 kB)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from nemo-toolkit==1.20.0) (1.2.0)
Collecting setuptools==65.5.1 (from nemo-toolkit==1.20.0)
  Downloading setuptools-65.5.1-py3-none-any.whl (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 25.6 MB/s eta 0:00:00
Requirement already satisfied: tensorboard in /usr/local/lib/python3.10/dist-packages (from nemo-toolkit==1.20.0) (2.9.0)
Collecting text-unidecode (from nemo-toolkit==1.20.0)
  Downloading text_unidecode-1.3-py2.py3-none-any.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.2/78.2 kB 152.6 MB/s eta 0:00:00
Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (from nemo-toolkit==1.20.0) (2.1.0a0+32f93b1)
Requirement already satisfied: tqdm>=4.41.0 in /usr/local/lib/python3.10/dist-packages (from nemo-toolkit==1.20.0) (4.66.1)
Collecting wget (from nemo-toolkit==1.20.0)
  Downloading wget-3.2.zip (10 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting wrapt (from nemo-toolkit==1.20.0)
  Obtaining dependency information for wrapt from https://files.pythonhosted.org/packages/49/83/b40bc1ad04a868b5b5bcec86349f06c1ee1ea7afe51dc3e46131e4f39308/wrapt-1.16.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading wrapt-1.16.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting boto3 (from nemo-toolkit==1.20.0)
  Obtaining dependency information for boto3 from https://files.pythonhosted.org/packages/0a/8e/fab7dbeb7251d8ff2f63812f3fdf57eef87d0a37e1422b31cb6b72082dd4/boto3-1.34.2-py3-none-any.whl.metadata
  Downloading boto3-1.34.2-py3-none-any.whl.metadata (6.6 kB)
Requirement already satisfied: einops in /usr/local/lib/python3.10/dist-packages (from nemo-toolkit==1.20.0) (0.6.1)
Collecting faiss-cpu (from nemo-toolkit==1.20.0)
  Downloading faiss_cpu-1.7.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.6/17.6 MB 117.7 MB/s eta 0:00:00
Collecting fasttext (from nemo-toolkit==1.20.0)
  Downloading fasttext-0.9.2.tar.gz (68 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 68.8/68.8 kB 248.9 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting flask-restful (from nemo-toolkit==1.20.0)
  Obtaining dependency information for flask-restful from https://files.pythonhosted.org/packages/d7/7b/f0b45f0df7d2978e5ae51804bb5939b7897b2ace24306009da0cc34d8d1f/Flask_RESTful-0.3.10-py2.py3-none-any.whl.metadata
  Downloading Flask_RESTful-0.3.10-py2.py3-none-any.whl.metadata (1.0 kB)
Collecting ftfy (from nemo-toolkit==1.20.0)
  Obtaining dependency information for ftfy from https://files.pythonhosted.org/packages/91/f8/dfa32d06cfcbdb76bc46e0f5d69c537de33f4cedb1a15cd4746ab45a6a26/ftfy-6.1.3-py3-none-any.whl.metadata
  Downloading ftfy-6.1.3-py3-none-any.whl.metadata (6.2 kB)
Collecting gdown (from nemo-toolkit==1.20.0)
  Downloading gdown-4.7.1-py3-none-any.whl (15 kB)
Collecting h5py (from nemo-toolkit==1.20.0)
  Obtaining dependency information for h5py from https://files.pythonhosted.org/packages/3b/d3/ecb4b3d2ec2c84132987e5f12ab1408f455bec1d90cd5bc408ebf37800f5/h5py-3.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading h5py-3.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)
Collecting ijson (from nemo-toolkit==1.20.0)
  Obtaining dependency information for ijson from https://files.pythonhosted.org/packages/6b/78/2cbeb7020a7a319d148c92331951cfc710864990e32ff6c7f4859729fb48/ijson-3.2.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading ijson-3.2.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)
Collecting jieba (from nemo-toolkit==1.20.0)
  Downloading jieba-0.42.1.tar.gz (19.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.2/19.2 MB 117.6 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting markdown2 (from nemo-toolkit==1.20.0)
  Obtaining dependency information for markdown2 from https://files.pythonhosted.org/packages/c9/38/183d3c253b9dc884c9d698dd5d68f258d0f853726260362f0fa49400279d/markdown2-2.4.11-py2.py3-none-any.whl.metadata
  Downloading markdown2-2.4.11-py2.py3-none-any.whl.metadata (2.0 kB)
Requirement already satisfied: matplotlib>=3.3.2 in /usr/local/lib/python3.10/dist-packages (from nemo-toolkit==1.20.0) (3.7.3)
Collecting megatron-core==0.2.0 (from nemo-toolkit==1.20.0)
  Obtaining dependency information for megatron-core==0.2.0 from https://files.pythonhosted.org/packages/33/f1/d94f2282b91950e31223efc39138748d71907dbf857e5523d1e73619fd62/megatron_core-0.2.0-py3-none-any.whl.metadata
  Downloading megatron_core-0.2.0-py3-none-any.whl.metadata (1.6 kB)
Collecting nltk>=3.6.5 (from nemo-toolkit==1.20.0)
  Downloading nltk-3.8.1-py3-none-any.whl (1.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 137.2 MB/s eta 0:00:00
Collecting opencc (from nemo-toolkit==1.20.0)
  Obtaining dependency information for opencc from https://files.pythonhosted.org/packages/df/27/3d4652dcf73d1ddde83348ab167dc33372822f96eac76fd6235d5144868a/OpenCC-1.1.7-cp310-cp310-manylinux1_x86_64.whl.metadata
  Downloading OpenCC-1.1.7-cp310-cp310-manylinux1_x86_64.whl.metadata (12 kB)
Collecting pangu (from nemo-toolkit==1.20.0)
  Downloading pangu-4.0.6.1-py3-none-any.whl (6.4 kB)
Collecting rapidfuzz (from nemo-toolkit==1.20.0)
  Obtaining dependency information for rapidfuzz from https://files.pythonhosted.org/packages/58/69/55f7a10f5760d8539081718f1a87172cd5b0ea21f6c1232c40a6b7cf9470/rapidfuzz-3.5.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading rapidfuzz-3.5.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
Collecting rouge-score (from nemo-toolkit==1.20.0)
  Downloading rouge_score-0.1.2.tar.gz (17 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting sacrebleu[ja] (from nemo-toolkit==1.20.0)
  Obtaining dependency information for sacrebleu[ja] from https://files.pythonhosted.org/packages/de/ea/025db0a39337b63d4728a900d262c39c3029b0fe76a9876ce6297b1aa6a0/sacrebleu-2.4.0-py3-none-any.whl.metadata
  Downloading sacrebleu-2.4.0-py3-none-any.whl.metadata (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.4/57.4 kB 118.5 MB/s eta 0:00:00
Collecting sentence-transformers (from nemo-toolkit==1.20.0)
  Downloading sentence-transformers-2.2.2.tar.gz (85 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.0/86.0 kB 129.7 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting hydra-core<1.3,>=1.2.0 (from nemo-toolkit==1.20.0)
  Downloading hydra_core-1.2.0-py3-none-any.whl (151 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.1/151.1 kB 206.4 MB/s eta 0:00:00
Collecting omegaconf<2.3,>=2.2 (from nemo-toolkit==1.20.0)
  Downloading omegaconf-2.2.3-py3-none-any.whl (79 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.3/79.3 kB 251.8 MB/s eta 0:00:00
Collecting pytorch-lightning<=1.9.4,>=1.9.0 (from nemo-toolkit==1.20.0)
  Downloading pytorch_lightning-1.9.4-py3-none-any.whl (827 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 827.8/827.8 kB 150.9 MB/s eta 0:00:00
Collecting torchmetrics>=0.11.0 (from nemo-toolkit==1.20.0)
  Obtaining dependency information for torchmetrics>=0.11.0 from https://files.pythonhosted.org/packages/62/17/eedb48177a4679b75b82185492f8ad2b4d010e032fd38160e157b0e22028/torchmetrics-1.2.1-py3-none-any.whl.metadata
  Downloading torchmetrics-1.2.1-py3-none-any.whl.metadata (20 kB)
Collecting transformers>=4.0.1 (from nemo-toolkit==1.20.0)
  Obtaining dependency information for transformers>=4.0.1 from https://files.pythonhosted.org/packages/fc/04/0aad491cd98b09236c54ab849863ee85421eeda5138bbf9d33ecc594652b/transformers-4.36.1-py3-none-any.whl.metadata
  Downloading transformers-4.36.1-py3-none-any.whl.metadata (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.8/126.8 kB 266.5 MB/s eta 0:00:00
Collecting wandb (from nemo-toolkit==1.20.0)
  Obtaining dependency information for wandb from https://files.pythonhosted.org/packages/35/d3/6bfe29e4ba1eb2400d478caf8e3af9a1c366390390069cda59a7c6bf6063/wandb-0.16.1-py3-none-any.whl.metadata
  Downloading wandb-0.16.1-py3-none-any.whl.metadata (9.8 kB)
Collecting webdataset<=0.1.62,>=0.1.48 (from nemo-toolkit==1.20.0)
  Downloading webdataset-0.1.62-py3-none-any.whl (32 kB)
Collecting inflect (from nemo-toolkit==1.20.0)
  Obtaining dependency information for inflect from https://files.pythonhosted.org/packages/fb/c6/d9feb758be584f729424390af24687d3a4363d968164f94079f83cd536b4/inflect-7.0.0-py3-none-any.whl.metadata
  Downloading inflect-7.0.0-py3-none-any.whl.metadata (21 kB)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from nemo-toolkit==1.20.0) (1.5.3)
Collecting pydantic<2 (from nemo-toolkit==1.20.0)
  Obtaining dependency information for pydantic<2 from https://files.pythonhosted.org/packages/e0/2f/d6f17f8385d718233bcae893d27525443d41201c938b68a4af3d591a33e4/pydantic-1.10.13-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading pydantic-1.10.13-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (149 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.6/149.6 kB 254.7 MB/s eta 0:00:00
Collecting sacremoses>=0.0.43 (from nemo-toolkit==1.20.0)
  Obtaining dependency information for sacremoses>=0.0.43 from https://files.pythonhosted.org/packages/0b/f0/89ee2bc9da434bd78464f288fdb346bc2932f2ee80a90b2a4bbbac262c74/sacremoses-0.1.1-py3-none-any.whl.metadata
  Downloading sacremoses-0.1.1-py3-none-any.whl.metadata (8.3 kB)
Collecting sentencepiece<1.0.0 (from nemo-toolkit==1.20.0)
  Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 138.6 MB/s eta 0:00:00
Collecting youtokentome>=1.0.5 (from nemo-toolkit==1.20.0)
  Downloading youtokentome-1.0.6.tar.gz (86 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.7/86.7 kB 268.2 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting antlr4-python3-runtime==4.9.* (from hydra-core<1.3,>=1.2.0->nemo-toolkit==1.20.0)
  Downloading antlr4-python3-runtime-4.9.3.tar.gz (117 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.0/117.0 kB 246.9 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from hydra-core<1.3,>=1.2.0->nemo-toolkit==1.20.0) (23.1)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.3.2->nemo-toolkit==1.20.0) (1.1.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.3.2->nemo-toolkit==1.20.0) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.3.2->nemo-toolkit==1.20.0) (4.42.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.3.2->nemo-toolkit==1.20.0) (1.4.5)
Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.3.2->nemo-toolkit==1.20.0) (9.2.0)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.3.2->nemo-toolkit==1.20.0) (3.1.1)
Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk>=3.6.5->nemo-toolkit==1.20.0) (8.1.6)
Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk>=3.6.5->nemo-toolkit==1.20.0) (1.3.2)
Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk>=3.6.5->nemo-toolkit==1.20.0) (2023.8.8)
Requirement already satisfied: PyYAML>=5.1.0 in /usr/local/lib/python3.10/dist-packages (from omegaconf<2.3,>=2.2->nemo-toolkit==1.20.0) (6.0.1)
Requirement already satisfied: protobuf>=3.20.2 in /usr/local/lib/python3.10/dist-packages (from onnx>=1.7.0->nemo-toolkit==1.20.0) (4.24.3)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /usr/local/lib/python3.10/dist-packages (from onnx>=1.7.0->nemo-toolkit==1.20.0) (4.7.1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil->nemo-toolkit==1.20.0) (1.16.0)
Requirement already satisfied: fsspec[http]>2021.06.0 in /usr/local/lib/python3.10/dist-packages (from pytorch-lightning<=1.9.4,>=1.9.0->nemo-toolkit==1.20.0) (2023.6.0)
Collecting lightning-utilities>=0.6.0.post0 (from pytorch-lightning<=1.9.4,>=1.9.0->nemo-toolkit==1.20.0)
  Obtaining dependency information for lightning-utilities>=0.6.0.post0 from https://files.pythonhosted.org/packages/5e/f4/07b748cb9834848de16aaeb1ae38bc9cfcfe3adc22ee2c8ebbe11db82795/lightning_utilities-0.10.0-py3-none-any.whl.metadata
  Downloading lightning_utilities-0.10.0-py3-none-any.whl.metadata (4.8 kB)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch->nemo-toolkit==1.20.0) (3.12.4)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch->nemo-toolkit==1.20.0) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch->nemo-toolkit==1.20.0) (2.6.3)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch->nemo-toolkit==1.20.0) (3.1.2)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers>=4.0.1->nemo-toolkit==1.20.0) (2.31.0)
Collecting tokenizers<0.19,>=0.14 (from transformers>=4.0.1->nemo-toolkit==1.20.0)
  Obtaining dependency information for tokenizers<0.19,>=0.14 from https://files.pythonhosted.org/packages/eb/3d/eee5f3c572a3f4db2ebabf5bd4f284f356078a5b5d27e6229b4450d5c5e4/tokenizers-0.15.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading tokenizers-0.15.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)
Collecting safetensors>=0.3.1 (from transformers>=4.0.1->nemo-toolkit==1.20.0)
  Obtaining dependency information for safetensors>=0.3.1 from https://files.pythonhosted.org/packages/35/8f/892d2e1bcfceb7ee3f9b055ac4bb111e31d25f0a38c7f44d1d59bf7a501a/safetensors-0.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading safetensors-0.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting braceexpand (from webdataset<=0.1.62,>=0.1.48->nemo-toolkit==1.20.0)
  Downloading braceexpand-0.1.7-py2.py3-none-any.whl (5.9 kB)
Collecting botocore<1.35.0,>=1.34.2 (from boto3->nemo-toolkit==1.20.0)
  Obtaining dependency information for botocore<1.35.0,>=1.34.2 from https://files.pythonhosted.org/packages/b7/52/541fbf07388a44693941c126d9a356687da749ceb2a03ad3f2d53735ff2d/botocore-1.34.2-py3-none-any.whl.metadata
  Downloading botocore-1.34.2-py3-none-any.whl.metadata (5.6 kB)
Collecting jmespath<2.0.0,>=0.7.1 (from boto3->nemo-toolkit==1.20.0)
  Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting s3transfer<0.10.0,>=0.9.0 (from boto3->nemo-toolkit==1.20.0)
  Obtaining dependency information for s3transfer<0.10.0,>=0.9.0 from https://files.pythonhosted.org/packages/fd/fb/46eda754e80fa2efd82981e37cd75cabbecef71df63843e6e94e12fae9db/s3transfer-0.9.0-py3-none-any.whl.metadata
  Downloading s3transfer-0.9.0-py3-none-any.whl.metadata (1.7 kB)
Requirement already satisfied: pybind11>=2.2 in /usr/local/lib/python3.10/dist-packages (from fasttext->nemo-toolkit==1.20.0) (2.11.1)
Collecting aniso8601>=0.82 (from flask-restful->nemo-toolkit==1.20.0)
  Downloading aniso8601-9.0.1-py2.py3-none-any.whl (52 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 52.8/52.8 kB 219.7 MB/s eta 0:00:00
Collecting Flask>=0.8 (from flask-restful->nemo-toolkit==1.20.0)
  Obtaining dependency information for Flask>=0.8 from https://files.pythonhosted.org/packages/36/42/015c23096649b908c809c69388a805a571a3bea44362fe87e33fc3afa01f/flask-3.0.0-py3-none-any.whl.metadata
  Downloading flask-3.0.0-py3-none-any.whl.metadata (3.6 kB)
Requirement already satisfied: pytz in /usr/local/lib/python3.10/dist-packages (from flask-restful->nemo-toolkit==1.20.0) (2023.3)
Collecting wcwidth<0.3.0,>=0.2.12 (from ftfy->nemo-toolkit==1.20.0)
  Obtaining dependency information for wcwidth<0.3.0,>=0.2.12 from https://files.pythonhosted.org/packages/31/b1/a59de0ad3aabb17523a39804f4c6df3ae87ead053a4e25362ae03d73d03a/wcwidth-0.2.12-py2.py3-none-any.whl.metadata
  Downloading wcwidth-0.2.12-py2.py3-none-any.whl.metadata (14 kB)
Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.10/dist-packages (from gdown->nemo-toolkit==1.20.0) (4.12.2)
Requirement already satisfied: llvmlite<0.41,>=0.40.0dev0 in /usr/local/lib/python3.10/dist-packages (from numba->nemo-toolkit==1.20.0) (0.40.1)
Requirement already satisfied: absl-py in /usr/local/lib/python3.10/dist-packages (from rouge-score->nemo-toolkit==1.20.0) (1.4.0)
Collecting ruamel.yaml.clib>=0.2.7 (from ruamel.yaml->nemo-toolkit==1.20.0)
  Obtaining dependency information for ruamel.yaml.clib>=0.2.7 from https://files.pythonhosted.org/packages/d3/62/c60b034d9a008bbd566eeecf53a5a4c73d191c8de261290db6761802b72d/ruamel.yaml.clib-0.2.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata
  Downloading ruamel.yaml.clib-0.2.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (2.2 kB)
Collecting portalocker (from sacrebleu[ja]->nemo-toolkit==1.20.0)
  Obtaining dependency information for portalocker from https://files.pythonhosted.org/packages/17/9e/87671efcca80ba6203811540ed1f9c0462c1609d2281d7b7f53cef05da3d/portalocker-2.8.2-py3-none-any.whl.metadata
  Downloading portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Requirement already satisfied: tabulate>=0.8.9 in /usr/local/lib/python3.10/dist-packages (from sacrebleu[ja]->nemo-toolkit==1.20.0) (0.9.0)
Collecting colorama (from sacrebleu[ja]->nemo-toolkit==1.20.0)
  Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting lxml (from sacrebleu[ja]->nemo-toolkit==1.20.0)
  Obtaining dependency information for lxml from https://files.pythonhosted.org/packages/3c/d2/11533f0bc47ff4d828a20cfb702f3453fe714bd5b475fcdc8cec6e6b7dcf/lxml-4.9.3-cp310-cp310-manylinux_2_28_x86_64.whl.metadata
  Downloading lxml-4.9.3-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting mecab-python3<=1.0.6,>=1.0.5 (from sacrebleu[ja]->nemo-toolkit==1.20.0)
  Downloading mecab_python3-1.0.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (581 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 581.6/581.6 kB 170.1 MB/s eta 0:00:00
Collecting ipadic<2.0,>=1.0 (from sacrebleu[ja]->nemo-toolkit==1.20.0)
  Downloading ipadic-1.0.0.tar.gz (13.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.4/13.4 MB 117.3 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: scipy>=1.3.2 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->nemo-toolkit==1.20.0) (1.11.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->nemo-toolkit==1.20.0) (3.2.0)
Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (from sentence-transformers->nemo-toolkit==1.20.0) (0.16.0a0)
Requirement already satisfied: grpcio>=1.24.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard->nemo-toolkit==1.20.0) (1.58.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard->nemo-toolkit==1.20.0) (2.23.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard->nemo-toolkit==1.20.0) (0.4.6)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard->nemo-toolkit==1.20.0) (3.4.4)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->nemo-toolkit==1.20.0) (0.6.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->nemo-toolkit==1.20.0) (1.8.1)
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard->nemo-toolkit==1.20.0) (2.3.7)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.10/dist-packages (from tensorboard->nemo-toolkit==1.20.0) (0.41.2)
Collecting GitPython!=3.1.29,>=1.0.0 (from wandb->nemo-toolkit==1.20.0)
  Obtaining dependency information for GitPython!=3.1.29,>=1.0.0 from https://files.pythonhosted.org/packages/8d/c4/82b858fb6483dfb5e338123c154d19c043305b01726a67d89532b8f8f01b/GitPython-3.1.40-py3-none-any.whl.metadata
  Downloading GitPython-3.1.40-py3-none-any.whl.metadata (12 kB)
Requirement already satisfied: psutil>=5.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb->nemo-toolkit==1.20.0) (5.9.4)
Collecting sentry-sdk>=1.0.0 (from wandb->nemo-toolkit==1.20.0)
  Obtaining dependency information for sentry-sdk>=1.0.0 from https://files.pythonhosted.org/packages/aa/39/a40c841782b775eec1602a82387b4e91322ccafd842fd60fc4deb9f13f7d/sentry_sdk-1.39.1-py2.py3-none-any.whl.metadata
  Downloading sentry_sdk-1.39.1-py2.py3-none-any.whl.metadata (9.7 kB)
Collecting docker-pycreds>=0.4.0 (from wandb->nemo-toolkit==1.20.0)
  Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting setproctitle (from wandb->nemo-toolkit==1.20.0)
  Obtaining dependency information for setproctitle from https://files.pythonhosted.org/packages/79/e7/54b36be02aee8ad573be68f6f46fd62838735c2f007b22df50eb5e13a20d/setproctitle-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading setproctitle-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.9 kB)
Collecting appdirs>=1.4.3 (from wandb->nemo-toolkit==1.20.0)
  Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Requirement already satisfied: urllib3<2.1,>=1.25.4 in /usr/local/lib/python3.10/dist-packages (from botocore<1.35.0,>=1.34.2->boto3->nemo-toolkit==1.20.0) (1.26.16)
Collecting werkzeug>=1.0.1 (from tensorboard->nemo-toolkit==1.20.0)
  Obtaining dependency information for werkzeug>=1.0.1 from https://files.pythonhosted.org/packages/c3/fc/254c3e9b5feb89ff5b9076a23218dafbc99c96ac5941e900b71206e6313b/werkzeug-3.0.1-py3-none-any.whl.metadata
  Downloading werkzeug-3.0.1-py3-none-any.whl.metadata (4.1 kB)
Collecting itsdangerous>=2.1.2 (from Flask>=0.8->flask-restful->nemo-toolkit==1.20.0)
  Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting blinker>=1.6.2 (from Flask>=0.8->flask-restful->nemo-toolkit==1.20.0)
  Obtaining dependency information for blinker>=1.6.2 from https://files.pythonhosted.org/packages/fa/2a/7f3714cbc6356a0efec525ce7a0613d581072ed6eb53eb7b9754f33db807/blinker-1.7.0-py3-none-any.whl.metadata
  Downloading blinker-1.7.0-py3-none-any.whl.metadata (1.9 kB)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.10/dist-packages (from fsspec[http]>2021.06.0->pytorch-lightning<=1.9.4,>=1.9.0->nemo-toolkit==1.20.0) (3.8.5)
Collecting gitdb<5,>=4.0.1 (from GitPython!=3.1.29,>=1.0.0->wandb->nemo-toolkit==1.20.0)
  Obtaining dependency information for gitdb<5,>=4.0.1 from https://files.pythonhosted.org/packages/fd/5b/8f0c4a5bb9fd491c277c21eff7ccae71b47d43c4446c9d0c6cff2fe8c2c4/gitdb-4.0.11-py3-none-any.whl.metadata
  Downloading gitdb-4.0.11-py3-none-any.whl.metadata (1.2 kB)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->nemo-toolkit==1.20.0) (5.3.1)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->nemo-toolkit==1.20.0) (0.3.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->nemo-toolkit==1.20.0) (4.9)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->nemo-toolkit==1.20.0) (1.3.1)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch->nemo-toolkit==1.20.0) (2.1.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers>=4.0.1->nemo-toolkit==1.20.0) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers>=4.0.1->nemo-toolkit==1.20.0) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers>=4.0.1->nemo-toolkit==1.20.0) (2023.7.22)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4->gdown->nemo-toolkit==1.20.0) (2.5)
Collecting PySocks!=1.5.7,>=1.5.6 (from requests->transformers>=4.0.1->nemo-toolkit==1.20.0)
  Downloading PySocks-1.7.1-py3-none-any.whl (16 kB)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->nemo-toolkit==1.20.0) (1.3.0)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning<=1.9.4,>=1.9.0->nemo-toolkit==1.20.0) (23.1.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning<=1.9.4,>=1.9.0->nemo-toolkit==1.20.0) (6.0.4)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning<=1.9.4,>=1.9.0->nemo-toolkit==1.20.0) (4.0.3)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning<=1.9.4,>=1.9.0->nemo-toolkit==1.20.0) (1.9.2)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning<=1.9.4,>=1.9.0->nemo-toolkit==1.20.0) (1.4.0)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning<=1.9.4,>=1.9.0->nemo-toolkit==1.20.0) (1.3.1)
Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->nemo-toolkit==1.20.0)
  Obtaining dependency information for smmap<6,>=3.0.1 from https://files.pythonhosted.org/packages/a7/a5/10f97f73544edcdef54409f1d839f6049a0d79df68adbc1ceb24d1aaca42/smmap-5.0.1-py3-none-any.whl.metadata
  Downloading smmap-5.0.1-py3-none-any.whl.metadata (4.3 kB)
Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->nemo-toolkit==1.20.0) (0.5.0)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->nemo-toolkit==1.20.0) (3.2.2)
Downloading megatron_core-0.2.0-py3-none-any.whl (46 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.0/46.0 kB 210.2 MB/s eta 0:00:00
Downloading pydantic-1.10.13-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 123.2 MB/s eta 0:00:00
Downloading sacremoses-0.1.1-py3-none-any.whl (897 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 897.5/897.5 kB 144.0 MB/s eta 0:00:00
Downloading torchmetrics-1.2.1-py3-none-any.whl (806 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 806.1/806.1 kB 139.9 MB/s eta 0:00:00
Downloading transformers-4.36.1-py3-none-any.whl (8.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.3/8.3 MB 120.0 MB/s eta 0:00:00
Downloading huggingface_hub-0.19.4-py3-none-any.whl (311 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 311.7/311.7 kB 209.8 MB/s eta 0:00:00
Downloading boto3-1.34.2-py3-none-any.whl (139 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.3/139.3 kB 269.4 MB/s eta 0:00:00
Downloading Flask_RESTful-0.3.10-py2.py3-none-any.whl (26 kB)
Downloading ftfy-6.1.3-py3-none-any.whl (53 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.4/53.4 kB 230.8 MB/s eta 0:00:00
Downloading h5py-3.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.8/4.8 MB 122.2 MB/s eta 0:00:00
Downloading ijson-3.2.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (111 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 111.8/111.8 kB 278.1 MB/s eta 0:00:00
Downloading inflect-7.0.0-py3-none-any.whl (34 kB)
Downloading markdown2-2.4.11-py2.py3-none-any.whl (41 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.1/41.1 kB 193.1 MB/s eta 0:00:00
Downloading OpenCC-1.1.7-cp310-cp310-manylinux1_x86_64.whl (779 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 779.8/779.8 kB 152.9 MB/s eta 0:00:00
Downloading rapidfuzz-3.5.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 123.7 MB/s eta 0:00:00
Downloading ruamel.yaml-0.18.5-py3-none-any.whl (116 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.4/116.4 kB 267.3 MB/s eta 0:00:00
Downloading wandb-0.16.1-py3-none-any.whl (2.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 130.5 MB/s eta 0:00:00
Downloading wrapt-1.16.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (80 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 80.3/80.3 kB 261.0 MB/s eta 0:00:00
Downloading botocore-1.34.2-py3-none-any.whl (11.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.8/11.8 MB 117.5 MB/s eta 0:00:00
Downloading flask-3.0.0-py3-none-any.whl (99 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.7/99.7 kB 272.5 MB/s eta 0:00:00
Downloading GitPython-3.1.40-py3-none-any.whl (190 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 190.6/190.6 kB 288.5 MB/s eta 0:00:00
Downloading lightning_utilities-0.10.0-py3-none-any.whl (24 kB)
Downloading ruamel.yaml.clib-0.2.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (526 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 526.7/526.7 kB 174.9 MB/s eta 0:00:00
Downloading s3transfer-0.9.0-py3-none-any.whl (82 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 82.0/82.0 kB 241.9 MB/s eta 0:00:00
Downloading safetensors-0.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 138.1 MB/s eta 0:00:00
Downloading sentry_sdk-1.39.1-py2.py3-none-any.whl (254 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 254.1/254.1 kB 276.4 MB/s eta 0:00:00
Downloading tokenizers-0.15.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.8/3.8 MB 123.9 MB/s eta 0:00:00
Downloading wcwidth-0.2.12-py2.py3-none-any.whl (34 kB)
Downloading werkzeug-3.0.1-py3-none-any.whl (226 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 226.7/226.7 kB 272.6 MB/s eta 0:00:00
Downloading lxml-4.9.3-cp310-cp310-manylinux_2_28_x86_64.whl (7.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.9/7.9 MB 120.3 MB/s eta 0:00:00
Downloading portalocker-2.8.2-py3-none-any.whl (17 kB)
Downloading sacrebleu-2.4.0-py3-none-any.whl (106 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.3/106.3 kB 280.8 MB/s eta 0:00:00
Downloading setproctitle-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30 kB)
Downloading blinker-1.7.0-py3-none-any.whl (13 kB)
Downloading gitdb-4.0.11-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.7/62.7 kB 245.8 MB/s eta 0:00:00
Downloading smmap-5.0.1-py3-none-any.whl (24 kB)
Building wheels for collected packages: antlr4-python3-runtime, youtokentome, fasttext, jieba, nemo-toolkit, rouge-score, sentence-transformers, wget, ipadic
  Building wheel for antlr4-python3-runtime (setup.py): started
  Building wheel for antlr4-python3-runtime (setup.py): finished with status 'done'
  Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.9.3-py3-none-any.whl size=144573 sha256=d8919eb0f5eda65508fb1233ee44ebc616076c9eac2049b1e64423e5468f82f6
  Stored in directory: /tmp/pip-ephem-wheel-cache-bipegysw/wheels/12/93/dd/1f6a127edc45659556564c5730f6d4e300888f4bca2d4c5a88
  Building wheel for youtokentome (setup.py): started
  Building wheel for youtokentome (setup.py): finished with status 'done'
  Created wheel for youtokentome: filename=youtokentome-1.0.6-cp310-cp310-linux_x86_64.whl size=1883754 sha256=7df41b193c0611a0f13b9fe86d6e9aac43bfaebb3de35b80a73f8a4be7bf60b6
  Stored in directory: /tmp/pip-ephem-wheel-cache-bipegysw/wheels/df/85/f8/301d2ba45f43f30bed2fe413efa760bc726b8b660ed9c2900c
  Building wheel for fasttext (setup.py): started
  Building wheel for fasttext (setup.py): finished with status 'done'
  Created wheel for fasttext: filename=fasttext-0.9.2-cp310-cp310-linux_x86_64.whl size=4199778 sha256=2d773d3c0ec2d4ad94718a98b069144ebf16fac7a6e255b77eb68a5446fbd424
  Stored in directory: /tmp/pip-ephem-wheel-cache-bipegysw/wheels/a5/13/75/f811c84a8ab36eedbaef977a6a58a98990e8e0f1967f98f394
  Building wheel for jieba (setup.py): started
  Building wheel for jieba (setup.py): finished with status 'done'
  Created wheel for jieba: filename=jieba-0.42.1-py3-none-any.whl size=19314476 sha256=672c64e7ff92b7a403a770f7f5d9c989cd22256c0d666ba8869f9622393a8ae5
  Stored in directory: /tmp/pip-ephem-wheel-cache-bipegysw/wheels/c9/69/31/d56d90b22a1777b0b231e234b00302a55be255930f8bd92dcd
  Building editable for nemo-toolkit (pyproject.toml): started
  Building editable for nemo-toolkit (pyproject.toml): finished with status 'done'
  Created wheel for nemo-toolkit: filename=nemo_toolkit-1.20.0-0.editable-py3-none-any.whl size=9116 sha256=566d3dff9242f027f6e3ce6b59b72b23f0c648bf74781861b007070181947654
  Stored in directory: /tmp/pip-ephem-wheel-cache-bipegysw/wheels/c1/36/f0/f03c1cea654945ca36db660623bda98e17cd0c10dbd4c715e0
  Building wheel for rouge-score (setup.py): started
  Building wheel for rouge-score (setup.py): finished with status 'done'
  Created wheel for rouge-score: filename=rouge_score-0.1.2-py3-none-any.whl size=24952 sha256=72734913790285dc600f518db8e6bccd38dc61f1a5f4692c58bade6a759334df
  Stored in directory: /tmp/pip-ephem-wheel-cache-bipegysw/wheels/5f/dd/89/461065a73be61a532ff8599a28e9beef17985c9e9c31e541b4
  Building wheel for sentence-transformers (setup.py): started
  Building wheel for sentence-transformers (setup.py): finished with status 'done'
  Created wheel for sentence-transformers: filename=sentence_transformers-2.2.2-py3-none-any.whl size=125940 sha256=25c2e174e0dd4c1dea3538afb5602f53dab30ef9083d14d00c3fbd3636a62832
  Stored in directory: /tmp/pip-ephem-wheel-cache-bipegysw/wheels/62/f2/10/1e606fd5f02395388f74e7462910fe851042f97238cbbd902f
  Building wheel for wget (setup.py): started
  Building wheel for wget (setup.py): finished with status 'done'
  Created wheel for wget: filename=wget-3.2-py3-none-any.whl size=9673 sha256=ca36b12fc827b9e45bf2bff94e376928bfbdda8c7b8f151fe1d7636f28b97341
  Stored in directory: /tmp/pip-ephem-wheel-cache-bipegysw/wheels/8b/f1/7f/5c94f0a7a505ca1c81cd1d9208ae2064675d97582078e6c769
  Building wheel for ipadic (setup.py): started
  Building wheel for ipadic (setup.py): finished with status 'done'
  Created wheel for ipadic: filename=ipadic-1.0.0-py3-none-any.whl size=13556721 sha256=ae725295f8fd4f4cc9f3419f68b767586f5033a2081fbf3b2861fd0bb5310e78
  Stored in directory: /tmp/pip-ephem-wheel-cache-bipegysw/wheels/5b/ea/e3/2f6e0860a327daba3b030853fce4483ed37468bbf1101c59c3
Successfully built antlr4-python3-runtime youtokentome fasttext jieba nemo-toolkit rouge-score sentence-transformers wget ipadic
Installing collected packages: wget, wcwidth, text-unidecode, sentencepiece, pangu, opencc, mecab-python3, jieba, ipadic, ijson, faiss-cpu, braceexpand, appdirs, antlr4-python3-runtime, aniso8601, youtokentome, wrapt, werkzeug, webdataset, smmap, setuptools, setproctitle, sentry-sdk, safetensors, sacremoses, ruamel.yaml.clib, rapidfuzz, PySocks, pydantic, portalocker, omegaconf, nltk, markdown2, lxml, jmespath, itsdangerous, h5py, ftfy, docker-pycreds, colorama, blinker, sacrebleu, ruamel.yaml, rouge-score, lightning-utilities, inflect, hydra-core, huggingface-hub, gitdb, Flask, fasttext, botocore, torchmetrics, tokenizers, s3transfer, megatron-core, GitPython, gdown, flask-restful, wandb, transformers, pytorch-lightning, boto3, sentence-transformers, nemo-toolkit
  Attempting uninstall: wcwidth
    Found existing installation: wcwidth 0.2.6
    Uninstalling wcwidth-0.2.6:
      Successfully uninstalled wcwidth-0.2.6
  Attempting uninstall: werkzeug
    Found existing installation: Werkzeug 2.3.7
    Uninstalling Werkzeug-2.3.7:
      Successfully uninstalled Werkzeug-2.3.7
  Attempting uninstall: setuptools
    Found existing installation: setuptools 68.2.2
    Uninstalling setuptools-68.2.2:
      Successfully uninstalled setuptools-68.2.2
  Attempting uninstall: pydantic
    Found existing installation: pydantic 2.3.0
    Uninstalling pydantic-2.3.0:
      Successfully uninstalled pydantic-2.3.0
Successfully installed Flask-3.0.0 GitPython-3.1.40 PySocks-1.7.1 aniso8601-9.0.1 antlr4-python3-runtime-4.9.3 appdirs-1.4.4 blinker-1.7.0 boto3-1.34.2 botocore-1.34.2 braceexpand-0.1.7 colorama-0.4.6 docker-pycreds-0.4.0 faiss-cpu-1.7.4 fasttext-0.9.2 flask-restful-0.3.10 ftfy-6.1.3 gdown-4.7.1 gitdb-4.0.11 h5py-3.10.0 huggingface-hub-0.19.4 hydra-core-1.2.0 ijson-3.2.3 inflect-7.0.0 ipadic-1.0.0 itsdangerous-2.1.2 jieba-0.42.1 jmespath-1.0.1 lightning-utilities-0.10.0 lxml-4.9.3 markdown2-2.4.11 mecab-python3-1.0.6 megatron-core-0.2.0 nemo-toolkit-1.20.0 nltk-3.8.1 omegaconf-2.2.3 opencc-1.1.7 pangu-4.0.6.1 portalocker-2.8.2 pydantic-1.10.13 pytorch-lightning-1.9.4 rapidfuzz-3.5.2 rouge-score-0.1.2 ruamel.yaml-0.18.5 ruamel.yaml.clib-0.2.8 s3transfer-0.9.0 sacrebleu-2.4.0 sacremoses-0.1.1 safetensors-0.4.1 sentence-transformers-2.2.2 sentencepiece-0.1.99 sentry-sdk-1.39.1 setproctitle-1.3.3 setuptools-65.5.1 smmap-5.0.1 text-unidecode-1.3 tokenizers-0.15.0 torchmetrics-1.2.1 transformers-4.36.1 wandb-0.16.1 wcwidth-0.2.12 webdataset-0.1.62 werkzeug-3.0.1 wget-3.2 wrapt-1.16.0 youtokentome-1.0.6
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: python -m pip install --upgrade pip
Removing intermediate container cf031a6a6c08
 ---> 323c33f40149
Step 23/33 : RUN MAX_JOBS=4 pip install --upgrade --no-deps flash-attn==2.0.8
 ---> Running in 324e4db29654
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting flash-attn==2.0.8
  Downloading flash_attn-2.0.8.tar.gz (2.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 2.1 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: flash-attn
  Building wheel for flash-attn (setup.py): started
  Building wheel for flash-attn (setup.py): still running...
  Building wheel for flash-attn (setup.py): still running...
  Building wheel for flash-attn (setup.py): still running...
  Building wheel for flash-attn (setup.py): still running...
  Building wheel for flash-attn (setup.py): still running...
  Building wheel for flash-attn (setup.py): still running...
  Building wheel for flash-attn (setup.py): still running...
  Building wheel for flash-attn (setup.py): still running...
  Building wheel for flash-attn (setup.py): still running...
  Building wheel for flash-attn (setup.py): finished with status 'done'
  Created wheel for flash-attn: filename=flash_attn-2.0.8-cp310-cp310-linux_x86_64.whl size=120873622 sha256=de6730fc453bcc9c7e2077e7035bb0b4c21679a1b9fb9b9161fba5c84b865551
  Stored in directory: /tmp/pip-ephem-wheel-cache-3_emqvxe/wheels/83/5a/6f/eb5a647d13ce1fcc1bca07ba898f04e83c7ea743333985cbe4
Successfully built flash-attn
Installing collected packages: flash-attn
  Attempting uninstall: flash-attn
    Found existing installation: flash-attn 2.0.4
    Uninstalling flash-attn-2.0.4:
      Successfully uninstalled flash-attn-2.0.4
Successfully installed flash-attn-2.0.8
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: python -m pip install --upgrade pip
Removing intermediate container 324e4db29654
 ---> 972bfa9b3139
Step 24/33 : RUN cd NeMo &&       cd nemo/collections/nlp/data/language_modeling/megatron &&       make
 ---> Running in 68c49c51c602
g++ -O3 -Wall -shared -std=c++11 -fPIC -fdiagnostics-color -I/usr/include/python3.10 -I/usr/local/lib/python3.10/dist-packages/pybind11/include helpers.cpp -o helpers.cpython-310-x86_64-linux-gnu.so
Removing intermediate container 68c49c51c602
 ---> 4c0e6e8070e6
Step 25/33 : ARG MEGATRON_REVISION=df7271285d3e29c13865515a51dbbd6e25f68f5f
 ---> Running in 1ef73d806de1
Removing intermediate container 1ef73d806de1
 ---> e044cb7632a5
Step 26/33 : ENV CUSTOM_MEGATRON_REVISION ${MEGATRON_REVISION}
 ---> Running in 8303941961f3
Removing intermediate container 8303941961f3
 ---> 108d144ebeb0
Step 27/33 : RUN if [ "${MEGATRON_REVISION}" != SKIP ]; then       pip uninstall -y megatron-core &&       pip install git+https://github.com/nvidia/Megatron-LM.git@${MEGATRON_REVISION}     ; fi
 ---> Running in e0915fa63bb6
Found existing installation: megatron-core 0.2.0
Uninstalling megatron-core-0.2.0:
  Successfully uninstalled megatron-core-0.2.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/nvidia/Megatron-LM.git@df7271285d3e29c13865515a51dbbd6e25f68f5f
  Cloning https://github.com/nvidia/Megatron-LM.git (to revision df7271285d3e29c13865515a51dbbd6e25f68f5f) to /tmp/pip-req-build-uzzezvpb
  Running command git clone --filter=blob:none --quiet https://github.com/nvidia/Megatron-LM.git /tmp/pip-req-build-uzzezvpb
  Running command git rev-parse -q --verify 'sha^df7271285d3e29c13865515a51dbbd6e25f68f5f'
  Running command git fetch -q https://github.com/nvidia/Megatron-LM.git df7271285d3e29c13865515a51dbbd6e25f68f5f
  Running command git checkout -q df7271285d3e29c13865515a51dbbd6e25f68f5f
  Resolved https://github.com/nvidia/Megatron-LM.git to commit df7271285d3e29c13865515a51dbbd6e25f68f5f
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (from megatron-core==0.2.0) (2.1.0a0+32f93b1)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch->megatron-core==0.2.0) (3.12.4)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch->megatron-core==0.2.0) (4.7.1)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch->megatron-core==0.2.0) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch->megatron-core==0.2.0) (2.6.3)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch->megatron-core==0.2.0) (3.1.2)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch->megatron-core==0.2.0) (2023.6.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch->megatron-core==0.2.0) (2.1.3)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->megatron-core==0.2.0) (1.3.0)
Building wheels for collected packages: megatron-core
  Building wheel for megatron-core (setup.py): started
  Building wheel for megatron-core (setup.py): finished with status 'done'
  Created wheel for megatron-core: filename=megatron_core-0.2.0-py3-none-any.whl size=46963 sha256=00eb964bd103d1d378ed96dfe6b28f1336338f5d1aacb0e274759cf836ca4814
  Stored in directory: /tmp/pip-ephem-wheel-cache-6esx2ot8/wheels/bc/3c/10/115bf770cb93ea6cdb3a445b0b5a225c3e57f47b47b8102365
Successfully built megatron-core
Installing collected packages: megatron-core
Successfully installed megatron-core-0.2.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: python -m pip install --upgrade pip
Removing intermediate container e0915fa63bb6
 ---> 8c6113d6bcfe
Step 28/33 : COPY requirements.txt .
 ---> 1f2af1cc1670
Step 29/33 : RUN pip install --no-cache-dir -r requirements.txt
 ---> Running in 61e873a90514
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/mlcommons/[email protected] (from -r requirements.txt (line 1))
  Cloning https://github.com/mlcommons/logging.git (to revision 3.1.0-rc1) to /tmp/pip-req-build-23e7csbt
  Running command git clone --filter=blob:none --quiet https://github.com/mlcommons/logging.git /tmp/pip-req-build-23e7csbt
  Running command git checkout -q b32424904879020a47c8d9813b439e4e3017f8d5
  Resolved https://github.com/mlcommons/logging.git to commit b32424904879020a47c8d9813b439e4e3017f8d5
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting git+https://github.com/NVIDIA/mlperf-common.git (from -r requirements.txt (line 2))
  Cloning https://github.com/NVIDIA/mlperf-common.git to /tmp/pip-req-build-77cd6o7y
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA/mlperf-common.git /tmp/pip-req-build-77cd6o7y
  Resolved https://github.com/NVIDIA/mlperf-common.git to commit 779c29968d9dd08feaa099bf916439558a62a45c
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting zarr==2.13 (from -r requirements.txt (line 3))
  Downloading zarr-2.13.0-py3-none-any.whl (190 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 190.0/190.0 kB 6.4 MB/s eta 0:00:00
Collecting tensorstore==0.1.27 (from -r requirements.txt (line 4))
  Downloading tensorstore-0.1.27-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.1/8.1 MB 7.9 MB/s eta 0:00:00
Collecting asciitree (from zarr==2.13->-r requirements.txt (line 3))
  Downloading asciitree-0.3.3.tar.gz (4.0 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: numpy>=1.7 in /usr/local/lib/python3.10/dist-packages (from zarr==2.13->-r requirements.txt (line 3)) (1.22.2)
Collecting fasteners (from zarr==2.13->-r requirements.txt (line 3))
  Obtaining dependency information for fasteners from https://files.pythonhosted.org/packages/61/bf/fd60001b3abc5222d8eaa4a204cd8c0ae78e75adc688f33ce4bf25b7fafa/fasteners-0.19-py3-none-any.whl.metadata
  Downloading fasteners-0.19-py3-none-any.whl.metadata (4.9 kB)
Collecting numcodecs>=0.10.0 (from zarr==2.13->-r requirements.txt (line 3))
  Obtaining dependency information for numcodecs>=0.10.0 from https://files.pythonhosted.org/packages/1e/b8/1040f299803eacc9c522fdc69a4dafc42ad0e8722bb48aa43d2310cf195b/numcodecs-0.12.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading numcodecs-0.12.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.8 kB)
Requirement already satisfied: pandas>=1.0 in /usr/local/lib/python3.10/dist-packages (from mlperf-logging==3.0.0->-r requirements.txt (line 1)) (1.5.3)
Requirement already satisfied: pyyaml>=5.4.1 in /usr/local/lib/python3.10/dist-packages (from mlperf-logging==3.0.0->-r requirements.txt (line 1)) (6.0.1)
Requirement already satisfied: scipy>=1.4.1 in /usr/local/lib/python3.10/dist-packages (from mlperf-logging==3.0.0->-r requirements.txt (line 1)) (1.11.1)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.0->mlperf-logging==3.0.0->-r requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.0->mlperf-logging==3.0.0->-r requirements.txt (line 1)) (2023.3)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas>=1.0->mlperf-logging==3.0.0->-r requirements.txt (line 1)) (1.16.0)
Downloading numcodecs-0.12.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.7/7.7 MB 70.8 MB/s eta 0:00:00
Downloading fasteners-0.19-py3-none-any.whl (18 kB)
Building wheels for collected packages: mlperf-logging, mlperf-common, asciitree
  Building wheel for mlperf-logging (setup.py): started
  Building wheel for mlperf-logging (setup.py): finished with status 'done'
  Created wheel for mlperf-logging: filename=mlperf_logging-3.0.0-py3-none-any.whl size=238649 sha256=894120f86ec7aa4c068ad7ab33dd353c6715063b5c9bb4bce8ff237a1dc58f36
  Stored in directory: /tmp/pip-ephem-wheel-cache-lahb4x14/wheels/28/99/ec/54d1122b8daf8ece8026fcc2d28ef65d12ca3cb461d325fd30
  Building wheel for mlperf-common (setup.py): started
  Building wheel for mlperf-common (setup.py): finished with status 'done'
  Created wheel for mlperf-common: filename=mlperf_common-0.3-py3-none-any.whl size=23720 sha256=391ee9ac942451104ac22731207660d4333d4f9f7022362eb9b4edfceeb5f608
  Stored in directory: /tmp/pip-ephem-wheel-cache-lahb4x14/wheels/9b/bb/32/dd53ce122fd18a798e25c0afba97467ffb555bde95bc40cad1
  Building wheel for asciitree (setup.py): started
  Building wheel for asciitree (setup.py): finished with status 'done'
  Created wheel for asciitree: filename=asciitree-0.3.3-py3-none-any.whl size=5048 sha256=d483563353a0198882f71d427319ecefd0edb235fa5fd9ee1547a24d6cdd9cae
  Stored in directory: /tmp/pip-ephem-wheel-cache-lahb4x14/wheels/7f/4e/be/1171b40f43b918087657ec57cf3b81fa1a2e027d8755baa184
Successfully built mlperf-logging mlperf-common asciitree
Installing collected packages: asciitree, tensorstore, numcodecs, mlperf-common, fasteners, zarr, mlperf-logging
Successfully installed asciitree-0.3.3 fasteners-0.19 mlperf-common-0.3 mlperf-logging-3.0.0 numcodecs-0.12.1 tensorstore-0.1.27 zarr-2.13.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: python -m pip install --upgrade pip
Removing intermediate container 61e873a90514
 ---> b453680b78b7
Step 30/33 : RUN pip install jet-api --upgrade --extra-index-url https://sc-hw-artf.nvidia.com/api/pypi/hw-joc-pypi/simple --extra-index-url https://sc-hw-artf.nvidia.com/api/pypi/compute-pypi/simple
 ---> Running in 07239dfa8b52
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com, https://sc-hw-artf.nvidia.com/api/pypi/hw-joc-pypi/simple, https://sc-hw-artf.nvidia.com/api/pypi/compute-pypi/simple
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f1be4b5eef0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/pypi/hw-joc-pypi/simple/jet-api/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f1be4b5f1f0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/pypi/hw-joc-pypi/simple/jet-api/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f1be4ceef20>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/pypi/hw-joc-pypi/simple/jet-api/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f1be4b5f580>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/pypi/hw-joc-pypi/simple/jet-api/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f1be4b5f8b0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/pypi/hw-joc-pypi/simple/jet-api/
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f1be4b5fac0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/pypi/compute-pypi/simple/jet-api/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f1be4b5fdc0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/pypi/compute-pypi/simple/jet-api/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f1be4b5ffa0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/pypi/compute-pypi/simple/jet-api/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f1be4bb0160>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/pypi/compute-pypi/simple/jet-api/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f1be4bb0310>: Failed to establish a new connection: [Errno -2] Name or service not known')': /api/pypi/compute-pypi/simple/jet-api/
ERROR: Could not find a version that satisfies the requirement jet-api (from versions: none)
ERROR: No matching distribution found for jet-api

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: python -m pip install --upgrade pip
The command '/bin/sh -c pip install jet-api --upgrade --extra-index-url https://sc-hw-artf.nvidia.com/api/pypi/hw-joc-pypi/simple --extra-index-url https://sc-hw-artf.nvidia.com/api/pypi/compute-pypi/simple' returned a non-zero code: 1
@jndinesh
Copy link
Author

jndinesh commented Dec 17, 2023 •
edited
Loading

Further investigation revealed that the host sc-hw-artf.nvidia.com(https://sc-hw-artf.nvidia.com/) on port 443 is not listening.

Test 1:
curl -kv https://sc-hw-artf.nvidia.com/

  • Trying 143.244.220.150:443...
  • connect to 143.244.220.150 port 443 failed: Operation timed out
  • Failed to connect to sc-hw-artf.nvidia.com port 443 after 75039 ms: Couldn't connect to server
  • Closing connection 0
    curl: (28) Failed to connect to sc-hw-artf.nvidia.com port 443 after 75039 ms: Couldn't connect to server

Test 2:
nc -zv sc-hw-artf.nvidia.com 443
nc: connectx to sc-hw-artf.nvidia.com port 443 (tcp) failed: Operation timed out

@ShriyaPalsamudram
Copy link
Contributor

ShriyaPalsamudram commented Dec 18, 2023 •
edited
Loading

Submitted this PR: #6 to fix this issue, JET installation is not necessary for the benchmark so just deleting those lines should be sufficient. Please approve the PR if this fix works for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jndinesh @ShriyaPalsamudram