fix(huggingface-hub): allow offline mode by maxstrobel · Pull Request #11901 · NVIDIA-NeMo/NeMo

maxstrobel · 2025-01-20T15:07:36Z

Allow to reuse already cached models without the need for a connection to the internet or to the HuggingFace model hub.

What does this PR do ?

Allows to use cached models without internet connection

Collection: [Note which collection this PR will affect]

Changelog

Allow to reuse already cached models without the need for a connection to the internet or to the HuggingFace model hub.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

@ericharper, @titu1994, @blisc @okuchaiev

Additional Information

Fixes Broken offline mode of NeMo #11899

Allow to reuse already cached models without the need for a connection to the internet or to the HuggingFace model hub. Signed-off-by: Strobel Maximilian (IFAG PSS SIS SCE ACM) <Maximilian.Strobel@infineon.com>

Signed-off-by: maxstrobel <maxstrobel@users.noreply.github.com>

maxstrobel · 2025-01-30T15:59:47Z

push

github-actions · 2025-02-03T08:14:46Z

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

Your code was analyzed with PyLint. The following annotations have been identified:

************* Module nemo.core.classes.common
nemo/core/classes/common.py:688:0: C0301: Line too long (120/119) (line-too-long)
nemo/core/classes/common.py:814:0: C0301: Line too long (124/119) (line-too-long)
nemo/core/classes/common.py:471:0: C0115: Missing class docstring (missing-class-docstring)
nemo/core/classes/common.py:567:0: C0115: Missing class docstring (missing-class-docstring)
nemo/core/classes/common.py:642:0: C0115: Missing class docstring (missing-class-docstring)
nemo/core/classes/common.py:1023:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/core/classes/common.py:1138:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/core/classes/common.py:29:0: W0611: Unused Iterable imported from typing (unused-import)

-----------------------------------
Your code has been rated at 9.81/10

Mitigation guide:

Add sensible and useful docstrings to functions and methods
For trivial methods like getter/setters, consider adding # pylint: disable=C0116 inside the function itself
To disable multiple functions/methods at once, put a # pylint: disable=C0116 before the first and a # pylint: enable=C0116 after the last.

By applying these rules, we reduce the occurance of this message in future.

Thank you for improving NeMo's documentation!

github-actions · 2025-02-18T01:57:46Z

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

maxstrobel · 2025-02-20T10:12:11Z

ping @ericharper, @titu1994, @blisc @okuchaiev

github-actions · 2025-03-07T02:01:52Z

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

maxstrobel · 2025-03-07T13:27:25Z

@-ing some high contrib people that have been active recently for bump

@ko3n1g @akoumpa @cuichenx

nithinraok

LGTM, thanks much for the PR. and sorry for the delayed response on this.

Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>

github-actions · 2025-03-30T02:08:24Z

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

nithinraok

LGTM!

* fix(huggingface-hub): allow offline mode Allow to reuse already cached models without the need for a connection to the internet or to the HuggingFace model hub. Signed-off-by: Strobel Maximilian (IFAG PSS SIS SCE ACM) <Maximilian.Strobel@infineon.com> * Apply isort and black reformatting Signed-off-by: maxstrobel <maxstrobel@users.noreply.github.com> * flake8 common.py Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> * flake common.py Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> --------- Signed-off-by: Strobel Maximilian (IFAG PSS SIS SCE ACM) <Maximilian.Strobel@infineon.com> Signed-off-by: maxstrobel <maxstrobel@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: maxstrobel <maxstrobel@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Yuanzhe Dong <yudong@nvidia.com>

* fix(huggingface-hub): allow offline mode Allow to reuse already cached models without the need for a connection to the internet or to the HuggingFace model hub. Signed-off-by: Strobel Maximilian (IFAG PSS SIS SCE ACM) <Maximilian.Strobel@infineon.com> * Apply isort and black reformatting Signed-off-by: maxstrobel <maxstrobel@users.noreply.github.com> * flake8 common.py Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> * flake common.py Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> --------- Signed-off-by: Strobel Maximilian (IFAG PSS SIS SCE ACM) <Maximilian.Strobel@infineon.com> Signed-off-by: maxstrobel <maxstrobel@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: maxstrobel <maxstrobel@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>

* refactor 1: typos, yaml updates, code changes Signed-off-by: Jason <jasoli@nvidia.com> * add new test Signed-off-by: Jason <jasoli@nvidia.com> * add the new tests Signed-off-by: Jason <jasoli@nvidia.com> * pull in latest container build from main; hopefully it works without merging the rest of main Signed-off-by: Jason <jasoli@nvidia.com> * update repo location Signed-off-by: Jason <jasoli@nvidia.com> * try removing wait-in-queue from needs Signed-off-by: Jason <jasoli@nvidia.com> * fix tests Signed-off-by: Jason <jasoli@nvidia.com> * Update nemo/collections/tts/data/text_to_speech_dataset.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * undo dataset change Signed-off-by: Jason <jasoli@nvidia.com> * remove infer and eval tests for now Signed-off-by: Jason <jasoli@nvidia.com> * fix(huggingface-hub): allow offline mode (#11901) * fix(huggingface-hub): allow offline mode Allow to reuse already cached models without the need for a connection to the internet or to the HuggingFace model hub. Signed-off-by: Strobel Maximilian (IFAG PSS SIS SCE ACM) <Maximilian.Strobel@infineon.com> * Apply isort and black reformatting Signed-off-by: maxstrobel <maxstrobel@users.noreply.github.com> * flake8 common.py Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> * flake common.py Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> --------- Signed-off-by: Strobel Maximilian (IFAG PSS SIS SCE ACM) <Maximilian.Strobel@infineon.com> Signed-off-by: maxstrobel <maxstrobel@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: maxstrobel <maxstrobel@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> * fix L2 tests to use new json, cleanup disk usage, and add target checks Signed-off-by: Jason <jasoli@nvidia.com> * undo comment Signed-off-by: Jason <jasoli@nvidia.com> --------- Signed-off-by: Jason <jasoli@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Strobel Maximilian (IFAG PSS SIS SCE ACM) <Maximilian.Strobel@infineon.com> Signed-off-by: maxstrobel <maxstrobel@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Max Strobel <max.strobel28@gmail.com> Co-authored-by: maxstrobel <maxstrobel@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>

- Remove numpy < 2 constraint from k2-asr and nemo-asr packages - Update nemo_toolkit[asr] from < 2.2 to >= 2.6.1 - Includes ASR numpy 2.0 support (added in v2.6.1) - Includes offline mode fix (NVIDIA-NeMo/NeMo#11901, added in v2.4.0)

fix(huggingface-hub): allow offline mode

a8611ea

Allow to reuse already cached models without the need for a connection to the internet or to the HuggingFace model hub. Signed-off-by: Strobel Maximilian (IFAG PSS SIS SCE ACM) <Maximilian.Strobel@infineon.com>

github-actions Bot added the core Changes to NeMo Core label Jan 20, 2025

Apply isort and black reformatting

f978b20

Signed-off-by: maxstrobel <maxstrobel@users.noreply.github.com>

Merge branch 'main' into fix/offline-mode

57c7bab

github-actions Bot added the stale label Feb 18, 2025

github-actions Bot removed the stale label Feb 21, 2025

github-actions Bot added the stale label Mar 7, 2025

Merge branch 'main' into fix/offline-mode

47827ad

github-actions Bot removed the stale label Mar 8, 2025

nithinraok previously approved these changes Mar 10, 2025

View reviewed changes

nithinraok added the Run CICD label Mar 10, 2025

Merge branch 'main' into fix/offline-mode

67a1560

ko3n1g added Run CICD and removed Run CICD labels Mar 13, 2025

flake8 common.py

ee7bdbe

Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>

akoumpa dismissed nithinraok’s stale review via ee7bdbe March 15, 2025 04:40

ko3n1g added Run CICD and removed Run CICD labels Mar 15, 2025

flake common.py

f567bb3

Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>

ko3n1g added Run CICD and removed Run CICD labels Mar 15, 2025

github-actions Bot added the stale label Mar 30, 2025

nithinraok removed the stale label Mar 30, 2025

ko3n1g added Run CICD and removed Run CICD labels Apr 9, 2025

chtruong814 added Run CICD and removed Run CICD labels Apr 10, 2025

ko3n1g added Run CICD and removed Run CICD labels Apr 10, 2025

chtruong814 added Run CICD and removed Run CICD labels Apr 10, 2025

nithinraok approved these changes Apr 11, 2025

View reviewed changes

chtruong814 added Run CICD and removed Run CICD labels Apr 12, 2025

cuichenx merged commit 7d9e572 into NVIDIA-NeMo:main Apr 17, 2025
419 of 1936 checks passed

This was referenced Aug 6, 2025

[Feature Request] Support offline model loading for cached models modular/modular#5149

Closed

[Feature Request] Support offline model loading for cached models swi/modular#1

Open

aikawa-ohno mentioned this pull request Oct 21, 2025

NeMoオフラインエラー reazon-research/ReazonSpeech#71

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(huggingface-hub): allow offline mode#11901

fix(huggingface-hub): allow offline mode#11901
cuichenx merged 8 commits intoNVIDIA-NeMo:mainfrom
maxstrobel:fix/offline-mode

maxstrobel commented Jan 20, 2025

Uh oh!

maxstrobel commented Jan 30, 2025

Uh oh!

github-actions Bot commented Feb 3, 2025

Uh oh!

github-actions Bot commented Feb 18, 2025

Uh oh!

maxstrobel commented Feb 20, 2025

Uh oh!

github-actions Bot commented Mar 7, 2025

Uh oh!

maxstrobel commented Mar 7, 2025

Uh oh!

nithinraok left a comment

Uh oh!

github-actions Bot commented Mar 30, 2025

Uh oh!

nithinraok left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

maxstrobel commented Jan 20, 2025

What does this PR do ?

Changelog

Usage

GitHub Actions CI

Before your PR is "Ready for review"

Who can review?

Additional Information

Uh oh!

maxstrobel commented Jan 30, 2025

Uh oh!

github-actions Bot commented Feb 3, 2025

Uh oh!

github-actions Bot commented Feb 18, 2025

Uh oh!

maxstrobel commented Feb 20, 2025

Uh oh!

github-actions Bot commented Mar 7, 2025

Uh oh!

maxstrobel commented Mar 7, 2025

Uh oh!

nithinraok left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Mar 30, 2025

Uh oh!

nithinraok left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants