Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BERT Model To NeMo2.0 #11333

Merged
merged 59 commits into from
Dec 5, 2024
Merged

Add BERT Model To NeMo2.0 #11333

merged 59 commits into from
Dec 5, 2024

Commits on Sep 4, 2024

  1. add bert in nemo2

    @suiyoubi
    suiyoubi committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    b922807 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Add bert loss

    @suiyoubi
    suiyoubi committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    0e0822c View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2024

  1. Merge branch 'main' into aot/bert-nemo-ux 

    # Conflicts:
#	nemo/collections/llm/__init__.py
    @suiyoubi
    suiyoubi committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    ccab2d5 View commit details
    Browse the repository at this point in the history

  2. update bert with latest nemo

    @suiyoubi
    suiyoubi committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    70973cd View commit details
    Browse the repository at this point in the history

  3. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    686eed1 View commit details
    Browse the repository at this point in the history

Commits on Nov 18, 2024

  1. Bert update

    @suiyoubi
    suiyoubi committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    794951e View commit details
    Browse the repository at this point in the history

  2. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    bf91626 View commit details
    Browse the repository at this point in the history

  3. import optimize

    @suiyoubi
    suiyoubi committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    c2b2949 View commit details
    Browse the repository at this point in the history

  4. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    f64457f View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2024

  1. pylint

    @suiyoubi
    suiyoubi committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    d70c098 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/aot/bert-nemo-ux' into aot/bert-… 

    …nemo-ux
    @suiyoubi
    suiyoubi committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    27707f5 View commit details
    Browse the repository at this point in the history

  3. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    b91c247 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/main' into aot/bert-nemo-ux 

    # Conflicts:
#	nemo/lightning/megatron_parallel.py
    @suiyoubi
    suiyoubi committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    146c21b View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'origin/aot/bert-nemo-ux' into aot/bert-… 

    …nemo-ux
    @suiyoubi
    suiyoubi committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    3a8841c View commit details
    Browse the repository at this point in the history

  6. pylint

    @suiyoubi
    suiyoubi committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    3247f36 View commit details
    Browse the repository at this point in the history

  7. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    bcc9b7c View commit details
    Browse the repository at this point in the history

  8. pylint

    @suiyoubi
    suiyoubi committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    5f19763 View commit details
    Browse the repository at this point in the history

  9. Merge remote-tracking branch 'origin/aot/bert-nemo-ux' into aot/bert-… 

    …nemo-ux
    @suiyoubi
    suiyoubi committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    9bbef09 View commit details
    Browse the repository at this point in the history

  10. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    f42ec30 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2024

  1. more comment

    @suiyoubi
    suiyoubi committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    388ed8a View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/aot/bert-nemo-ux' into aot/bert-… 

    …nemo-ux
    @suiyoubi
    suiyoubi committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    11a141e View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2024

  1. Merge remote-tracking branch 'origin/main' into aot/bert-nemo-ux

    @suiyoubi
    suiyoubi committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    8dad5a5 View commit details
    Browse the repository at this point in the history

  2. use lightning package

    @suiyoubi
    suiyoubi committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    f00d2df View commit details
    Browse the repository at this point in the history

  3. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    ba73a4d View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2024

  1. comments resolved

    @suiyoubi
    suiyoubi committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    546a646 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/aot/bert-nemo-ux' into aot/bert-… 

    …nemo-ux
    @suiyoubi
    suiyoubi committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    2cf371d View commit details
    Browse the repository at this point in the history

  3. Add loss

    @suiyoubi
    suiyoubi committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    659e29c View commit details
    Browse the repository at this point in the history

  4. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    2c542d0 View commit details
    Browse the repository at this point in the history

  5. Fix NaN loss when resume

    @suiyoubi
    suiyoubi committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    451a1a0 View commit details
    Browse the repository at this point in the history

  6. Merge remote-tracking branch 'origin/aot/bert-nemo-ux' into aot/bert-… 

    …nemo-ux
    @suiyoubi
    suiyoubi committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    c72f1be View commit details
    Browse the repository at this point in the history

  7. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    d2bbe9c View commit details
    Browse the repository at this point in the history

  8. add default for num_tokentypes

    @suiyoubi
    suiyoubi committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    846b236 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2024

  1. update forward output to be a dict

    @suiyoubi
    suiyoubi committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    cc1893d View commit details
    Browse the repository at this point in the history

  2. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    9654cfd View commit details
    Browse the repository at this point in the history

  3. ReName to HuggingFace Bert Model

    @suiyoubi
    suiyoubi committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    8658f53 View commit details
    Browse the repository at this point in the history

Commits on Nov 26, 2024

  1. Add BertLarge

    @suiyoubi
    suiyoubi committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    cb1332d View commit details
    Browse the repository at this point in the history

  2. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    7ae576c View commit details
    Browse the repository at this point in the history

  3. Add BERT Tests

    @suiyoubi
    suiyoubi committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    3826aa3 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/aot/bert-nemo-ux' into aot/bert-… 

    …nemo-ux
    @suiyoubi
    suiyoubi committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    51692ad View commit details
    Browse the repository at this point in the history

  5. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    ea969ed View commit details
    Browse the repository at this point in the history

  6. typo

    @suiyoubi
    suiyoubi committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    06f03ff View commit details
    Browse the repository at this point in the history

  7. add exporter

    @suiyoubi
    suiyoubi committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    f61877a View commit details
    Browse the repository at this point in the history

  8. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    79acd82 View commit details
    Browse the repository at this point in the history

  9. Fix Unit Tests

    @suiyoubi
    suiyoubi committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    19337e3 View commit details
    Browse the repository at this point in the history

  10. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    b15051d View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2024

  1. add sig

    @suiyoubi
    suiyoubi committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    cca4c79 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2024

  1. rename pretraining dataset testing for bert

    @suiyoubi
    suiyoubi committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    db19dd1 View commit details
    Browse the repository at this point in the history

  2. delete pretraining dataset testing for bert

    @suiyoubi
    suiyoubi committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    a2a859c View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into aot/bert-nemo-ux

    @suiyoubi
    suiyoubi authored Nov 28, 2024
    Configuration menu
    Copy the full SHA
    1e9bc3f View commit details
    Browse the repository at this point in the history

Commits on Dec 2, 2024

  1. resolve TE dependency

    @suiyoubi
    suiyoubi committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    69b6128 View commit details
    Browse the repository at this point in the history

  2. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    27b56a9 View commit details
    Browse the repository at this point in the history

  3. resolve TE dependency

    @suiyoubi
    suiyoubi committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    56fa3f1 View commit details
    Browse the repository at this point in the history

  4. resolve TE dependency

    @suiyoubi
    suiyoubi committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    9b11d84 View commit details
    Browse the repository at this point in the history

  5. resolve TE dependency

    @suiyoubi
    suiyoubi committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    4a7f608 View commit details
    Browse the repository at this point in the history

  6. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    5acf9aa View commit details
    Browse the repository at this point in the history

  7. resolve TE dependency

    @suiyoubi
    suiyoubi committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    d363a11 View commit details
    Browse the repository at this point in the history

  8. Apply isort and black reformatting 

    Signed-off-by: suiyoubi <[email protected]>
    @suiyoubi
    suiyoubi committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    c0a7836 View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2024

  1. Merge branch 'main' into aot/bert-nemo-ux

    @suiyoubi
    suiyoubi authored Dec 3, 2024
    Configuration menu
    Copy the full SHA
    de1c788 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into aot/bert-nemo-ux

    @suiyoubi
    suiyoubi authored Dec 3, 2024
    Configuration menu
    Copy the full SHA
    8e28ff5 View commit details
    Browse the repository at this point in the history