Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fixes to accelerator so that iterable pretraining datasets work #1759

Merged
merged 7 commits into from
Jul 17, 2024
Merged

fixes to accelerator so that iterable pretraining datasets work #1759

merged 7 commits into from
Jul 17, 2024

Commits on Jul 17, 2024

  1. fixes to accelerator so that iterable pretraining datasets work

    @winglian
    winglian committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    1be7132 View commit details
    Browse the repository at this point in the history

  2. fix the pretraining test params

    @winglian
    winglian committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    02d99af View commit details
    Browse the repository at this point in the history

  3. split batches, not dispatch batches needs to be set

    @winglian
    winglian committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    69d827e View commit details
    Browse the repository at this point in the history

  4. update c4 datasets

    @winglian
    winglian committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    9466e68 View commit details
    Browse the repository at this point in the history

  5. set epochs in pretrain config test

    @winglian
    winglian committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    c7f1776 View commit details
    Browse the repository at this point in the history

  6. need to set both split_batches and dispatch_batches to false for pret… 

    …raining
    @winglian
    winglian committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    e757bad View commit details
    Browse the repository at this point in the history

  7. fix bool val in comment [skip ci]

    @winglian
    winglian committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    c32aa18 View commit details
    Browse the repository at this point in the history