Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add rwkv support #1198

Merged
merged 26 commits into from
May 6, 2024
Merged

add rwkv support #1198

merged 26 commits into from
May 6, 2024

Commits on Mar 31, 2024

  1. add rwkv support

    @jahatef
    jahatef committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    2741e89 View commit details
    Browse the repository at this point in the history

  2. Update init_functions.py

    @jahatef
    jahatef committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    282f800 View commit details
    Browse the repository at this point in the history

  3. rwkv model files

    @jahatef
    jahatef committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    09ba65e View commit details
    Browse the repository at this point in the history

  4. Merge branch 'rwkv' of github.com:EleutherAI/gpt-neox into rwkv

    @jahatef
    jahatef committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    8892432 View commit details
    Browse the repository at this point in the history

  5. configs

    @jahatef
    jahatef committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    04b8fdb View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. kernels

    @jahatef
    jahatef committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    6e79cc2 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. Cleanup

    @jahatef
    jahatef committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    cb49ff6 View commit details
    Browse the repository at this point in the history

  2. Update 760M.yml

    @jahatef
    jahatef committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    96ea6f5 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. remove preffn and mishglu

    @jahatef
    jahatef committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    54f2775 View commit details
    Browse the repository at this point in the history

  2. merge

    @jahatef
    jahatef committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    3606bdd View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. Merge branch 'main' into rwkv

    @Quentin-Anthony
    Quentin-Anthony committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    8d60cef View commit details
    Browse the repository at this point in the history

  2. Update NeoXArgs docs automatically

    github-actions committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    276ffa9 View commit details
    Browse the repository at this point in the history

  3. Add RWKV parallelism assertions

    @Quentin-Anthony
    Quentin-Anthony committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    e20138c View commit details
    Browse the repository at this point in the history

  4. Update NeoXArgs docs automatically

    github-actions committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    428aad5 View commit details
    Browse the repository at this point in the history

  5. pre-commit and config cleanup

    @Quentin-Anthony
    Quentin-Anthony committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    1b0bbab View commit details
    Browse the repository at this point in the history

  6. Merge branch 'rwkv' of https://github.com/EleutherAI/gpt-neox into rwkv

    @Quentin-Anthony
    Quentin-Anthony committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    7550d64 View commit details
    Browse the repository at this point in the history

  7. Update NeoXArgs docs automatically

    github-actions committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    c0af563 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. rwkv logging

    @jahatef
    jahatef committed May 3, 2024
    Configuration menu
    Copy the full SHA
    1eb5f51 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rwkv' of https://github.com/EleutherAI/gpt-neox into rwkv

    @jahatef
    jahatef committed May 3, 2024
    Configuration menu
    Copy the full SHA
    330a802 View commit details
    Browse the repository at this point in the history

Commits on May 4, 2024

  1. Merge branch 'main' into rwkv

    @Quentin-Anthony
    Quentin-Anthony committed May 4, 2024
    Configuration menu
    Copy the full SHA
    1103663 View commit details
    Browse the repository at this point in the history

  2. Update NeoXArgs docs automatically

    github-actions committed May 4, 2024
    Configuration menu
    Copy the full SHA
    a599ac7 View commit details
    Browse the repository at this point in the history

  3. Add rwkv version dirname, make hdim 3.5x

    @Quentin-Anthony
    Quentin-Anthony committed May 4, 2024
    Configuration menu
    Copy the full SHA
    682f7e5 View commit details
    Browse the repository at this point in the history

  4. pre-commit

    @Quentin-Anthony
    Quentin-Anthony committed May 4, 2024
    Configuration menu
    Copy the full SHA
    921c41a View commit details
    Browse the repository at this point in the history

  5. Update NeoXArgs docs automatically

    github-actions committed May 4, 2024
    Configuration menu
    Copy the full SHA
    8f60a43 View commit details
    Browse the repository at this point in the history

Commits on May 5, 2024

  1. fix bug and set batch size to 32

    @jahatef
    jahatef committed May 5, 2024
    Configuration menu
    Copy the full SHA
    6fb840e View commit details
    Browse the repository at this point in the history

  2. Update NeoXArgs docs automatically

    github-actions committed May 5, 2024
    Configuration menu
    Copy the full SHA
    dd0138e View commit details
    Browse the repository at this point in the history