Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WebGPU EP] Support GroupQueryAttention #22658

Open
wants to merge 44 commits into
base: main
Choose a base branch
from
Open

[WebGPU EP] Support GroupQueryAttention #22658

wants to merge 44 commits into from
+962 −517

Commits on Nov 13, 2024

  1. Added attention_common.h

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    0a5d212 View commit details
    Browse the repository at this point in the history

  2. wip

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    5bfa070 View commit details
    Browse the repository at this point in the history

  3. Fix compilation errors

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    e6615e9 View commit details
    Browse the repository at this point in the history

  4. lint

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    449afb4 View commit details
    Browse the repository at this point in the history

  5. Modified MultiHeadAttention to not derive from AttentionBase class

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    8d10472 View commit details
    Browse the repository at this point in the history

  6. Uncomment GQA registration

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    4ea58d1 View commit details
    Browse the repository at this point in the history

  7. Moved TransferBSToBNSH and ApplyAttention declaration to attention_co… 

    …mmon.h from attention.h
    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    4bcf257 View commit details
    Browse the repository at this point in the history

  8. Revert "Modified MultiHeadAttention to not derive from AttentionBase … 

    …class"

This reverts commit ba45303.
    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    5c5c934 View commit details
    Browse the repository at this point in the history

  9. Converted CheckInput function to template to fix compiler/linker mult… 

    …iple definitions error.
    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    e716546 View commit details
    Browse the repository at this point in the history

  10. lint

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    aba59e5 View commit details
    Browse the repository at this point in the history

  11. Fixed conflicts.

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    067ecd1 View commit details
    Browse the repository at this point in the history

  12. copying errors

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    53f1c78 View commit details
    Browse the repository at this point in the history

  13. Fixed inplacesoftmax dispatch

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    f4dc9fc View commit details
    Browse the repository at this point in the history

  14. Initialize required parameter data

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    3d1af1c View commit details
    Browse the repository at this point in the history

  15. Map total_seqlen_tensor input to CPU

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    2eaeebc View commit details
    Browse the repository at this point in the history

  16. Use uniforms variable name consistently to avoid confusion.

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    9c828cc View commit details
    Browse the repository at this point in the history

  17. Keep InplaceSoftmax dispatch 3-dim.

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    26caa06 View commit details
    Browse the repository at this point in the history

  18. Formatting changes.

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    64b093f View commit details
    Browse the repository at this point in the history

  19. Use total_seqlen_tensor input only to determin is_first_prompt.

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    a8bd38b View commit details
    Browse the repository at this point in the history

  20. initialize is_packed_qkv_

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    d613df4 View commit details
    Browse the repository at this point in the history

  21. Handle past key/value and present key/value buffer sharing.

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    0fedb9f View commit details
    Browse the repository at this point in the history

  22. lint

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    993140b View commit details
    Browse the repository at this point in the history

  23. Added past_present_share_buffer to the hint. typo

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    7502493 View commit details
    Browse the repository at this point in the history

  24. past_present_share_buffer related changes.

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    5f1fdae View commit details
    Browse the repository at this point in the history

  25. lint

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    6d2bd68 View commit details
    Browse the repository at this point in the history

  26. Fix integer division

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    82a005d View commit details
    Browse the repository at this point in the history

  27. Updated hints

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    fd9409f View commit details
    Browse the repository at this point in the history

  28. match jsep code

    @satyajandhyala
    satyajandhyala committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    15c96b3 View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2024

  1. Fixed a minor issue

    @satyajandhyala
    satyajandhyala committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    72601d1 View commit details
    Browse the repository at this point in the history

  2. lint

    @satyajandhyala
    satyajandhyala committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    65495b6 View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2024

  1. Fix a bug using total_sequence_length instead of uniform.total_sequen… 

    …ce_length_comp
    @satyajandhyala
    satyajandhyala committed Nov 16, 2024
    Configuration menu
    Copy the full SHA
    63f20ed View commit details
    Browse the repository at this point in the history

  2. Revert "match jsep code" 

    This reverts commit 15c96b3.
    @satyajandhyala
    satyajandhyala committed Nov 16, 2024
    Configuration menu
    Copy the full SHA
    0102206 View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2024

  1. Removed is_first_prompt from uniforms.

    @satyajandhyala
    satyajandhyala committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    71ed10c View commit details
    Browse the repository at this point in the history

Commits on Nov 18, 2024

  1. Updated hints

    @satyajandhyala
    satyajandhyala committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    9c08c82 View commit details
    Browse the repository at this point in the history

  2. Use kv_num_heads instead num_heads for key/value input shape conversion.

    @satyajandhyala
    satyajandhyala committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    eb5d7b4 View commit details
    Browse the repository at this point in the history

  3. lint

    @satyajandhyala
    satyajandhyala committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    7a2d3b6 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of https://github.com/microsoft/onnxruntime into … 

    …sajandhy/webgpu-ep-gqa-new
    @satyajandhyala
    satyajandhyala committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    664022f View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2024

  1. changed variable name

    @satyajandhyala
    satyajandhyala committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    a48d782 View commit details
    Browse the repository at this point in the history

  2. Removed is_first_prompt from uniforms, used in a condition generating… 

    … shader code and added to hint.
    @satyajandhyala
    satyajandhyala committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    4334b39 View commit details
    Browse the repository at this point in the history

  3. error

    @satyajandhyala
    satyajandhyala committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    d53d7ef View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2024

  1. initialize scale

    @satyajandhyala
    satyajandhyala committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    5dc95c8 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2024

  1. Calculate output chunk size based on whether the kernel is GQA or not.

    @satyajandhyala
    satyajandhyala committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    e448b1a View commit details
    Browse the repository at this point in the history

Commits on Nov 23, 2024

  1. Revert "Calculate output chunk size based on whether the kernel is GQ… 

    …A or not."

This reverts commit e448b1a.
    @satyajandhyala
    satyajandhyala committed Nov 23, 2024
    Configuration menu
    Copy the full SHA
    60af2f5 View commit details
    Browse the repository at this point in the history

  2. Bug fix

    @satyajandhyala
    satyajandhyala committed Nov 23, 2024
    Configuration menu
    Copy the full SHA
    47e6f52 View commit details
    Browse the repository at this point in the history