Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Swin Transformer #16

Merged
merged 13 commits into from
Aug 19, 2023
Merged

Add Swin Transformer #16

merged 13 commits into from
Aug 19, 2023

Commits on Aug 8, 2023

  1. add initial window attention

    @gau-nernst
    gau-nernst committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    fa64602 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2023

  1. move MLP to vit.py. fix dropout during inference

    @gau-nernst
    gau-nernst committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    6413aa7 View commit details
    Browse the repository at this point in the history

  2. add dropout to MLP

    @gau-nernst
    gau-nernst committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    a41efbe View commit details
    Browse the repository at this point in the history

  3. add window partition and unpartition

    @gau-nernst
    gau-nernst committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    443f3c1 View commit details
    Browse the repository at this point in the history

  4. add shifted window. add more stuff

    @gau-nernst
    gau-nernst committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    9fcb837 View commit details
    Browse the repository at this point in the history

  5. make swin run. fix attn_bias

    @gau-nernst
    gau-nernst committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    bfc91e5 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2023

  1. add S3 config

    @gau-nernst
    gau-nernst committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    7948762 View commit details
    Browse the repository at this point in the history

  2. add downsampling at beginning of stage

    @gau-nernst
    gau-nernst committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    232a50b View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. add Swin

    @gau-nernst
    gau-nernst committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    fd9a270 View commit details
    Browse the repository at this point in the history

  2. add WIP weight loading

    @gau-nernst
    gau-nernst committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    0e95db4 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2023

  1. fix some bugs with Swin

    @gau-nernst
    gau-nernst committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    c90825f View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2023

  1. don't shift when input_size = window_size

    @gau-nernst
    gau-nernst committed Aug 19, 2023
    Configuration menu
    Copy the full SHA
    f6e48d8 View commit details
    Browse the repository at this point in the history

  2. use faster weight transpose

    @gau-nernst
    gau-nernst committed Aug 19, 2023
    Configuration menu
    Copy the full SHA
    b964ffe View commit details
    Browse the repository at this point in the history