Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Core] Registry for processing model inputs #5214

Merged
merged 63 commits into from
Jun 28, 2024
Merged

[Core] Registry for processing model inputs #5214

merged 63 commits into from
Jun 28, 2024

Commits on Jun 3, 2024

  1. Introduce a higher level INPUT_REGISTRY

    @DarkLight1337
    DarkLight1337 committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    34bfa79 View commit details
    Browse the repository at this point in the history

  2. Move dummy data generation to input registry

    @DarkLight1337
    DarkLight1337 committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    df2aa19 View commit details
    Browse the repository at this point in the history

  3. Update docs

    @DarkLight1337
    DarkLight1337 committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    c72d2b3 View commit details
    Browse the repository at this point in the history

  4. Rename process_input to map_input

    @DarkLight1337
    DarkLight1337 committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    d8c6488 View commit details
    Browse the repository at this point in the history

  5. Reorder arguments

    @DarkLight1337
    DarkLight1337 committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    f18de48 View commit details
    Browse the repository at this point in the history

  6. Apply input processor

    @DarkLight1337
    DarkLight1337 committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    653537d View commit details
    Browse the repository at this point in the history

  7. Remove VisionLanguageConfig from input mapper

    @DarkLight1337
    DarkLight1337 committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    a2f5a3c View commit details
    Browse the repository at this point in the history

  8. Fix bad use of functools.partial

    @DarkLight1337
    DarkLight1337 committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    378ad80 View commit details
    Browse the repository at this point in the history

  9. Use default input processor

    @DarkLight1337
    DarkLight1337 committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    7aa3778 View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2024

  1. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    c774168 View commit details
    Browse the repository at this point in the history

  2. Fix wrong arguments

    @DarkLight1337
    DarkLight1337 committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    532f863 View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2024

  1. Use pillow image instead of tensor to avoid bypassing the processor b… 

    …y default
    @DarkLight1337
    DarkLight1337 committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    080d40c View commit details
    Browse the repository at this point in the history

  2. Update interface of dummy data factory and input processor

    @DarkLight1337
    DarkLight1337 committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    662693a View commit details
    Browse the repository at this point in the history

  3. Use InputContext to handle checked type cast of config types

    @DarkLight1337
    DarkLight1337 committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    9bc5fcc View commit details
    Browse the repository at this point in the history

  4. Fix LLaVA-NeXT input processor and cleanup code

    @DarkLight1337
    DarkLight1337 committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    29c3bb3 View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2024

  1. Add sanity check

    @DarkLight1337
    DarkLight1337 committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    7bb6cbf View commit details
    Browse the repository at this point in the history

  2. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    ccf49c4 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    3482d32 View commit details
    Browse the repository at this point in the history

Commits on Jun 8, 2024

  1. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 8, 2024
    Configuration menu
    Copy the full SHA
    8ea8468 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 8, 2024
    Configuration menu
    Copy the full SHA
    be3d64f View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2024

  1. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    2ff5be6 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Update LLaVA-NeXT

    @DarkLight1337
    DarkLight1337 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    8e2ff86 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    553f684 View commit details
    Browse the repository at this point in the history

  3. Update name

    @DarkLight1337
    DarkLight1337 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    b134dfc View commit details
    Browse the repository at this point in the history

  4. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    7e33706 View commit details
    Browse the repository at this point in the history

  5. Remove MULTIMODAL convenience property as it was causing some (impo… 

    …rt-related?) problems with ray workers
    @DarkLight1337
    DarkLight1337 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    3fb622c View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Add docs

    @DarkLight1337
    DarkLight1337 committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    6a70e4f View commit details
    Browse the repository at this point in the history

  2. Add docs

    @DarkLight1337
    DarkLight1337 committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    52a0116 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    b7a8683 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2024

  1. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    25f9949 View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2024

  1. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 15, 2024
    Configuration menu
    Copy the full SHA
    fd7d954 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 15, 2024
    Configuration menu
    Copy the full SHA
    49dac3e View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    0104218 View commit details
    Browse the repository at this point in the history

  2. Set up dummy data factory for phi3v

    @DarkLight1337
    DarkLight1337 committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    18cc7e0 View commit details
    Browse the repository at this point in the history

  3. Move dummy data factories to model files

    @DarkLight1337
    DarkLight1337 committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    2291617 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    adf5503 View commit details
    Browse the repository at this point in the history

  5. Fix wrong feature size

    @DarkLight1337
    DarkLight1337 committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    fecf1f0 View commit details
    Browse the repository at this point in the history

  6. Fix wrong feature size

    @DarkLight1337
    DarkLight1337 committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    086e0fe View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    c036b86 View commit details
    Browse the repository at this point in the history

  2. Remove redundant code

    @DarkLight1337
    DarkLight1337 committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    bfa5aa9 View commit details
    Browse the repository at this point in the history

  3. Apply isort

    @DarkLight1337
    DarkLight1337 committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    07e695d View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Move DummyImageDataFactories into CLIP model file

    @DarkLight1337
    DarkLight1337 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    7229b07 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    d9a4150 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    4b947ad View commit details
    Browse the repository at this point in the history

  3. Clarify docs and add todo

    @DarkLight1337
    DarkLight1337 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    9e82a26 View commit details
    Browse the repository at this point in the history

  4. Expand docs

    @DarkLight1337
    DarkLight1337 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    6b19e6c View commit details
    Browse the repository at this point in the history

  5. Add ref

    @DarkLight1337
    DarkLight1337 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    f451668 View commit details
    Browse the repository at this point in the history

  6. Add docs

    @DarkLight1337
    DarkLight1337 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    1abb8a7 View commit details
    Browse the repository at this point in the history

  7. Fix name

    @DarkLight1337
    DarkLight1337 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    698830f View commit details
    Browse the repository at this point in the history

  8. Fix and add links

    @DarkLight1337
    DarkLight1337 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    36ab12d View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. modify llava_next

    @ywang96
    ywang96 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    bf3281c View commit details
    Browse the repository at this point in the history

  2. Update comment

    @DarkLight1337
    DarkLight1337 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    56e2d3b View commit details
    Browse the repository at this point in the history

  3. Update docs

    @DarkLight1337
    DarkLight1337 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    d2f8c6d View commit details
    Browse the repository at this point in the history

  4. Use dynamic image feature size calculation

    @DarkLight1337
    DarkLight1337 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    7c197d2 View commit details
    Browse the repository at this point in the history

  5. Fix phi3v not handling image_sizes correctly

    @DarkLight1337
    DarkLight1337 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    f5ffd3e View commit details
    Browse the repository at this point in the history

  6. Apply formatter

    @DarkLight1337
    DarkLight1337 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    66aad21 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    f2e4633 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    a6e3162 View commit details
    Browse the repository at this point in the history

  9. Fix config

    @DarkLight1337
    DarkLight1337 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    ce06541 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    7e80ecc View commit details
    Browse the repository at this point in the history

  2. Merge branch 'upstream' into mm-image-tokenizer

    @DarkLight1337
    DarkLight1337 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    487d742 View commit details
    Browse the repository at this point in the history

  3. update example

    @ywang96
    ywang96 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    43350b8 View commit details
    Browse the repository at this point in the history

  4. update doc

    @ywang96
    ywang96 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    57791de View commit details
    Browse the repository at this point in the history