Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial version of the HTTP API + DF upgrade #31

Merged
merged 19 commits into from
Jul 27, 2022
Merged

Initial version of the HTTP API + DF upgrade #31

merged 19 commits into from
Jul 27, 2022

Commits on Jul 27, 2022

  1. Add a Warp server to execute SELECT queries

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    8fc9577 View commit details
    Browse the repository at this point in the history

  2. Register the default DataFusion catalog in main

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    7caaf22 View commit details
    Browse the repository at this point in the history

  3. Add configuration parsing for the HTTP frontend

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    4bac231 View commit details
    Browse the repository at this point in the history

  4. Move the HTTP frontend code to the module

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    d6fd956 View commit details
    Browse the repository at this point in the history

  5. Add Clone to Config instances

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    4be7e90 View commit details
    Browse the repository at this point in the history

  6. Pass the config to the HTTP frontend

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    6b500ac View commit details
    Browse the repository at this point in the history

  7. Run the PG/HTTP frontends selectively in main 

    Because we prepare a vector of all futures (PG/HTTP frontends), there's a
lifetime issue with the config structs that I don't completely understand
(they basically need to live as long as the `'static` lifetime). We can't pass
them by reference to the PG/HTTP frontend because that reference might not live
as long as static, since we're doing this as a separate function that's called
from main (?)). Our `prepare_frontends` function can't own the config either
(it has to outlive it). To fix this, we pass the config to the PG/HTTP frontends
by-value and clone it.
    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    d7aa069 View commit details
    Browse the repository at this point in the history

  8. Upgrade DataFusion / Convergence

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    2fbfb7f View commit details
    Browse the repository at this point in the history

  9. Add a sample HTTP config

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    8a7d056 View commit details
    Browse the repository at this point in the history

  10. Break the SeafowlContext code into a trait 

    This is used so that we can mock SeafowlContext easier in unit tests.
    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    5616e42 View commit details
    Browse the repository at this point in the history

  11. Break the HTTP server code up (extract the filter)

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    b5b7ece View commit details
    Browse the repository at this point in the history

  12. Start adding HTTP server tests

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    e6c62a1 View commit details
    Browse the repository at this point in the history

  13. Move make_session to the context test_utils

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    c4dde4c View commit details
    Browse the repository at this point in the history

  14. Add unit tests for the HTTP GET endpoint 

    (this mocking is getting really painful)
    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    b14d13f View commit details
    Browse the repository at this point in the history

  15. Fix test failures related to the DF upgrade 

      - Parquet hashes changed (probably fine, due to changes in ArrowWriter / DF's
    execution output?)
  - Output of `SHOW TABLES` changed (new column)
    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    c2f2d74 View commit details
    Browse the repository at this point in the history

  16. Factor out config-to-context creation code

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    6177769 View commit details
    Browse the repository at this point in the history

  17. Fix serde for in-memory object storage (use memory)

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    87abbc1 View commit details
    Browse the repository at this point in the history

  18. Allow overriding the schema for Postgres catalog

    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    dff760b View commit details
    Browse the repository at this point in the history

  19. Use an actual config / build_context in e2e tests 

    This is to exercise the actual context creation code and remove code duplication
    @mildbyte
    mildbyte committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    c4112ea View commit details
    Browse the repository at this point in the history