Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PoC] Set routing shard in SQL via the relation partition #1808

Conversation

acarbonetto
Copy link
Collaborator

@acarbonetto acarbonetto commented Jul 5, 2023

Description

Allows user to set routing shard via the relation partition keys. For example:

select _id, _index, _routing, int0 from calcs_routing PARTITION(KEY1, KEY2) where _routing = 'KEY1' or _routing = 'KEY2'

Note that multiple keys can be send to OpenSearch storage by using comma-delimited list.

TODO

  • Remove quotes/backticks from around the routingId before sending
  • Rebase
  • Propose PPL syntax

Issues Resolved

Note (out of scope): PPL syntax is still TBD

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@codecov
Copy link

codecov bot commented Jul 5, 2023

Codecov Report

Merging #1808 (c3e594f) into main (1a7134b) will decrease coverage by 3.10%.
The diff coverage is 33.33%.

❗ Current head c3e594f differs from pull request most recent head e2b2b04. Consider uploading reports for the commit e2b2b04 to get more accurate results

@@             Coverage Diff              @@
##               main    #1808      +/-   ##
============================================
- Coverage     97.39%   94.30%   -3.10%     
+ Complexity     4603      946    -3657     
============================================
  Files           401      100     -301     
  Lines         11397     2772    -8625     
  Branches        835      241     -594     
============================================
- Hits          11100     2614    -8486     
+ Misses          290      154     -136     
+ Partials          7        4       -3     
Flag Coverage Δ
sql-engine 94.30% <33.33%> (-3.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
...earch/sql/datasources/exceptions/ErrorMessage.java 0.00% <ø> (ø)
...model/transport/CreateDataSourceActionRequest.java 27.27% <ø> (ø)
...odel/transport/CreateDataSourceActionResponse.java 0.00% <ø> (ø)
...model/transport/DeleteDataSourceActionRequest.java 18.75% <ø> (ø)
...odel/transport/DeleteDataSourceActionResponse.java 0.00% <ø> (ø)
...es/model/transport/GetDataSourceActionRequest.java 27.27% <ø> (ø)
...s/model/transport/GetDataSourceActionResponse.java 0.00% <ø> (ø)
...model/transport/UpdateDataSourceActionRequest.java 27.27% <ø> (ø)
...odel/transport/UpdateDataSourceActionResponse.java 0.00% <ø> (ø)
...ql/datasources/rest/RestDataSourceQueryAction.java 0.00% <ø> (ø)
... and 5 more

... and 306 files with indirect coverage changes

@Yury-Fridlyand Yury-Fridlyand marked this pull request as draft July 5, 2023 18:19
* Fix core refactor: StreamIO from common to core.common

Signed-off-by: acarbonetto <[email protected]>

* Fix core refactor: StreamIO from common to core.common

Signed-off-by: acarbonetto <[email protected]>

---------

Signed-off-by: acarbonetto <[email protected]>
Signed-off-by: acarbonetto <[email protected]>
@acarbonetto
Copy link
Collaborator Author

Closing Proof of Concept

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow individual shards to be targeted during query execution [FEATURE]
1 participant