[FLINK-25568] Support Elasticsearch Source Connector #62
Conversation
|
Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html) |
|
Support for FilterPushDown, ProjectPushDown, and LimitPushDown to optimize query performance. Have these functions been implemented? I don't seem to have seen this part of the code |
Hello, I am very happy to read your comments. Currently this PR belongs to draft status. For the three push-downs you mentioned. As shown in FLIP-127, I am going to implement related implementations of FilterPushDown, ProjectPushDown, and LimitPushDown and optimize es query expressions. Currently, only the ProjectPushDown interface is implemented, please refer to Elasticsearch7DynamicSource#applyProjection. Due to busy work, I will finish the development after a while. |
|
@kaori-seasons Are you still planning to work on this PR? |
...n/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticSearchInputFormatBase.java
Show resolved
Hide resolved
...a/org/apache/flink/streaming/connectors/elasticsearch/table/Elasticsearch7DynamicSource.java
Show resolved
Hide resolved
|
# Conflicts: # flink-connector-elasticsearch-base/pom.xml # flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchApiCallBridge.java # flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/table/ElasticsearchRowDataLookupFunction.java # flink-connector-elasticsearch-base/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBaseTest.java # flink-connector-elasticsearch6/src/main/java/org/apache/flink/streaming/connectors/elasticsearch6/Elasticsearch6ApiCallBridge.java # flink-connector-elasticsearch7/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/table/Elasticsearch7DynamicSource.java # flink-connector-elasticsearch7/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/table/Elasticsearch7DynamicTableFactory.java # flink-connector-elasticsearch7/src/main/java/org/apache/flink/streaming/connectors/elasticsearch7/Elasticsearch7ApiCallBridge.java # flink-connector-elasticsearch7/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/table/Elasticsearch7DynamicSinkITCase.java # flink-connector-elasticsearch7/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/table/Elasticsearch7DynamicTableFactoryTest.java # pom.xml
According to FLIP-127 to achieve source read function