Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DSIP-83][datasource-plugin] Add DolphinDB datasource #16962

Open
wants to merge 7 commits into
base: dev
Choose a base branch
from
Open

[DSIP-83][datasource-plugin] Add DolphinDB datasource #16962

wants to merge 7 commits into from

Conversation

eco8848
Copy link

@eco8848 eco8848 commented Jan 15, 2025 •
edited by SbloodyS
Loading

Purpose of the pull request

  • This pull request adds DolphinDB datasource.

close #16818

Brief change log

  • Add DolphinDB datasource module.
  • Add DolphinDB documentation.
  • Add DolphinDB constant definitions, connection parameters, data source processors and other components.
  • Add DolphinDB JDBC driver dependency.

Verify this pull request

This change added tests and can be verified as follows:

  • Add DataSourceUtilsTest class to test the datasource utility methods.
  • Add DolphinDBDataSourceChannelFactoryTest class to verify the creation methods of the datasource channel factory.
  • Add DolphinDBDataSourceChannelTest class to validate the creation of the datasource client through the channel.
  • Add DolphinDBDataSourceProcessorTest class to test various functionalities of the datasource processor.

@boring-cyborg boring-cyborg
Copy link

boring-cyborg bot commented Jan 15, 2025

Thanks for opening this pull request! Please check out our contributing guidelines. (https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)

@github-actions github-actions bot added UI ui and front end related backend test document labels Jan 15, 2025
@eco8848
Copy link
Author

eco8848 commented Jan 15, 2025

Related DSIP #16818

@SbloodyS SbloodyS changed the title [Feature-15953][datasource-plugin] Add DolphinDB datasource [DSIP-83][datasource-plugin] Add DolphinDB datasource Jan 15, 2025
@SbloodyS SbloodyS added this to the 3.3.0 milestone Jan 15, 2025
@SbloodyS SbloodyS added the DSIP label Jan 15, 2025
github-advanced-security[bot]
github-advanced-security bot found potential problems Jan 15, 2025
View reviewed changes
.../apache/dolphinscheduler/plugin/datasource/dolphindb/param/DolphinDBDataSourceProcessor.java

String[] hostSeperator = connectionParams.getAddress().split(Constants.DOUBLE_SLASH);
String[] hostPortArray = hostSeperator[hostSeperator.length - 1].split(Constants.COMMA);
dataSourceParamDTO.setPort(Integer.parseInt(hostPortArray[0].split(Constants.COLON)[1]));

Check notice

Code scanning / CodeQL

Missing catch of NumberFormatException Note

Potential uncaught 'java.lang.NumberFormatException'.
SbloodyS
SbloodyS reviewed Jan 22, 2025
View reviewed changes
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add dolphinscheduler-datasource-dolphindb to https://github.com/apache/dolphinscheduler/blob/dev/config/plugins_config

@davidzollo davidzollo added the first time contributor First-time contributor label Jan 22, 2025
@SbloodyS
Copy link
Member

SbloodyS commented Feb 4, 2025

Please fix CI errors. @eco8848

@eco8848
Copy link
Author

eco8848 commented Feb 11, 2025

I have referred to the official license requirements at https://dolphinscheduler.apache.org/zh-cn/docs/3.2.2/contribute/join/DS-License. It is necessary to add a description of the newly introduced third-party library in tools/dependencies/known-dependencies.txt, but it seems to conflict with #16838. What should I do now to comply with the requirements? @SbloodyS

@eco8848
Copy link
Author

eco8848 commented Feb 13, 2025

The CI errors have already been fixed. @SbloodyS

@eco8848 eco8848 requested a review from SbloodyS February 13, 2025 02:03
@SbloodyS SbloodyS removed this from the 3.3.0 milestone Feb 23, 2025
@eco8848
Copy link
Author

eco8848 commented Feb 28, 2025

I saw that there is a new check requires a milestone label. So is there a general plan for the integration of this PR? @SbloodyS

@davidzollo davidzollo added 3.3.0 and removed 3.3.0 labels Mar 17, 2025
@davidzollo davidzollo added this to the 3.3.0 milestone Mar 17, 2025
@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed

Issues
30 New issues
0 Accepted issues

Measures
0 Security Hotspots
64.9% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@SbloodyS SbloodyS modified the milestones: 3.3.0-alpha, 3.3.1 Mar 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@EricGao888 EricGao888 Awaiting requested review from EricGao888 EricGao888 is a code owner

@songjianet songjianet Awaiting requested review from songjianet songjianet is a code owner

@caishunfeng caishunfeng Awaiting requested review from caishunfeng caishunfeng is a code owner

@ruanwenjun ruanwenjun Awaiting requested review from ruanwenjun ruanwenjun is a code owner

@SbloodyS SbloodyS Awaiting requested review from SbloodyS SbloodyS is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees

@eco8848 eco8848

Labels
backend document DSIP first time contributor First-time contributor test UI ui and front end related
Projects
None yet
Milestone
3.3.1
Development

Successfully merging this pull request may close these issues.

[DSIP-83][Feature] Add DolphinDB datasource support to DolphinScheduler
3 participants
@eco8848 @SbloodyS @davidzollo