[CALCITE-6800] Enable a few existing functions to HIVE library #4165

xuzifu666 · 2025-01-24T03:11:27Z

Enable some methods to hive which had already been implemented for other libraries.
issue: https://issues.apache.org/jira/browse/CALCITE-6800
supported functions as follow:

DATE_ADD
DATE_SUB
DECODE
LPAD
RPAD
LTRIM
RTRIM
SPACE
GREATEST
LEAST
REGEXP_REPLACE_3
REPEAT
SOUNDEX
TO_DATE
MD5
SHA1
LOG_MYSQL

caicancai

It seems that the bounds test of the function is not considered for the time being.

LGTM

core/src/main/java/org/apache/calcite/sql/fun/SqlLibraryOperators.java

site/_docs/reference.md

ILuffZhe · 2025-01-24T06:33:28Z

@xuzifu666 The CI failed ,please make sure your change can pass on your machine, then push it so we can go forward.

xuzifu666 · 2025-01-24T11:36:32Z

@xuzifu666 The CI failed ,please make sure your change can pass on your machine, then push it so we can go forward.

@ILuffZhe CI is success, PTAL when you are free time, thanks~

ILuffZhe

It's really close now, could you please take a look? @xuzifu666

site/_docs/reference.md

core/src/main/java/org/apache/calcite/sql/fun/SqlLibraryOperators.java

xuzifu666 · 2025-01-25T12:09:04Z

@ILuffZhe addressed，thanks for your review，PTAL

ILuffZhe · 2025-01-25T13:04:09Z

site/_docs/reference.md

@@ -2852,7 +2852,7 @@ In the following:
 | b | DATE(string)                                   | Equivalent to `CAST(string AS DATE)`
 | b | DATE(year, month, day)                         | Returns a DATE value for *year*, *month*, and *day* (all of type INTEGER)
 | q r f | DATEADD(timeUnit, integer, datetime)       | Equivalent to `TIMESTAMPADD(timeUnit, integer, datetime)`
-| q r f | DATEDIFF(timeUnit, datetime, datetime2)    | Equivalent to `TIMESTAMPDIFF(timeUnit, datetime, datetime2)`
+| q r f h | DATEDIFF(timeUnit, datetime, datetime2)    | Equivalent to `TIMESTAMPDIFF(timeUnit, datetime, datetime2)`


Is this a unintentional mistake?

Had removed it.

ILuffZhe · 2025-01-25T13:24:43Z

Hi @xuzifu666 , you don't need to force-push every time, which makes it not easier to review.
In this case, make sure all the functions enabled in Hive library is document in reference.md, and SqlOperatorTest should also cover those functions (the last commit still missed some tests).
Please take a look and cc me when you are ready.

By the way, I think it might be better to add a new JIRA ticket, CALCITE-6241 is for Spark library.

xuzifu666 · 2025-01-25T15:12:20Z

Hi @xuzifu666 , you don't need to force-push every time, which makes it not easier to review. In this case, make sure all the functions enabled in Hive library is document in reference.md, and SqlOperatorTest should also cover those functions (the last commit still missed some tests). Please take a look and cc me when you are ready.

By the way, I think it might be better to add a new JIRA ticket, CALCITE-6241 is for Spark library.

Very thanks for your remind, I had add all changes in doc and add all cases in Test, I also make a new jira https://issues.apache.org/jira/browse/CALCITE-6800 about the pr, due to had a keyword 'hive' in title which not pass for ut, so I had rebase for the error, PTAL when you free time. @ILuffZhe

ILuffZhe · 2025-01-26T09:13:01Z

It seems your changes in PR is not consistent with what described in JIRA.

xuzifu666 · 2025-01-26T09:30:10Z

It seems your changes in PR is not consistent with what described in JIRA.

This pr is aim to enable some existing functions to Hive library, I had changed update jira for all functions. Would you please give a review again please. Thanks~ @ILuffZhe

ILuffZhe · 2025-01-26T09:47:40Z

It seems your changes in PR is not consistent with what described in JIRA.

This pr is aim to enable some existing functions to Hive library, I had changed purpose about it. Would you give some advice if it is not fitable. Thanks~ @ILuffZhe

This case should only enable existing functions which have the same behavior in Hive.
If not, maybe we should file a new JIRA ticket and implement it if you want.

xuzifu666 · 2025-01-26T10:07:50Z

It seems your changes in PR is not consistent with what described in JIRA.

This pr is aim to enable some existing functions to Hive library, I had changed purpose about it. Would you give some advice if it is not fitable. Thanks~ @ILuffZhe

This case should only enable existing functions which have the same behavior in Hive.
If not, maybe we should file a new JIRA ticket and implement it if you want.

This case is only enable existing functions which have the same behavior in Hive，not implement for new functions，so keep it can be fine without file a new jira ticket or can merge this pr firstly? @ILuffZhe

xuzifu666 · 2025-01-29T11:17:32Z

@rubenada could you give a final review，thanks~

core/src/main/java/org/apache/calcite/sql/fun/SqlLibraryOperators.java

rubenada · 2025-01-29T18:27:59Z

Overall looks good, one minor comment:

SPACE is missing from the list of functions on the PR / Jira description, but it was added in the code
INSTR is part of the list; however this function was already defined for Hive

ILuffZhe · 2025-01-30T01:55:29Z

It seems your changes in PR is not consistent with what described in JIRA.

This pr is aim to enable some existing functions to Hive library, I had changed purpose about it. Would you give some advice if it is not fitable. Thanks~ @ILuffZhe

This case should only enable existing functions which have the same behavior in Hive.
If not, maybe we should file a new JIRA ticket and implement it if you want.

This case is only enable existing functions which have the same behavior in Hive，not implement for new functions，so keep it can be fine without file a new jira ticket or can merge this pr firstly? @ILuffZhe

Sorry for the delay, been a little busy these days.

ILuffZhe

LGTM. Just s few comments @rubenada left should be addressed.

xuzifu666 · 2025-01-30T03:25:00Z

Overall looks good, one minor comment:

SPACE is missing from the list of functions on the PR / Jira description, but it was added in the code

INSTR is part of the list; however this function was already defined for Hive

@rubenada Thanks for your reminder, 2 is fixed now, and 1 I had added a space in the funtion list of PR / Jira description.

sonarqubecloud · 2025-01-30T03:46:46Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
20.9% Duplication on New Code

See analysis details on SonarQube Cloud

rubenada · 2025-01-30T07:39:21Z

@xuzifu666 sorry, but IIANM the list of functions in the description of the Jira/PR is still inaccurate: it mentions INSTR function (and it shouldn't, since this function was already defined for Hive); and it does not mention the SPACE function (and it should, since the changes consider it). Basically INSTR should be replaced by SPACE in the list, or am I missing something?

xuzifu666 · 2025-01-30T12:08:42Z

@xuzifu666 sorry, but IIANM the list of functions in the description of the Jira/PR is still inaccurate: it mentions INSTR function (and it shouldn't, since this function was already defined for Hive); and it does not mention the SPACE function (and it should, since the changes consider it). Basically INSTR should be replaced by SPACE in the list, or am I missing something?

Oh, it's my mistake, very sorry and I had removed INSTR and added SPACE to description of JIRA/PR and addressed all comments, very thanks for your review.@rubenada

caicancai · 2025-01-31T01:46:16Z

@rubenada @ILuffZhe thanks for your review

caicancai approved these changes Jan 24, 2025

View reviewed changes

xuzifu666 force-pushed the calcite-6241 branch from 0054198 to 1f24557 Compare January 24, 2025 05:33

ILuffZhe reviewed Jan 24, 2025

View reviewed changes

core/src/main/java/org/apache/calcite/sql/fun/SqlLibraryOperators.java Outdated Show resolved Hide resolved

core/src/main/java/org/apache/calcite/sql/fun/SqlLibraryOperators.java Outdated Show resolved Hide resolved

site/_docs/reference.md Outdated Show resolved Hide resolved

xuzifu666 force-pushed the calcite-6241 branch 2 times, most recently from 79cfb54 to 440101a Compare January 24, 2025 06:14

xuzifu666 force-pushed the calcite-6241 branch 3 times, most recently from f54e426 to dc885b1 Compare January 24, 2025 11:11

xuzifu666 closed this Jan 24, 2025

xuzifu666 reopened this Jan 24, 2025

ILuffZhe reviewed Jan 25, 2025

View reviewed changes

site/_docs/reference.md Show resolved Hide resolved

core/src/main/java/org/apache/calcite/sql/fun/SqlLibraryOperators.java Outdated Show resolved Hide resolved

xuzifu666 force-pushed the calcite-6241 branch from dc885b1 to b78fba4 Compare January 25, 2025 11:43

ILuffZhe reviewed Jan 25, 2025

View reviewed changes

ILuffZhe self-assigned this Jan 25, 2025

xuzifu666 changed the title ~~[CALCITE-6241] Enable a few existing functions to hive library~~ [CALCITE-6800] Enable a few existing functions to hive library Jan 25, 2025

xuzifu666 force-pushed the calcite-6241 branch from 5c1de46 to b5a02f8 Compare January 25, 2025 15:27

xuzifu666 changed the title ~~[CALCITE-6800] Enable a few existing functions to hive library~~ [CALCITE-6800] Enable a few existing functions to HIVE library Jan 25, 2025

xuzifu666 requested a review from ILuffZhe January 27, 2025 10:06

caicancai added the LGTM-will-merge-soon Overall PR looks OK. Only minor things left. label Jan 29, 2025

rubenada reviewed Jan 29, 2025

View reviewed changes

core/src/main/java/org/apache/calcite/sql/fun/SqlLibraryOperators.java Outdated Show resolved Hide resolved

ILuffZhe approved these changes Jan 30, 2025

View reviewed changes

[CALCITE-6800] Enable a few existing functions to Hive library

4de9dd9

xuzifu666 force-pushed the calcite-6241 branch from b5a02f8 to 4de9dd9 Compare January 30, 2025 03:23

rubenada approved these changes Jan 30, 2025

View reviewed changes

caicancai merged commit f10d0ce into apache:main Jan 31, 2025
20 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CALCITE-6800] Enable a few existing functions to HIVE library #4165

[CALCITE-6800] Enable a few existing functions to HIVE library #4165

xuzifu666 commented Jan 24, 2025 •

edited

Loading

caicancai left a comment

ILuffZhe commented Jan 24, 2025

xuzifu666 commented Jan 24, 2025

ILuffZhe left a comment

xuzifu666 commented Jan 25, 2025

ILuffZhe Jan 25, 2025

xuzifu666 Jan 25, 2025

ILuffZhe commented Jan 25, 2025 •

edited

Loading

xuzifu666 commented Jan 25, 2025 •

edited

Loading

ILuffZhe commented Jan 26, 2025

xuzifu666 commented Jan 26, 2025 •

edited

Loading

ILuffZhe commented Jan 26, 2025

xuzifu666 commented Jan 26, 2025 •

edited

Loading

xuzifu666 commented Jan 29, 2025

rubenada commented Jan 29, 2025

ILuffZhe commented Jan 30, 2025

ILuffZhe left a comment

xuzifu666 commented Jan 30, 2025

sonarqubecloud bot commented Jan 30, 2025

rubenada commented Jan 30, 2025

xuzifu666 commented Jan 30, 2025 •

edited

Loading

caicancai commented Jan 31, 2025

[CALCITE-6800] Enable a few existing functions to HIVE library #4165

[CALCITE-6800] Enable a few existing functions to HIVE library #4165

Conversation

xuzifu666 commented Jan 24, 2025 • edited Loading

caicancai left a comment

Choose a reason for hiding this comment

ILuffZhe commented Jan 24, 2025

xuzifu666 commented Jan 24, 2025

ILuffZhe left a comment

Choose a reason for hiding this comment

xuzifu666 commented Jan 25, 2025

ILuffZhe Jan 25, 2025

Choose a reason for hiding this comment

xuzifu666 Jan 25, 2025

Choose a reason for hiding this comment

ILuffZhe commented Jan 25, 2025 • edited Loading

xuzifu666 commented Jan 25, 2025 • edited Loading

ILuffZhe commented Jan 26, 2025

xuzifu666 commented Jan 26, 2025 • edited Loading

ILuffZhe commented Jan 26, 2025

xuzifu666 commented Jan 26, 2025 • edited Loading

xuzifu666 commented Jan 29, 2025

rubenada commented Jan 29, 2025

ILuffZhe commented Jan 30, 2025

ILuffZhe left a comment

Choose a reason for hiding this comment

xuzifu666 commented Jan 30, 2025

sonarqubecloud bot commented Jan 30, 2025

Quality Gate passed

rubenada commented Jan 30, 2025

xuzifu666 commented Jan 30, 2025 • edited Loading

caicancai commented Jan 31, 2025

xuzifu666 commented Jan 24, 2025 •

edited

Loading

ILuffZhe commented Jan 25, 2025 •

edited

Loading

xuzifu666 commented Jan 25, 2025 •

edited

Loading

xuzifu666 commented Jan 26, 2025 •

edited

Loading

xuzifu666 commented Jan 26, 2025 •

edited

Loading

xuzifu666 commented Jan 30, 2025 •

edited

Loading