Skip to content

Conversation

@vinodkc
Copy link
Contributor

@vinodkc vinodkc commented Nov 20, 2025

What changes were proposed in this pull request?

This PR adds six numeric conversion functions for the TIME type, mirroring the existing pattern for TIMESTAMP:

Constructor Functions (Numeric → TIME):

  • time_from_seconds(seconds) - Supports fractional seconds via NumericType
  • time_from_millis(millis) - IntegralType input
  • time_from_micros(micros) - IntegralType input

Extractor Functions (TIME → Numeric):

  • time_to_seconds(time) - Returns DECIMAL(14,6) to preserve fractional seconds
  • time_to_millis(time) - Returns BIGINT
  • time_to_micros(time) - Returns BIGINT

eg

-- Constructor functions (Numeric → TIME)
SELECT time_from_seconds(52200);           -- 14:30:00
SELECT time_from_seconds(52200.123456);    -- 14:30:00.123456
SELECT time_from_millis(52200123);         -- 14:30:00.123
SELECT time_from_micros(52200123456);      -- 14:30:00.123456

-- Extractor functions (TIME → Numeric)
SELECT time_to_seconds(TIME'14:30:00.123456');  -- 52200.123456
SELECT time_to_millis(TIME'14:30:00.123');      -- 52200123
SELECT time_to_micros(TIME'14:30:00.123456');   -- 52200123456

Why are the changes needed?

The TIME type lacks numeric conversion functions, making it difficult to:

  • Create TIME values from numeric representations (common in data ingestion)
  • Extract numeric values for calculations or external system integration

TIMESTAMP has equivalent functions (timestamp_seconds(), unix_seconds(), etc.), and TIME should achieve feature parity.

Does this PR introduce any user-facing change?

Yes, adds six new SQL functions:

How was this patch tested?

Unit Tests (TimeExpressionsSuite.scala):
SQL Integration Tests (time.sql)

Was this patch authored or co-authored using generative AI tooling?

Yes.
Generated-by: Claude 3.5 Sonnet

AI assistance was used for:

  • Code pattern analysis and design discussions
  • Implementation guidance following Spark conventions
  • Test case generation and organization
  • Documentation and examples

Additional context

Q: Why does time_to_seconds() return DECIMAL(14,6) instead of BIGINT?
A: To preserve fractional seconds and enable exact round-trip conversions:

SELECT time_to_seconds(time_from_seconds(52200.123456));
-- Returns: 52200.123456 (exact match) ✓

If we returned BIGINT, fractional seconds would be lost.

Q: Why does time_from_seconds() accept NumericType instead of just IntegralType?
A: To support fractional seconds for maximum flexibility:

SELECT time_from_seconds(52200.5);      -- DECIMAL: 14:30:00.5
SELECT time_from_seconds(52200.5::float);  -- FLOAT: 14:30:00.5

This mirrors timestamp_seconds() which also accepts NumericType.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you for proposing this for Apache Spark 4.2.0, @vinodkc .

@dongjoon-hyun
Copy link
Member

Merged to master for Apache Spark 4.2.0.

zifeif2 pushed a commit to zifeif2/spark that referenced this pull request Nov 25, 2025
### What changes were proposed in this pull request?

This PR adds six numeric conversion functions for the TIME type, mirroring the existing pattern for TIMESTAMP:

Constructor Functions (Numeric → TIME):

- time_from_seconds(seconds) - Supports fractional seconds via NumericType
- time_from_millis(millis) - IntegralType input
- time_from_micros(micros) - IntegralType input

Extractor Functions (TIME → Numeric):
- time_to_seconds(time) - Returns DECIMAL(14,6) to preserve fractional seconds
- time_to_millis(time) - Returns BIGINT
- time_to_micros(time) - Returns BIGINT

eg
```
-- Constructor functions (Numeric → TIME)
SELECT time_from_seconds(52200);           -- 14:30:00
SELECT time_from_seconds(52200.123456);    -- 14:30:00.123456
SELECT time_from_millis(52200123);         -- 14:30:00.123
SELECT time_from_micros(52200123456);      -- 14:30:00.123456

-- Extractor functions (TIME → Numeric)
SELECT time_to_seconds(TIME'14:30:00.123456');  -- 52200.123456
SELECT time_to_millis(TIME'14:30:00.123');      -- 52200123
SELECT time_to_micros(TIME'14:30:00.123456');   -- 52200123456
```
### Why are the changes needed?

The TIME type lacks numeric conversion functions, making it difficult to:

- Create TIME values from numeric representations (common in data ingestion)
- Extract numeric values for calculations or external system integration

TIMESTAMP has equivalent functions (timestamp_seconds(), unix_seconds(), etc.), and TIME should achieve feature parity.

### Does this PR introduce _any_ user-facing change?

Yes,  adds six new SQL functions:

### How was this patch tested?

Unit Tests (TimeExpressionsSuite.scala):
SQL Integration Tests (time.sql)

### Was this patch authored or co-authored using generative AI tooling?

Yes.
Generated-by:  Claude 3.5 Sonnet

 AI assistance was used for:
- Code pattern analysis and design discussions
- Implementation guidance following Spark conventions
- Test case generation and organization
- Documentation and examples

### Additional context
Q: Why does time_to_seconds() return DECIMAL(14,6) instead of BIGINT?
A: To preserve fractional seconds and enable exact round-trip conversions:
```
SELECT time_to_seconds(time_from_seconds(52200.123456));
-- Returns: 52200.123456 (exact match) ✓
```
If we returned BIGINT, fractional seconds would be lost.

Q: Why does time_from_seconds() accept NumericType instead of just IntegralType?
A: To support fractional seconds for maximum flexibility:
```
SELECT time_from_seconds(52200.5);      -- DECIMAL: 14:30:00.5
SELECT time_from_seconds(52200.5::float);  -- FLOAT: 14:30:00.5
```
This mirrors timestamp_seconds() which also accepts NumericType.

Closes apache#53147 from vinodkc/br_time_numeric_conversion.

Authored-by: vinodkc <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
huangxiaopingRD pushed a commit to huangxiaopingRD/spark that referenced this pull request Nov 25, 2025
### What changes were proposed in this pull request?

This PR adds six numeric conversion functions for the TIME type, mirroring the existing pattern for TIMESTAMP:

Constructor Functions (Numeric → TIME):

- time_from_seconds(seconds) - Supports fractional seconds via NumericType
- time_from_millis(millis) - IntegralType input
- time_from_micros(micros) - IntegralType input

Extractor Functions (TIME → Numeric):
- time_to_seconds(time) - Returns DECIMAL(14,6) to preserve fractional seconds
- time_to_millis(time) - Returns BIGINT
- time_to_micros(time) - Returns BIGINT

eg
```
-- Constructor functions (Numeric → TIME)
SELECT time_from_seconds(52200);           -- 14:30:00
SELECT time_from_seconds(52200.123456);    -- 14:30:00.123456
SELECT time_from_millis(52200123);         -- 14:30:00.123
SELECT time_from_micros(52200123456);      -- 14:30:00.123456

-- Extractor functions (TIME → Numeric)
SELECT time_to_seconds(TIME'14:30:00.123456');  -- 52200.123456
SELECT time_to_millis(TIME'14:30:00.123');      -- 52200123
SELECT time_to_micros(TIME'14:30:00.123456');   -- 52200123456
```
### Why are the changes needed?

The TIME type lacks numeric conversion functions, making it difficult to:

- Create TIME values from numeric representations (common in data ingestion)
- Extract numeric values for calculations or external system integration

TIMESTAMP has equivalent functions (timestamp_seconds(), unix_seconds(), etc.), and TIME should achieve feature parity.

### Does this PR introduce _any_ user-facing change?

Yes,  adds six new SQL functions:

### How was this patch tested?

Unit Tests (TimeExpressionsSuite.scala):
SQL Integration Tests (time.sql)

### Was this patch authored or co-authored using generative AI tooling?

Yes.
Generated-by:  Claude 3.5 Sonnet

 AI assistance was used for:
- Code pattern analysis and design discussions
- Implementation guidance following Spark conventions
- Test case generation and organization
- Documentation and examples

### Additional context
Q: Why does time_to_seconds() return DECIMAL(14,6) instead of BIGINT?
A: To preserve fractional seconds and enable exact round-trip conversions:
```
SELECT time_to_seconds(time_from_seconds(52200.123456));
-- Returns: 52200.123456 (exact match) ✓
```
If we returned BIGINT, fractional seconds would be lost.

Q: Why does time_from_seconds() accept NumericType instead of just IntegralType?
A: To support fractional seconds for maximum flexibility:
```
SELECT time_from_seconds(52200.5);      -- DECIMAL: 14:30:00.5
SELECT time_from_seconds(52200.5::float);  -- FLOAT: 14:30:00.5
```
This mirrors timestamp_seconds() which also accepts NumericType.

Closes apache#53147 from vinodkc/br_time_numeric_conversion.

Authored-by: vinodkc <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants