Skip to content

Conversation

@MichaelScofield
Copy link
Collaborator

@MichaelScofield MichaelScofield commented Nov 4, 2025

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

What's changed and what's your intention?

please see the sqlness test json-structured.result first

for compatibility consideration, the table created with datatype JSON is treated as the old json type

PR Checklist

Please convert it to a draft if some of the following conditions are not met.

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR requires documentation updates.
  • API changes are backward compatible.
  • Schema or data changes are backward compatible.

@github-actions github-actions bot added size/M docs-not-required This change does not impact docs. labels Nov 4, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for structured JSON format in the database by introducing a new format = "structured" option for JSON columns. The key changes include:

  • Added structured JSON format support that uses native struct types instead of JSONB binary format
  • Modified JSON datatype parsing to distinguish between structured (native) and unstructured (JSONB) formats
  • Updated value-to-gRPC conversion functions to properly handle struct field metadata and return Result types for better error handling
  • Updated the greptime-proto dependency to a newer revision that includes struct field support in protobuf definitions

Reviewed Changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/cases/standalone/common/types/json/json-structured.sql Added test SQL script for structured JSON format feature
tests/cases/standalone/common/types/json/json-structured.result Added expected test results showing structured JSON output format
src/sql/src/statements/transform/type_alias.rs Updated sql_data_type_to_concrete_data_type calls to include column extensions parameter
src/sql/src/statements.rs Modified type conversion to support JSON format options via column extensions
src/sql/src/parsers/create_parser/json.rs Changed return type from OptionMap to Option<OptionMap> to distinguish presence of options
src/sql/src/parsers/create_parser.rs Simplified JSON options assignment and updated type conversion call
src/sql/src/parser.rs Updated test to pass default column extensions
src/operator/src/req_convert/insert/table_to_region.rs Added error handling for vectors_to_rows conversion
src/operator/src/req_convert/insert/stmt_to_region.rs Simplified error handling for value conversion
src/operator/src/req_convert/insert/fill_impure_default.rs Changed from to_proto_value to value_to_grpc_value with error handling
src/operator/src/req_convert/delete/table_to_region.rs Added error handling for vectors_to_rows conversion
src/operator/src/expr_helper.rs Updated type conversion call to pass default extensions
src/operator/src/error.rs Added new ConvertToGrpcValue error variant
src/mito2/src/request.rs Replaced to_proto_value with error-returning value_to_grpc_value
src/mito2/src/memtable/bulk/part.rs Updated to handle conversion errors from value_to_grpc_value
src/mito2/src/error.rs Added new ConvertToGrpcValue error variant
src/flow/src/repr.rs Changed From<Row> to TryFrom<Row> for proper error handling
src/flow/src/error.rs Added new ConvertToGrpcRow error variant
src/flow/src/adapter.rs Updated to use try_into() instead of into() for row conversion
src/datatypes/src/types/json_type.rs Changed internal JSON plain field name to avoid conflicts
src/datatypes/src/types.rs Made json_type module public for external access
src/common/sql/src/convert.rs Added JSON type check to skip unnecessary casting
src/common/recordbatch/src/recordbatch.rs Added logic to amend schema with struct array types dynamically
src/api/src/helper.rs Major refactoring: removed to_proto_value, made value_to_grpc_value return Result, added struct field metadata
Cargo.toml Updated greptime-proto dependency to newer revision
Cargo.lock Updated lockfile with new proto dependency revision

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@MichaelScofield MichaelScofield force-pushed the feat/json-type-simple-read-write branch 3 times, most recently from 2273fb0 to e2abb51 Compare November 6, 2025 12:56
@MichaelScofield MichaelScofield force-pushed the feat/json-type-simple-read-write branch from e2abb51 to 53a2f59 Compare November 21, 2025 07:53
@github-actions github-actions bot added size/L and removed size/M labels Nov 21, 2025
@MichaelScofield MichaelScofield force-pushed the feat/json-type-simple-read-write branch from 53a2f59 to 5e4bf28 Compare November 21, 2025 08:36
@MichaelScofield
Copy link
Collaborator Author

successor of #7248

@MichaelScofield MichaelScofield force-pushed the feat/json-type-simple-read-write branch 4 times, most recently from 26c47c4 to eb62b69 Compare November 24, 2025 03:32
@github-actions github-actions bot added size/M and removed size/L labels Nov 24, 2025
@MichaelScofield MichaelScofield force-pushed the feat/json-type-simple-read-write branch from eb62b69 to 70b4f73 Compare November 24, 2025 07:16
@MichaelScofield MichaelScofield marked this pull request as ready for review November 24, 2025 07:56
@MichaelScofield MichaelScofield requested review from a team, evenyag and v0y4g3r as code owners November 24, 2025 07:56
@sunng87 sunng87 requested a review from fengys1996 November 24, 2025 10:06
@MichaelScofield
Copy link
Collaborator Author

@sunng87 @fengys1996 PTAL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-not-required This change does not impact docs. size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants