-
Notifications
You must be signed in to change notification settings - Fork 425
feat: simple read write new json type values #7175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds support for structured JSON format in the database by introducing a new format = "structured" option for JSON columns. The key changes include:
- Added structured JSON format support that uses native struct types instead of JSONB binary format
- Modified JSON datatype parsing to distinguish between structured (native) and unstructured (JSONB) formats
- Updated value-to-gRPC conversion functions to properly handle struct field metadata and return
Resulttypes for better error handling - Updated the
greptime-protodependency to a newer revision that includes struct field support in protobuf definitions
Reviewed Changes
Copilot reviewed 25 out of 26 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
tests/cases/standalone/common/types/json/json-structured.sql |
Added test SQL script for structured JSON format feature |
tests/cases/standalone/common/types/json/json-structured.result |
Added expected test results showing structured JSON output format |
src/sql/src/statements/transform/type_alias.rs |
Updated sql_data_type_to_concrete_data_type calls to include column extensions parameter |
src/sql/src/statements.rs |
Modified type conversion to support JSON format options via column extensions |
src/sql/src/parsers/create_parser/json.rs |
Changed return type from OptionMap to Option<OptionMap> to distinguish presence of options |
src/sql/src/parsers/create_parser.rs |
Simplified JSON options assignment and updated type conversion call |
src/sql/src/parser.rs |
Updated test to pass default column extensions |
src/operator/src/req_convert/insert/table_to_region.rs |
Added error handling for vectors_to_rows conversion |
src/operator/src/req_convert/insert/stmt_to_region.rs |
Simplified error handling for value conversion |
src/operator/src/req_convert/insert/fill_impure_default.rs |
Changed from to_proto_value to value_to_grpc_value with error handling |
src/operator/src/req_convert/delete/table_to_region.rs |
Added error handling for vectors_to_rows conversion |
src/operator/src/expr_helper.rs |
Updated type conversion call to pass default extensions |
src/operator/src/error.rs |
Added new ConvertToGrpcValue error variant |
src/mito2/src/request.rs |
Replaced to_proto_value with error-returning value_to_grpc_value |
src/mito2/src/memtable/bulk/part.rs |
Updated to handle conversion errors from value_to_grpc_value |
src/mito2/src/error.rs |
Added new ConvertToGrpcValue error variant |
src/flow/src/repr.rs |
Changed From<Row> to TryFrom<Row> for proper error handling |
src/flow/src/error.rs |
Added new ConvertToGrpcRow error variant |
src/flow/src/adapter.rs |
Updated to use try_into() instead of into() for row conversion |
src/datatypes/src/types/json_type.rs |
Changed internal JSON plain field name to avoid conflicts |
src/datatypes/src/types.rs |
Made json_type module public for external access |
src/common/sql/src/convert.rs |
Added JSON type check to skip unnecessary casting |
src/common/recordbatch/src/recordbatch.rs |
Added logic to amend schema with struct array types dynamically |
src/api/src/helper.rs |
Major refactoring: removed to_proto_value, made value_to_grpc_value return Result, added struct field metadata |
Cargo.toml |
Updated greptime-proto dependency to newer revision |
Cargo.lock |
Updated lockfile with new proto dependency revision |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
2273fb0 to
e2abb51
Compare
tests/cases/standalone/common/types/json/json-structured.result
Outdated
Show resolved
Hide resolved
e2abb51 to
53a2f59
Compare
53a2f59 to
5e4bf28
Compare
|
successor of #7248 |
26c47c4 to
eb62b69
Compare
Signed-off-by: luofucong <[email protected]>
eb62b69 to
70b4f73
Compare
|
@sunng87 @fengys1996 PTAL |
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
please see the sqlness test
json-structured.resultfirstfor compatibility consideration, the table created with datatype
JSONis treated as the old json typePR Checklist
Please convert it to a draft if some of the following conditions are not met.