-
Notifications
You must be signed in to change notification settings - Fork 412
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
*: only enable new string serdes format when MppVersion >= MppVersionV3 #9759
base: master
Are you sure you want to change the base?
Conversation
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
PR of tidb is pingcap/tidb#58652. Merge this PR before merging the PR of tidb. |
/retest |
std::unique_ptr<ChunkCodecStream> ArrowChunkCodec::newCodecStream(const std::vector<tipb::FieldType> & field_types) | ||
std::unique_ptr<ChunkCodecStream> ArrowChunkCodec::newCodecStream( | ||
const std::vector<tipb::FieldType> & field_types, | ||
MppVersion) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add mpp_version
for CHBlockChunkCodecStream
, useless for other codec.
writeStringBinary(CodecUtils::convertDataTypeNameByMppVersion(column.type->getName(), mpp_version), *output); | ||
|
||
if (rows) | ||
WriteColumnData(*column.type, column.column, *output, 0, 0); | ||
CHBlockChunkCodec::WriteColumnData(*column.type, column.column, *output, 0, 0, mpp_version); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about getting a ser_type
and use it for writing down the data type name and also writeColumnData?
auto ser_type = CodecUtils::convertDataTypeNameByMppVersion(column.type->getName(), mpp_version);
writeStringBinary(ser_type, *output);
if (rows)
CHBlockChunkCodec::WriteColumnData(*ser_type, column.column, *output, 0, 0);
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Has refined.
Keep convertDataTypeByMppVersion
and remove convertDataTypeNameByMppVersion
What problem does this PR solve?
Issue Number: close #9673
Problem Summary:
What is changed and how it works?
tidb will send an
MppVersion
field to tiflash.When rolling upgrade, tiflash will be upgraded before tidb.
During the process of upgrading tiflash, tidb has not been upgraded yet, so all mpp versions received are
MppVersionsV2
, tiflash still using the old format to exchange data.When upgrading tidb, the new tidb uses
MppVersionsV3
to send requests, and tiflash starts using the new format to exchange data (at this point, all tiflash has been upgraded, so there will be no compatibility issues).This PR pass
mpp_version
toCHBlockChunkCodec
andCHBlockChunkCodecV1
for encoding: if mpp_version <= MppVersion2, use the legacy format of string.When decoding,
CHBlockChunkCodec
andCHBlockChunkCodecV1
respect to the type name from encoder.PR of tidb is copr: add MppVersionV3 tidb#58652.
Check List
Tests
Side effects
Documentation
Release note