Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HPCC4J-650 DFSClient should retain integer subtypes #764

Open
wants to merge 1 commit into
base: candidate-9.2.x
Choose a base branch
from

Conversation

jpmcmu
Copy link
Contributor

@jpmcmu jpmcmu commented Sep 24, 2024

  • Modified record translation code to maintain integer subtype
  • Fixed record translation test case
  • Added index record translation to test case

Signed-off-by: James McMullan [email protected]

Type of change:

  • This change is a bug fix (non-breaking change which fixes an issue).
  • This change is a new feature (non-breaking change which adds functionality).
  • This change is a breaking change (fix or feature that will cause existing behavior to change).

Checklist:

  • I have created a corresponding JIRA ticket for this submission
  • My code follows the code style of this project.
    • I have applied the Eclipse code-format template provided.
  • My change requires a change to the documentation.
    • I have updated the documentation accordingly, or...
    • I have created a JIRA ticket to update the documentation.
    • Any new interfaces or exported functions are appropriately commented.
  • I have read the HPCC Systems CONTRIBUTORS document (https://github.com/hpcc-systems/HPCC-Platform/wiki/Guide-for-contributors).
  • The change has been fully tested:
    • This change does not cause any existing JUnits to fail.
    • I have include JUnit coverage to test this change
    • I have performed system test and covered possible regressions and side effects.
  • I have given due consideration to all of the following potential concerns:
    • Scalability
    • Performance
    • Security
    • Thread-safety
    • Premature optimization
    • This change fixes the problem, not just the symptom

Testing:

Copy link

Jira Issue: https://hpccsystems.atlassian.net/browse/HPCC4J-650

Jirabot Action Result:
Workflow Transition To: Merge Pending
Updated PR

@jpmcmu
Copy link
Contributor Author

jpmcmu commented Sep 24, 2024

@rpastrana Note: I am still working on correcting index record definition translation structure.

Copy link
Member

@rpastrana rpastrana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpmcmu looks good. but there's a couple of minor issues. Once resolved please squash

@@ -743,6 +760,10 @@ private static int getJsonTypeDefinition(FieldDef field, HashMap<Integer, Intege
{
char delim = 0x0001;
childJson.put("xpath", childField.getFieldName() + delim + "Row");
} else if (childField.getFieldType() == FieldType.SET)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor format issue, missing new line

@@ -743,6 +760,10 @@ private static int getJsonTypeDefinition(FieldDef field, HashMap<Integer, Intege
{
char delim = 0x0001;
childJson.put("xpath", childField.getFieldName() + delim + "Row");
} else if (childField.getFieldType() == FieldType.SET)
{
char delim = 0x0001;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it feels like this variable deserves to be a static final with an explicit self documenting name

@jpmcmu jpmcmu marked this pull request as ready for review September 27, 2024 01:49
- Modified record translation code to maintain integer subtype
- Fixed record translation test case
- Added index record translation to test case

Signed-off-by: James McMullan [email protected]
@jpmcmu jpmcmu changed the title WIP: HPCC4J-650 DFSClient should retain integer subtypes HPCC4J-650 DFSClient should retain integer subtypes Oct 1, 2024
Copy link
Member

@rpastrana rpastrana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpmcmu just a couple of comments

this.defs = new FieldDef[rhs.defs.length];
for (int i = 0; i < rhs.defs.length; i++)
{
this.defs[i] = new FieldDef(rhs.defs[i]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we creating new FielDefs here, can we just copy rhs.defs[i] directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, I changed this to construct new FieldDefs here because I am going to potentially modify the SrcType later for KEYED_INTS. So I cannot share child FieldDefs any longer, as modifying the projected record definition would modify the original record definitions fields as well.

Copy link
Member

@rpastrana rpastrana Oct 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, I changed this to construct new FieldDefs here because I am going to potentially modify the SrcType later for KEYED_INTS. So I cannot share child FieldDefs any longer, as modifying the projected record definition would modify the original record definitions fields as well.

Fair enough, are there any situations where we could avoid creating the new FieldDef? If so, is it worthwhile attempting to detect the condition in order to avoid the ctr call?

* @return true when biased
* @return true when biased
*
* @deprecated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add a comment indicating which method to use instead of this deprecated method


int childTypeHash = getJsonTypeDefinition(nonKeyedField, typeDefinitionMap, typeDefinitions);
int childTypeIndex = typeDefinitionMap.get(childTypeHash);
String childTypeName = "ty" + (childTypeIndex + 1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's difficult to rationalize the significance of this literal "ty" let's externalize into a self documenting var

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants