HPCC-33694 parquetTypes.ecl failing in containerized environment

[jackdelv]

Inside implementDeleteExternalFile, findPlane winds up being called to find the storage plane and verify it against the host and path arguments passed into the function.

The error occurred in containerized where the host string that was passed in was ".". In bare-metal this is not an issue because it has a host group defined that matches ".", but in containerized there is no hostGroup definition inside storage/planes. The check was added to allow the plane to be matched by the prefix and ignore the "." host only in containerized.

Type of change:

• This change is a bug fix (non-breaking change which fixes an issue).
• This change is a new feature (non-breaking change which adds functionality).
• This change improves the code (refactor or other change that does not change the functionality)
• This change fixes warnings (the fix does not alter the functionality or the generated code)
• This change is a breaking change (fix or feature that will cause existing behavior to change).
• This change alters the query API (existing queries will have to be recompiled)

Checklist:

• My code follows the code style of this project.
  • My code does not create any new warnings from compiler, build system, or lint.
• The commit message is properly formatted and free of typos.
  • The commit message title makes sense in a changelog, by itself.
  • The commit is signed.
• My change requires a change to the documentation.
  • I have updated the documentation accordingly, or...
  • I have created a JIRA ticket to update the documentation.
  • Any new interfaces or exported functions are appropriately commented.
• I have read the CONTRIBUTORS document.
• The change has been fully tested:
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • I have checked that this change does not introduce memory leaks.
  • I have used Valgrind or similar tools to check for potential issues.
• I have given due consideration to all of the following potential concerns:
  • Scalability
  • Performance
  • Security
  • Thread-safety
  • Cloud-compatibility
  • Premature optimization
  • Existing deployed queries will not be broken
  • This change fixes the problem, not just the symptom
  • The target branch of this pull request is appropriate for such a change.
• There are no similar instances of the same problem that should be addressed
  • I have addressed them here
  • I have raised JIRA issues to address them separately
• This is a user interface / front-end modification
  • I have tested my changes in multiple modern browsers
  • The component(s) render as expected

Smoketest:

• Send notifications about my Pull Request position in Smoketest queue.
• Test my draft Pull Request.

Testing:

[github-actions]

Jira Issue: https://hpccsystems.atlassian.net//browse/HPCC-33694

Jirabot Action Result:
Workflow Transition To: Merge Pending
Updated PR

[jakesmith]

@jackdelv - how/when can host be "." in containerized? i.e. where is the "." coming from?

[jakesmith]

@jackdelv was this intentional a draft?

[jackdelv]

@jakesmith Yes, this was intentionally marked as draft as I'm not sure this is the correct solution. The "." actually comes from my test file parquetTypes.ecl. I had assumed (possibly incorrectly) that the "." was a valid location to pass into DeleteExternalFile because there are many examples of "." being passed in for other cases. Would a better solution be to change the calls in parquetTypes.ecl to FileServices.DeleteExternalFile('localhost', dropzoneDirectory + 'BooleanTest.parquet')? That would work, but I'm not sure if the other cases pose a problem.

[AttilaVamos]

I can't tell whether this is a proper fix or not and this change doesn't cause any other problem, based on it is in Dali. @jakesmith can answer this question.
I can't test this change without build an image (what I would like to avoid), deploy it into AKS (or Minikube) and run the failed test on it.

[jakesmith]

I don't think this change is correct, "." shouldn't mean anything for a plane.

"." doesn't really make any sense for cloud. Doesn't rally make sense for bare-metal either.

There are other tests that use "." too, but they should all be changed not to.
They should instead use the dropzone plane name.

[jakesmith]

There's a getDefaultDropZone() (badly named) which returns the default dropzone path.
We need a method to return the default dropzone name, and then use that to pass to DeleteExternalFile.
We almost have it, we have a GetDropZones() which returns all dropzones, but..

@jackdelv - I'll add a GetDefaultDropZoneName() as a separate PR, then this test can use it to pass to DeletedExternalFile.
(NB: all other tests that are using "." for DeleteExternalFile should also be fixed).

[jakesmith]

GetDefaultDropZoneName() JIRA - https://hpccsystems.atlassian.net/browse/HPCC-33720