Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Label SMB Reads with metadata about SortedBucketIO class #5389

Open
clairemcginty opened this issue Jun 7, 2024 · 0 comments
Open

Label SMB Reads with metadata about SortedBucketIO class #5389

clairemcginty opened this issue Jun 7, 2024 · 0 comments

Comments

@clairemcginty
Copy link
Contributor

It would be helpful if SMB read transforms were labeled with info about their component read types (Avro, Parquet, etc)

Proposal: Make the BucketedInput abstract class extend HasDisplayData and have all typed SortedBucketIO.Read/SortedBucketIO.Transform builders implement it with relevant info (i.e for AvroSortedBucketIO, include schema name/generic|specific, input path, etc). Then SortedBucketSource/SortedBucketTransform can append it to their transform labels here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant