You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently when instantiating S3SeekableInputStream and LogicalIoFactory we create ParquetColumnPrefetchStore, even though this may not be required if we are not further reading parquet files.
We need to look at how to improve this.
The text was updated successfully, but these errors were encountered:
Thank you for opening this! Wasn't there a PR floating around that separated LogicalIO into format specific and generic one?
I think this could be a good place to start to tease this apart - can we do this first? The generic one most definitely won't need to take anything Parquet specific.
Currently when instantiating
S3SeekableInputStream
andLogicalIoFactory
we createParquetColumnPrefetchStore
, even though this may not be required if we are not further reading parquet files.We need to look at how to improve this.
The text was updated successfully, but these errors were encountered: