Improve GET request annotation

[ahmarsuhail]

Tell us more about this new feature.

We currently annotate S3 GET requests as "Async" or "Sync" and pass this into the referrer header. This is useful when looking at access logs and performing regression analysis.

This annotation can be improved/fixed to make this analysis easier:

1/ All sequential prefetch requests are currently marked as SYNC, even though these are ASYNC.
2/ All parquet column prefetch requests are marked as ASYNC, we can make this more granular by adding more read modes:
COLUMN_PREFETCH,
FOOTER_PREFETCH,
REMAINDER_COLUMN_PREFETCH

This will allow us to quickly identify where an S3 request came from.

Code of Conduct

• I agree to follow this project's Code of Conduct