Implement Arrow flight SQL interface for kyuubi

[msucharda]

Hello,

with recent release of Arrow flight SQL, would it make sense to implement it in Kyuubi? Since we are using Kyuubi as ODBC endpoint for our BI tools, we sometimes need to stream large amounts of data. Currenty we are facing issues with limitations of ODBC (throughput) and Spark itself (SQL query result gets collected and then streamed via ODBC, causing OOM issues).

Arrow flight might solve both of these issues and it seems, comunity is moving towards it.

Here are some itroductory information: https://arrow.apache.org/blog/2022/02/16/introducing-arrow-flight-sql/

[ulysses-you]

thank you @Suchosh

I did a quick look. It's a so powerful framework ! Some important things for Kyuubi:

• Flight leverage the columnar format of Arrow to optimize the transport
• Flight construct the transport over grpc
• Flight is going to provide a JDBC driver

I think it's a good thing for Kyuubi. Look forwad to next.

[pan3793]

I'm going to start a PoC #3356