Support for user defined FFI functions

[westhide]

Description

Support transport UDF FFI library from client to Scheduler&Executor, and provoid interface load FFI into TaskDefinition's function_registry.
Relate to 2.2 Support for user defined functions

Describe the solution you'd like

1. Rust FFI

Datafusion already support loading rust FFI module, base on abi_stable

For rust ABI, we can send rust FFI library from client side to Scheduler&Executor, and load udf functiuons from the FFI library.

2. C/C++ FFI

For C/C++ FFI, we can provide a interface wrapper base on rust FFI by use cxx crate.

3. Wasm FFI

As @drauschenbach suggest, WASM is possible way to express UDF's in Rust. We can consider provide wasmtime udf for better secure and standards-compliant.

Describe alternatives you've considered

N/A

Additional context

N/A

[milenkovicm]

I'm not sure what's your idea @westhide, udf support is not one size fits all solution, thus I'm not sure how it fits with current solution. We spent a lot of time removing code, I'd be careful adding code.

Can you please elaborate a bit more?

[unknown-no]

Related WASM UDFs

[westhide]

Consider implement wasm UDF with Zero-copy Apache Arrow with WebAssembly