[Proposal] Allow to use Spark Session in DataExporter

[rafaelcanto]

Is your feature request related to a problem? Please describe.
We need to scale our data ingestion and would be nice to have support to Spark when writing the outputs from DataExporter. When using the pyarrow, we also noted the number of parquet files grows and with maybe would be good also to allow to control this as a parameter.

Describe the solution you'd like
We would like to allow optionally passing a spark session instance to FocusConverter class to be passed down to DataExporter.

Describe alternatives you've considered
We're considering to fork the repository itself, but maybe the solution is needed by others. We didn't find other options to speedup the convertion. Any ideas are welcome!

[rafaelcanto]

If you don't mind, I can develope the proposal and send it as a PR. Let me know if this make any sense.

[rafaelcanto]

Hi! Anybody here?

[varunmittal91]

Hi @rafaelcanto that sounds great please let me know if there is anything I can help with.

[rafaelcanto]

Good! Thanks. I'll start the development this week and I'll keep you posted.