Parallel output in tpchgen-cli (Nx faster, where N is number of cores)

[alamb]

• closes Implement multi-threading data generation #10
• closes (POC) Line item generation in parallel #34
• builds on Implement output as CSV #54

Features:

1. Logging support via -v and RUST_LOG
2. Parallel generation for all tables
3. No new dependencies on the tpch core crate and minor changes

Performance 🌶️

In single threaded mode, the generator can create 250MB/sec with each core which is 4GB/s on my 16 core laptop

This scales linearly with

Here are performance measurements for SF=100 on my Mac M3 (with 16 cores)

branch	speed	speed (writing to /dev/null)
main	6m1.984s	N/A
this branch	1m0.099s	0m28.732s

time target/release/tpchgen-cli -s 100 --output-dir=/tmp/tpchdbgen-rs

It actually turns out this can fully saturate the disk bandwidth on my Mac M3 laptop which caps out at 2GB/sec

If I hard code the generator to just throw the data away rather than writing, it keeps all the cores busy and writes at a blistering 4GB/sec

[alamb]

Ok, I need to stop for today, but this is looking promising

[alamb]

I plan to merge this PR tomorrow unless anyone else would like time to comment on it.

[clflushopt]

I will take a look at it tonight ! Thanks @alamb

[clflushopt]

LGTM; awesome stuff !!!

[alamb]

BTW with the other improvements on main, when merging it takes less than 5 seconds on my laptop to make TPCH SF 10 🚀

time target/release/tpchgen-cli -s 10  --output-dir=/tmp/tpchdbgen-rs

real	0m4.895s
user	0m36.096s
sys	0m2.910s