[Help] How to generate the results file?

[pdet]

Hey guys, first of all very cool project and congratulations on the paper!

Also thanks for the effort in open-sourcing and documenting it, I managed to rather quickly add DuckDB up because the code/docs are pretty clean :-)

I got to the point where I can run
docker-compose up duckdb-client

Where results/duckdb/polluted_files is then created with the read/written files from DuckDB and in the loading folder, with an extra file duckdb_time.csv with the timings of each query.

However no duckdb_results.csv file was generated, by looking in the code I can't really figure out which script is generating it, could you point me out how to do is?

Many thanks and congrats again!

[vitaglianog]

Hello @pdet !
Thanks for the kind words and for your effort to apply Pollock to DuckDB.
If you successfully populated the folder results/duckdb/polluted_files/loading/ with the query results, you should then run:
docker-compose run evaluate python3 evaluate.py --sut duckdb
This script will compute the Pollock metrics/score, produce the duckdb_results.csv and also update the general results\aggregate_results_polluted_files.csv.

You can find more details on this procedure in the readme section "Extra: Benchmark a single SUT", and perhaps you can also run the benchmark on the survey files by changing the dataset in the .env file.

Let me know if you are successful, and if you feel like you can create a PR - it would be nice to include DuckDB in our repository!