csbench 1.2

• Rename --allow-nonzero to --ignore-failure
• Add --load-csv option to load benchmark data
• Add --load-bin and --save-bin options to save and load benchmark data from/to custom binary format
• Multiple benchmark data files can be merged using --load-bin, allowing complex multi-step benchmarks
• Add --rename-name option to rename benchmark by name
• Add --baseline-name option to set baseline by name
• Rename --scan to --param-range and --scanl to --param
• Add ability to calculate p-values using Welch's t-test instead of MWU. This is contolled using new --stat-test option.
• Add --sort option to select order of benchmarks in report
• Improvements of command-line report
• Improvements of progress bar
• Man page and --help are updated with list of all options

csbench 1.1

Big update of functionality

• Move execution of custom measurements from main measurement loop. Execute them after all timings have been finished. This is done to minize work done during benchmark execution.
• Add ability for benchmarks to be suspended and resumed later - this is used when execuing in rounds. Round is a sequence of consequetive runs, optionally beginning with warmup.
• Round stop, warmup stop and benchmark stop are all controlled with 4 parameters: total number of rounds, maximum running time, minimum and maximum run count.
• Add --inputs option to specify string to be used for input.
• Add --inputd option to specify directory, all files from which should be used as input.
• Allow variables in --input and --inputs strings.
• Add option --common-args to specify string to be appended to each of the benchmarked commands.

csbench 1.0

Initial version of csbench.