Add 4.14+trunk+nnp and 4.14+trunk+nnp+flambda

[gadmm]

I think it would be a good idea to add the latest OCaml 4 as a comparison point and also to help with 4.14 development. I expect that it fails of course the parallel test, but I hope that it will give helpful figures for the sequential test.

• I have included both flambda and non-flambda configurations. I was hoping to test both -O2 and -O3 for flambda, but unfortunately I have not found the config option to pass either option during compilation.
• If tracking 4.14+trunk is going to be too expensive, then we could track instead a fixed commit (e.g. 4.14.0).

[shakthimaan]

Thanks for your PR!

At present, we are tracking 5.1.0+trunk, but, we do have 4.14.0 building with Sandmark for both sequential and parallel benchmarks. We will soon add support for 4.14 in the nightly runs, and the issue can be tracked at Support more than one development variant for nightly runs. As soon as we have nightly runs for 4.14.0, we will merge this PR.

I was hoping to test both -O2 and -O3 for flambda, but unfortunately I have not found the config option to pass either option during compilation.

You can add it in the configure entry in the JSON file? This lists the options to be passed when building the compiler. See: https://github.com/ocaml-bench/sandmark-nightly-config/blob/main/README.md?plain=1#L45

[gadmm]

Thanks for the reply

You can add it in the configure entry in the JSON file? This lists the options to be passed when building the compiler. See: https://github.com/ocaml-bench/sandmark-nightly-config/blob/main/README.md?plain=1#L45

The option -O2 and -O3 must be passed to the compiler when building the benchmarks. The configure field does not seem to do this. The table misses a field for the options used while building the benchmarks, or am I missing something?

[shakthimaan]

After we add support for 4.14, please update the PR to use a short-time interval, such as a week or ten days. Thanks!

[gadmm]

The idea is to follow the development branch for 4.14. But I suppose this depends on #6.

[shakthimaan]

Yes, that is correct!

[shakthimaan]

The option -O2 and -O3 must be passed to the compiler when building the benchmarks.

At present, one can:

1. Build the compiler passing configure options (such as "-q"),
2. Specify runparams which are passed to OCAMLRUNPARAM (such as v=0x400), and
3. Pass environment options such as OCAMLRUNPARAM='s=2M' when executing the benchmarks.

The configure field does not seem to do this.

Yes, that is correct.

The table misses a field for the options used while building the benchmarks, or am I missing something?

We have exposed minimal options for the sandmark-nightly runs for simplicity.

Sandmark does have multiple options available which are documented in the Makefile Variables seciton at the end of https://github.com/ocaml-bench/sandmark/blob/main/README.md. But, there is not yet an option to customize the building of the benchmarks themselves. We will create an issue to handle this request.

[shakthimaan]

We have 4.14 building for Sandmark for sequential benchmarks. We are just waiting (@Sudha247) for a version of domainslib to use with 4.14 in order to run the parallel benchmarks.
(Note: Since there is no parallelism in 4.14, the parallel benchmarks will run sequentially only).