Provide benchmarks on MSVC

[brunocodutra]

We will have to deal with broken support for newer standards.
Two approaches strike me as the most obvious:

• Manually implement some nasty logic to decide what libraries to enable for each compiler version.
• Try compiling each benchmark (rendered for n=1) before deciding whether the selected compiler version is able to handle a library.

[brunocodutra]

I'm fond of the second approach, because it requires the least maintenance and because it looks cool.

[brunocodutra]

I'm fond of the second approach, because it requires the least maintenance and because it looks cool.

Forget it, too complex to be worth it.

[ldionne]

Sorry, I was out for the weekend. The issue with the second approach is that we won't know whether it's a compiler failure or a problem in the benchmark itself. IOW, we wouldn't be able to "test" that the benchmarks are not broken.

[brunocodutra]

Sorry, I was out for the weekend.

No worries.

Do you think you can setup Appveyor to push benchmark results in a manner similar to Travis?

EDIT: For MSVC only

[brunocodutra]

Also, just realized we never tested MSVC, as the Ninja generator picks MinGW.

EDIT: Fixed by 78b35bd (WIP)

[brunocodutra]

Surprise surprise, MSVC sets __cplusplus to 199711L.

Ideas?

[ldionne]

That's because MSVC is not C++11/14 compliant AFAIK.

[ldionne]

So there's nothing to do here; any benchmark that actually require full C++11/14 compliance can't support MSVC. Now, we could go with more granularity and require specific features cxx_constexpr, etc.. using COMPILER_FEATURES in CMake, but that's really not worth the added complexity, I think.

[brunocodutra]

That's because MSVC is not C++11/14 compliant AFAIK.

It isn't, but most libraries we actually care about don't need it either. Also see #93.

[brunocodutra]

As a matter of fact, Metal works from MSVC 14 onwards and Brigand from MSVC 12 onwards, not to mention MPL and Fusion which date back to over a decade ago.

I haven't found any special requirements for Meta, so I thought it could be handled by MSVC 14 least, but it appears it cannot. Also surprising is the fact MSVC 12 doesn't appear to handle std::tuples well and that it also chokes on test.one_template_many_datasets. I'll update the version checking later today before submitting a PR.

[ldionne]

Ok, so if I'm not mistaken we only need to run and push the benchmarks on MSVC from Appveyor, right?

[brunocodutra]

Ok, so if I'm not mistaken we only need to run and push the benchmarks on MSVC from Appveyor, right?

Correct, then update index.html on gh-pages.

[ldionne]

^ Because GCC 5 is already generated on Travis.