Provide a non-invasive fix for the stopwatch issue

[JanVogelsang]

This PR is a simplified version of #3581 which doesn't require C++20.
It simplifies the stopwatch implementation by combining all template specializations into a single one which uses if constexpr constructs and some clever std::conditional_t usage to use the correct timer containers and functionality based on compile flags.

[heplesser]

@JanVogelsang Concerning the placement of the assert_thread_parallel() test in

https://github.com/JanVogelsang/nest-simulator/blob/0373a93b8cc88d8d46843e04f59fb428eb38eeae/nestkernel/stopwatch_impl.h#L39-L46

and corresponding locations, consider what happens when we add nodes to the network in

https://github.com/JanVogelsang/nest-simulator/blob/0373a93b8cc88d8d46843e04f59fb428eb38eeae/nestkernel/node_manager.cpp#L104-L106

Here, sw_construction_create_ is a master-only timer and it is started in a serial context. Indeed, because it is a master-only timer, it must only be started/stopped from a serial context. For this timer, because it is master-only, we will have enable_timer == true (always, because it is a normal granularity timer) and use_timer_array == False (always, because it is a master-only timer). Now if we had assert_thread_parallel() inside the enable_timer but outside the use_timer_array conditions, the assertion will also be tested for master-only timers correctly called from serial context and then it must fail.

So the assertion must only be tested when we actually have a threaded timer, i.e., inside the use_timer_array. Then it needs to be tested, because if we started/stopped threaded timers in a serial context we would end up with inconsistent results.

No, strictly speaking, we could argue that timers started/stopped from parallel contexts should always use a timer array. The with-threaded-timers option should not exist. Then, instead of the #pragma omp master we could have a assert_single_threaded(). But since we want to support single-valued timers also in parallel contexts for backward compatibility, we need the pragma and cannot have the assertion. I will suggest a comment.

[heplesser]

@JanVogelsang Very nice work! I just added one suggestion for a comment.

[heplesser]

BTW, I have checked all four detailed/threaded combinations and they work as intended. Very nice that now only the actually active timers show in the kernel dictionary.

[JanVogelsang]

I see, so essentially we have two different non-threaded timer types:

1. True "master-only" timers which are called in a parallel environment, but only by the master thread.
2. Timers in a serial environment.
   In theory we could distinguish these two types as those are different cases, but as the #pragma omp master works even in serial environments, it's fine to just use the same code for both cases.

[heplesser]

I see, so essentially we have two different non-threaded timer types:

1. True "master-only" timers which are called in a parallel environment, but only by the master thread.
2. Timers in a serial environment.
   In theory we could distinguish these two types as those are different cases, but as the #pragma omp master works even in serial environments, it's fine to just use the same code for both cases.

I would state it the other way around. I would consider timers such as

  Stopwatch< StopwatchGranularity::Normal, 
             StopwatchParallelism::MasterOnly > sw_construction_create_;

which are explicitly defined as MasterOnly as "true master-only" timers. These should only ever be called from a serial context.

Then we have timers declared as thread-parallel, e.g.,

Stopwatch< StopwatchGranularity::Normal, 
           StopwatchParallelism::Threaded > sw_communicate_prepare_;

which we coerce to run master-only when Dwith-threaded-timers=OFF is set. This does not really make sense, we only do it for backward compatibility with the way we took measurements in earlier versions of NEST.

[mlober]

Thanks for fixing this so quickly. All my tests with the various configuration options were successful.

[gtrensch]

Thank you for this contribution. 👍