test_instance_check_performance fails on some architectures

[mcepl]

When packaging this package for openSUSE I have hit the similar problem on two non-x86_64 architectures with the test test_instance_check_performance. On i586 I get:

[  168s] ======================================================================
[  168s] FAIL: test_instance_check_performance (tests.test_performance.PerformanceTest)
[  168s] ----------------------------------------------------------------------
[  168s] Traceback (most recent call last):
[  168s]   File "/home/abuild/rpmbuild/BUILD/nptyping-2.0.1/tests/test_performance.py", line 25, in test_instance_check_performance
[  168s]     self.assertLess(second_time_sec, first_time_sec)
[  168s] AssertionError: 0.0007986779999953342 not less than 0.00042071000000021286

(this is the complete build log on i586 with all packages used and steps taken to reproduce).

And this happens on aarch64:

[  164s] ======================================================================
[  164s] FAIL: test_instance_check_performance (tests.test_performance.PerformanceTest)
[  164s] ----------------------------------------------------------------------
[  164s] Traceback (most recent call last):
[  164s]   File "/home/abuild/rpmbuild/BUILD/nptyping-2.0.1/tests/test_performance.py", line 26, in test_instance_check_performance
[  164s]     self.assertLess(second_time_sec, 0.0002)
[  164s] AssertionError: 0.0002229950000014469 not less than 0.0002

(again, the complete build log).

Any idea, what's going on, please?

[ramonhagenaars]

Performance tests are a bit tricky, because they can be influenced by external processes that eat up your CPU. That would be my first guess as to why these are failing.

The first one (on i586) I find especially peculiar: second_time_sec should be lower due to the use of caching. Maybe on that architecture, the lru caching is not optimally implemented or something?

[ggardet]

This is on build farms, so the CPUs can be very busy. I guess we could increase the threshold?

[ramonhagenaars]

Yes I would say so.