fairer array sorting algorithm

[rBangay]

What does this PR change?

Replace the existing shuffle array sort function with an algorithm that fairly randomises the contents. The new approach is based on the fisher-yates algothrithm (https://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle) a nice example of which you can see here: Fisher–Yates Shuffle

Current situation/background

@johnduffell rightly pointed out in a previous pr (#1415 (comment)) that the randomisation of the output array wasn't all that random;

results from his testing:

Object.entries([...Array(1000000).keys().map(() => [1,2,3,4,5,6,7,8,9,10].sort( () => .5 - Math.random() )[0])].sort().reduce((acc, num) => {const e = {...acc};e[num]=acc[num] + 1; return e;}, {1:0,2:0,3:0,4:0,5:0,6:0,7:0,8:0,9:0,10:0})).map(([num, count]) => `${num}: ${count/10000}%`).map((a) => console.log(a))
1: 19.5461%
2: 7.2563%
3: 9.6343%
4: 12.8512%
5: 8.3207%
6: 8.9094%
7: 9.9178%
8: 11.1279%
9: 6.0317%
10: 6.4046%

It shows a weighting more specifically to the first element in the array (ideally these figures ought to be as close to 10% each in order to show a fair distribution)

and with the new Fisher–Yates Shuffle algorithm we can see the results much more evenly distributed:

1: 9.9887%
2: 10.0008%
3: 9.9885%
4: 10.0155%
5: 10.0069%
6: 9.9565%
7: 9.9425%
8: 9.9936%
9: 10.0209%
10: 10.0861%

[johnduffell]

nice one! good bit of algorithms. why not add a unit test to it to round it off?

[johnduffell]

as the last step, this will always swap the first element with itself. Which is not in itself a problem as it's a no-op, but it was interesting.

[johnduffell]

nice work, big improvement 👏

[rBangay]

nice one! good bit of algorithms. why not add a unit test to it to round it off?

I have added a unit test, but it was tricky to know how to properly test this. I didn't want to add a test that ran too many cycles to find whether the spread was even or not so in the end I just wrote a test that calls the function 100 times and checks that the output array order is not the same as the input array order .... the input array size was 10, I guess in theory the test could fail if by chance the shuffle resulted in an array with the elements in the same order but I fugured that was incredibly unlikely enough to make this test valid.

[johnduffell]

nice one! good bit of algorithms. why not add a unit test to it to round it off?

I have added a unit test, but it was tricky to know how to properly test this. I didn't want to add a test that ran too many cycles to find whether the spread was even or not so in the end I just wrote a test that calls the function 100 times and checks that the output array order is not the same as the input array order .... the input array size was 10, I guess in theory the test could fail if by chance the shuffle resulted in an array with the elements in the same order but I fugured that was incredibly unlikely enough to make this test valid.

probably not worth going around the houses too much but if we're running it 100 times we should expect each number should be in each position at least a few times

[johnduffell]

nice work 👏

[prout-bot]

Seen on PROD (merged by @rBangay 12 minutes and 40 seconds ago) Please check your changes!