Added methods to sort and squash segments in the IbdFinder output.

[gtsambos]

Fixes #2459 -- see the issue for discussion about why this is needed. Draft only at the moment.

PR Checklist:

• Tests that fully cover new/changed functionality.
• Documentation including tutorial content if appropriate.
• Changelogs, if there are API changes.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.72%. Comparing base (fb87f87) to head (7fe0bb8).
Report is 6 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2460   +/-   ##
=======================================
  Coverage   89.72%   89.72%           
=======================================
  Files          29       29           
  Lines       31578    31578           
  Branches     6117     6117           
=======================================
  Hits        28333    28333           
  Misses       1853     1853           
  Partials     1392     1392           

Flag	Coverage Δ
c-tests	86.55% <ø> (ø)
lwt-tests	80.78% <ø> (ø)
python-c-tests	88.98% <ø> (ø)
python-tests	99.01% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

[gtsambos]

Hey @hyanwong! Unfortunately, we both made some pretty involved changes to the test_highlevel.py file in the past few days. I think I've fixed up the merge conflict, but I can't figure out how to get my tests to run with the argument that I added into get_example_tree_sequences(). Currently I'm just skipping these tests because I don't fully understand the syntax of your additions -- do you know how to ensure these tests 'see' the argument I've added in?

[hyanwong]

It was @benjeffery who fixed up get_example_tree_sequences() to run in parallel, cached, so that parameterize worked properly. He might be able to suggest a fix?

[benjeffery]

I'll take a look now

[benjeffery]

I think the simplest thing here is to remove custom_max.
I've done this in a3aa3da
Currently, I've set the N numbers to those generated when custom_max was 15. Once all the IBD tests pass we should add in some even numbers again and skip them in the IBD tests? (I'm assuming that you need them to be odd, as all those generated by custom_max were?

[gtsambos]

Thanks @benjeffery and sorry for the late reply -- I had to take a few days off to deal with some urgent moving-to-the-US things.

Currently, I've set the N numbers to those generated when custom_max was 15. Once all the IBD tests pass we should add in some even numbers again and skip them in the IBD tests? (I'm assuming that you need them to be odd, as all those generated by custom_max were?

I added this argument because some of my changes in this PR make IbdFinder run much more slowly, to the point where they look like they're hanging. (This is likely a Python-specific problem -- as we discuss in #2459, the C code works differently because it uses an AVL tree to do the sorting). But I didn't want to omit the tests entirely, as they cover some useful edge cases, so I wanted to run them with smaller sample sizes than the ones hard-coded in.

[benjeffery]

I added this argument because some of my changes in this PR make IbdFinder run much more slowly, to the point where they look like they're hanging. (This is likely a Python-specific problem -- as we discuss in #2459, the C code works differently because it uses an AVL tree to do the sorting). But I didn't want to omit the tests entirely, as they cover some useful edge cases, so I wanted to run them with smaller sample sizes than the ones hard-coded in.

We can leave the slow examples in the examples - but skip them in these tests?

[benjeffery]

I've rebased this to main - the tests here only add a few seconds on my machine, will see home CI behaves. @jeromekelleher I think this is pretty complete if you want to have a quick look to check?

[benjeffery]

@Mergifyio update

[mergify]

update

✅ Branch has been successfully updated