Run more tests in parallel

[hauntsaninja]

At some point, Github Actions runners started having more cores: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories

[manmartgarc]

Couldn't we use -n auto? Here is a GitHub action using that and using 4 workers. https://github.com/manmartgarc/stochatreat/actions/runs/8866655781/job/24344339586

[hauntsaninja]

I'm not sure of the history here / I'm not sure what the exact specs of the machines underlying Github runners are.

But reading the xdist code: https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/plugin.py#L32
If you have psutil installed in your env (which mypy does), psutil.cpu_count() will give you the incorrect value in containerised environments. We'd want the sched_getaffinity number or len(psutil.Process().cpu_affinity()). (It is of course also possible that Github is actually running these on 4 core machines, in which case the numbers would be the same)

[manmartgarc]

Seems like it's 4; and using -n auto runs 4 workers with pytest-xdist. I just think it might be nice not to have to change this again in the future manually; would approve if I could 😄 .

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

[JukkaL]

Faster CI sounds great! What's the impact on the various checks we run?

[manmartgarc]

Faster CI sounds great! What's the impact on the various checks we run?

Latest one in main here, latest one from this PR here.

Seems like 2h 23m 57s to 2h 1m 12s ~15% percent change decrease 🐎

[JukkaL]

The improvements look good, and in line with what I'd expect, if the underlying hardware can run 2 threads per core.