Revert to default fullLTO on Clang

[Fidget-Spinner]

Feature or enhancement

Proposal:

Python 3.12 changed the default for Clang to ThinLTO. However, many people were unaware of this change and did not update their build scripts. This leaves a lot of perf on the table for macOS and possibly some other platforms.

1. CPython was bitten by this: Python3.13 performance Issue with python.org macOS installers on ARM Macs #122580
2. Faster CPython was bitten by this: clang builds not using full LTO faster-cpython/bench_runner#342
3. python-build-standalone (Astral) also seems to not have the fix: On Clang, LTO is ThinLTO, which leaves a lot of performance out for macOS astral-sh/python-build-standalone#528
4. msys2/mingw also seems to have not noticed: python: build with full lto msys2/MINGW-packages#23384

This seems to be confusing and tripping up a lot of people. I propose we change the --with-lto default back to full.

cc @brandtbucher @corona10

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• gh-130048: Reintroduce full LTO as default #130049
• Revert "gh-130048: Reintroduce full LTO as default on Clang (GH-130049)" #130088

[brandtbucher]

Heh, this was on my backlog of issues to open. I fully support this... to me, --with-lto has always meant "take a long time to compile the fastest executable".

[corona10]

Okay, but when the people can be ready to change if we want to change the policy? Why not propose to people to use lto=full if they want? (Yeah It's grumpy comment but I just ask)

[corona10]

Or criteria will be any platform(at least tier1)should not be slow down if we want to change the default build policy?
It means that we would have same criteria for other build configuration too.

[Fidget-Spinner]

Okay, but when the people can be ready to change if we want to change the policy? Why not propose to people to use lto=full if they want? (Yeah It's grumpy comment but I just ask)

I think the problem is that people assume lto means lto full, and they forget that it's lto thin now on clang.

Or criteria will be any platform(at least tier1)should not be slow down if we want to change the default build policy?

Yeah that seems reasonable.

[Fidget-Spinner]

For the record, @zanieb tested this on Clang 19.1 (not apple Clang), and couldn't reproduce the slowdowns seen on macOS. So I suspect the latest clang versions have fixed this bug astral-sh/python-build-standalone#529

[Fidget-Spinner]

I deem it too disruptive to change a configure option twice and flip flop between the defaults just based on what compiler is available. I am thus reverting the change. Sorry!

[brandtbucher]

Wait, why was this reverted? Full LTO should always be the fastest option, right?

[corona10]

http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html

"In a few cases ThinLTO even outperforms full LTO, most likely because the higher scalability of ThinLTO allows using a more aggressive backend optimization pipeline (similar to that of a non-LTO build)."

So not that always truth. It’s where to bet, but I believe that compiler people make parallel LTO performance be better for the performance stuff too. Basically it’s about the execution mechanism not about the optimization policy.

[zanieb]

My benchmarks, though not rigorous, showed full LTO to be a little slower astral-sh/python-build-standalone#529 (comment) 🤷‍♀ I could do more benchmarks if people want? Or explore the difference here instead of downstream in python-build-standalone where there may be confounding factors?