[Target Info] Compute resource-directory relative to the SDK on non-Darwin targets

[artemcm]

Since the new driver uses -print-target-info for the resource directory and the legacy C++ driver used the logic in getResourceDirPath(), mimic that logic now in the print-target-info job itself.

[artemcm]

@swift-ci test

[artemcm]

@swift-ci test macOS platform

[finagolfin]

Anything holding this up, @artemcm?

The new cross-compilation doc @compnerd wrote up in #73581 is wrong with the new swift-driver as long as this issue is not fixed, ie the Swift compiler won't even look in -sdk for the Swift SDK overlay, whereas it long has with the legacy Driver as I demonstrated in swiftlang/swift-driver#1562.

[artemcm]

@swift-ci test

[artemcm]

@swift-ci test

[artemcm]

@swift-ci Please Build Toolchain Windows Platform

[artemcm]

@swift-ci test

[hyp]

This doesn't work for Android, where the toolchain contains Swift resources, and SDK has the Swift runtime and swiftrt.o. With this patch, swift is unable to find the Clang builtins headers

[finagolfin]

This doesn't work for Android, where the toolchain contains Swift resources, and SDK has the Swift runtime and swiftrt.o.

That doesn't make any sense: can you clarify how you are building, ie what paths have what? It sounds like you are using a highly non-standard layout.

With this patch, swift is unable to find the Clang builtins headers

I suppose that's possible depending on your layout, but again, we'll need more info.

[hyp]

@finagolfin

The Android SDK is packaged separately from the toolchain in the following way when distributed with our windows toolchain, here are some of our Swift standard library / runtime pieces:

    Directory: S:\Program Files\Swift\Platforms\Android.platform\Developer\SDKs\Android.sdk\usr\lib\swift\android


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----          7/9/2024  11:17 AM                aarch64
d-----          7/9/2024  11:17 AM                Android.swiftmodule
d-----          7/9/2024  11:17 AM                Cxx.swiftmodule
d-----          7/9/2024  11:17 AM                CxxStdlib.swiftmodule
d-----          7/9/2024  11:17 AM                Dispatch.swiftmodule
d-----          7/9/2024  11:17 AM                Distributed.swiftmodule
d-----          7/9/2024  11:17 AM                Foundation.swiftmodule
d-----          7/9/2024  11:17 AM                FoundationNetworking.swiftmodule
d-----          7/9/2024  11:17 AM                FoundationXML.swiftmodule
d-----          7/9/2024  11:17 AM                Observation.swiftmodule
d-----          7/9/2024  11:17 AM                RegexBuilder.swiftmodule
d-----          7/9/2024  11:17 AM                Swift.swiftmodule
d-----          7/9/2024  11:17 AM                SwiftOnoneSupport.swiftmodule
d-----          7/9/2024  11:17 AM                Synchronization.swiftmodule
d-----          7/9/2024  11:17 AM                _Concurrency.swiftmodule
d-----          7/9/2024  11:17 AM                _Differentiation.swiftmodule
d-----          7/9/2024  11:17 AM                _math.swiftmodule
d-----          7/9/2024  11:17 AM                _RegexParser.swiftmodule
d-----          7/9/2024  11:17 AM                _StringProcessing.swiftmodule
d-----          7/9/2024  11:17 AM                _Volatile.swiftmodule
-a----          1/8/2024  12:42 PM            105 libcxxshim.h
-a----          1/8/2024  12:42 PM            141 libcxxshim.modulemap
-a----          3/6/2024  11:16 AM            986 libcxxstdlibshim.h

    Directory: S:\Program Files\Swift\Platforms\Android.platform\Developer\SDKs\Android.sdk\usr\lib\swift\android\aarch64


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----          7/9/2024   9:45 AM          13262 android.modulemap
-a----          7/9/2024  11:05 AM          30968 libBlocksRuntime.so
-a----          7/9/2024  11:05 AM        1521384 libdispatch.so
-a----          7/9/2024  11:10 AM       70614952 libFoundation.so
-a----          7/9/2024  11:10 AM        5398216 libFoundationNetworking.so
-a----          7/9/2024  11:10 AM        2860624 libFoundationXML.so
-a----          7/9/2024  10:59 AM          91624 libswiftAndroid.so
-a----          7/9/2024  10:59 AM       10005168 libswiftCore.so
-a----          7/9/2024  10:59 AM          99352 libswiftCxx.a
-a----          7/9/2024  10:59 AM         150846 libswiftCxxStdlib.a
-a----          7/9/2024  11:05 AM        1030232 libswiftDispatch.so
-a----          7/9/2024  10:59 AM         196896 libswiftDistributed.so
-a----          7/9/2024  10:59 AM         103680 libswiftObservation.so
-a----          7/9/2024  11:01 AM         153744 libswiftRegexBuilder.so
-a----          7/9/2024  10:54 AM        1742408 libswiftRemoteMirror.so
-a----          7/9/2024  10:59 AM         411200 libswiftSwiftOnoneSupport.so
-a----          7/9/2024  10:59 AM         124136 libswiftSynchronization.so
-a----          7/9/2024  10:59 AM        1388080 libswift_Concurrency.so
-a----          7/9/2024  10:59 AM         669048 libswift_Differentiation.so
-a----          7/9/2024  10:59 AM          10824 libswift_math.so
-a----          7/9/2024  11:00 AM        1656960 libswift_RegexParser.so
-a----          7/9/2024  11:01 AM        1320296 libswift_StringProcessing.so
-a----          7/9/2024  10:59 AM          13984 libswift_Volatile.so
-a----          7/9/2024   9:45 AM           3671 SwiftAndroidNDK.h
-a----         6/11/2024   4:15 PM           1334 SwiftBionic.h
-a----          7/9/2024  10:54 AM           6096 swiftrt.o

This SDK is passed to our compilation using the -sdk flag (-sdk S:\Program Files\Swift\Platforms\Android.platform\Developer\SDKs\Android.sdk).

We expect a file like swiftrt.o and all the SOs to be in the SDK.

However, the compiler and it's resource dir, with the clang builtin headers as located in the toolchain, just like the compiler binary itself:

    Directory: S:\Program Files\Swift\Toolchains\0.0.0+Asserts\usr\lib\swift


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         4/26/2024  10:14 AM                clang
d-----          7/9/2024  10:22 AM                host
d-----         4/26/2024  10:14 AM                migrator
d-----         4/26/2024  10:41 AM                pm
d-----         4/26/2024  10:14 AM                swiftToCxx
d-----          7/9/2024  11:28 AM                windows
d-----          6/6/2024  10:15 AM                _InternalSwiftScan
d-----         4/26/2024  10:14 AM                _InternalSwiftStaticMirror

This layout is somewhat similar to Darwin, where the standard library is separated out and is in the SDK as opposed to the toolchain itself (except for things like pre-compiled modules).
This is also how the windows SDK is separated out from the toolchain as well.

[finagolfin]

We expect a file like swiftrt.o and all the SOs to be in the SDK.

OK, the problem is not that, as other SDKs also have those Swift runtime files. The problem is that you're placing the shared libraries in the arch-specific directory usr\lib\swift\android\aarch64\, which is not how Unix platforms like linux or Android do it in the Swift compiler.

Why are you so particular about that, when you only have a single arch in that directory? I understand if you had multiple arches, but you don't.

I'm all for organizing the Swift runtime libraries for multiple arches on Unix platforms like that, as I submitted several pulls to move the Unix toolchain to that last year, #63782, but that approach was rejected, even when I said it could be a temporary measure.

The second difference in your non-standard layout is that you don't place the clang builtin headers in the SDK, as I think even Darwin platforms do. I understand that @compnerd wants to separate those out and change the meaning of -resource-dir to only refer to those "compiler resource" files, but I don't think you should start by shifting to that new model for a single SDK like this, ie the Android SDK on Windows.

I think it would be easier if we moved all SDKs to this new cross-compilation model at the same time, or at least all non-Darwin SDKs. Otherwise, we will be stuck special-casing particular SDKs like this with a bunch of driver logic until we move them all over.

Try something: apply this pull and remove your patch from swiftlang/swift-driver#1560. Then, move all the lib* files from Android.sdk\usr\lib\swift\android\aarch64\ into Android.sdk\usr\lib\swift\android\ and link or move Toolchains\0.0.0+Asserts\usr\lib\swift\clang\ into Android.sdk\usr\lib\swift\. With those two changes, this pull should get everything building for you again, depending on the other contents of Android.sdk\usr\lib\swift\.

[compnerd]

@finagolfin we only have a single directory there because it takes time to build. The installer will have all the architectures under the same directory, just like Windows already does with i686, x86_64, and aarch64.

Also, it is not a single SDK that we are changing to the resource headers being organised this way. Windows has long used this layout and the Android SDK we are building and packaging for Windows uses the same layout.

Right now, at least on Windows, there are only 2 SDKs - Windows and Android. I don't have a problem with the other SDKs also using the same layout though.

[finagolfin]

The installer will have all the architectures under the same directory, just like Windows already does with i686, x86_64, and aarch64.

OK, makes sense, but what is your plan here? When I suggested a similar organization last year, it was rejected. We shouldn't special-case only the Android SDK on Windows to work that way, but all other Unix platforms don't.

Also, it is not a single SDK that we are changing to the resource headers being organised this way.

I haven't used the Windows toolchain, but I find this hard to believe based on the common driver logic. For example, when you compile natively for Windows, what is the -resource-dir flag passed to the swift-frontend by swift-driver set to? I have detailed that extensively for linux in swiftlang/swift-driver#1562.

[compnerd]

OK, makes sense, but what is your plan here? When I suggested a similar organization last year, it was rejected. We shouldn't special-case only the Android SDK on Windows to work that way, but all other Unix platforms don't.

It has to be a gradual roll out. As we start building more of the platform SDKs, we will start fixing that. Right now the build setup makes this complicated to do as a generic, single shot application. As work progresses towards cleaning up the builds, we should be able to do this more readily across all the platforms.

I haven't used the Windows toolchain, but I find this hard to believe based on the common driver logic. For example, when you compile natively for Windows, what is the -resource-dir flag passed to the swift-frontend by swift-driver set to?

> swiftc -v -c reduced.swift -o NUL
compnerd.org Swift version 6.0-dev (LLVM 03696ed5c30fe6e, Swift c4b9e328ff84c4e)
Target: x86_64-unknown-windows-msvc
C:\Users\abdulras\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe -frontend -c -primary-file C:\Users\abdulras\AppData\Local\Temp\reduced.swift -target x86_64-unknown-windows-msvc -disable-objc-interop -sdk C:\Users\abdulras\AppData\Local\Programs\Swift\Platforms\0.0.0\Windows.platform\Developer\SDKs\Windows.sdk -color-diagnostics -empty-abi-descriptor -Xcc -working-directory -Xcc C:\Users\abdulras\AppData\Local\Temp -resource-dir C:\Users\abdulras\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\lib\swift -module-name NUL -plugin-path C:\Users\abdulras\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin -plugin-path C:\Users\abdulras\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\local\bin -o C:\Users\abdulras\AppData\Local\Temp\NUL

It is set to the expected path - the resource directory under the toolchain.

[finagolfin]

It has to be a gradual roll out. As we start building more of the platform SDKs, we will start fixing that. Right now the build setup makes this complicated to do as a generic, single shot application.

IMO, a gradual roll-out will be much more complicated, but I could be wrong.

It is set to the expected path - the resource directory under the toolchain.

Thanks, I will try running the Windows toolchain myself to see how it does things differently.

@etcwilde, it appears that Darwin, Unix, and Windows currently place all these SDK/resource files in different layouts. If we're going to put together cross-compilation SDK bundles that work on all platforms, we're going to have to come up with a way to regularize all this. Rolling it out gradually with a bunch of special-case logic like in swiftlang/swift-driver#1560, for Android in that case, seems like it's going to get messy to me.

You're working on cross-compilation support, what do you think? Pinging @MaxDesiatov too.

[finagolfin]

This just fixes the narrow problem of passing the right path for the Swift core modules back to swift-driver, but I raised a larger concern in the issue that motivated this pull, swiftlang/swift-driver#1562, that the swift-frontend is not looking in a passed in -sdk for those Swift core modules.

Can this pull be broadened to fix that too, especially since @compnerd has proposed looking in -sdk primarily for those Swift core modules?

Right now, the way it works is that swift-driver queries swift-frontend for what paths to use then uses those to set -sdk and -resource-dir flags to pass back to the swift-frontend for compiling new Swift files. During compilation, swift-frontend only checks -resource-dir and adjacent to the compiler for the Swift core modules like the stdlib.

I think we should change swift-frontend to look in -sdk and adjacent to the compiler instead and stop passing -resource-dir to the swift-frontend unless one was specified to swift-driver. I noted in the linked issue that the legacy C++ Driver worked that way, ie it did not pass -resource-dir to the swift-frontend unless one was passed to the Driver first.

This would be in keeping with Saleem's plan to de-emphasize -resource-dir to look for Swift core modules and look in -sdk instead.

[finagolfin]

I think we can move forward with this, indeed expand it as I laid out above. While it may not fix whatever additional issues Alex and Saleem have with their non-standard Android SDK, it does partially fix the swift-driver issue I originally raised of swift-driver not looking in -sdk for Swift modules and should be expanded more to emphasize usage of -sdk in swift-frontend.

Let me know what you all think.

[finagolfin]

@compnerd, you want to chime in on what I wrote last week? This pull fixes the issue that the swift-driver does not currently look in -sdk for Swift core modules, which your new cross-compilation doc says will eventually be the primary way to compile Swift files. I suggested that we go further and update where swift-frontend looks for those core modules too.

[finagolfin]

Ping @artemcm, been six months since you proposed this fix and nothing has happened for the last couple months. If you don't have time for this, would you like me to take over the fix for the linked swift-driver issue?