Draft: [WIP] New package realm, legion update, gasnet update

[rbberger]

Realm is being split into its own repository. This starts the process of upgrading the Spack packages to reflect this new reality. This is still WIP.

[rbberger]

@PHHargrove @bonachea FYI, given the changes in Legion/Realm and how they will move forward in building, I have to push forward with these GASNet package changes to support work at LANL. I'm currently using this branch for testing, but I'm planning to move the GASNet part into its own PR. I'm explicitly doing things against the wishes/recommendations of what is documented in the comments of that Spack package and tried to update the language accordingly. I'm happy to discuss these changes here or in another forum. @elliottslaughter FYI

[rbberger]

Note to self, this might have to be changed back to github. This is for testing only @elliottslaughter

[elliottslaughter]

Let me know when you want a review on this.

[mpokorny]

@rbberger is this something I could try out for Legion/Realm + Kokkos? I'm sort of in a jam, as my application needs to use a post 25.09 version of Legion for some new commits, and the current Spack package for Legion doesn't work correctly with +kokkos for 25.09 or later. I was able to get a successful build after modifying the current Spack package and some CMakeLists.txt files, but I'd rather try this out if it's usable in a testing situation. Note that I do not need gasnet for the tests I want to run in the near term.

[rbberger]

@mpokorny yes, this is my most current workable state.

[mpokorny]

I've found one small problem, which can be solved with a patch file for, and a patch to, the realm package. The problem is a malformed string() command in CMakeLists.txt, and it exists in 25.6.1-rc.4 but not main. What's the preferred way for me to provide the needed fixes (single line addition to package.py, and one new, single-line patch file)?

[mpokorny]

Another issue: max_dims variant value is not being propagated from legion to realm.

[rbberger]

@mpokorny I would wait for the actual release and not try to patch rc versions. I'll remove them prior to merge. Thanks for pointing out the missing dim propagation.

[mpokorny]

Also, the sysomp variant isn't being propagated from legion to realm.

[rbberger]

@mpokorny fyi, we noticed we can't build Realm with Clang as CUDA compiler due to missing fatbin compilation. So CUDA compiler has to remain nvcc, even if Kokkos was configured with Clang.

[mpokorny]

I've run into some issues here

spack-packages/repos/spack_repo/builtin/packages/legion/package.py

Line 438 in ac1c0d7

os.environ["KOKKOS_CXX_COMPILER"] = self["kokkos"].kokkos_cxx

with a cpu only build. There's a crash when accessing the kokkos_cxx property.
Upon inspection of the Spack packages and cmake files for Kokkos and Legion, I'm pretty certain that Legion (or Realm) doesn't use or need the value of the environment variable that is set here for any version of Kokkos >= 3.2 (or so). Given that Realm now (or soon) will require Kokkos >= 4, it should be safe to remove this line, at least conditionally. I removed it unconditionally in the cpu-only build, with no ill effect (although I still need to do more verification).

[rbberger]

@mpokorny sry, should have told you that for the Realm changes in StanfordLegion/realm#385 I'm only using #2941 (note, no real GASNET support except external)

[rbberger]

Given that Realm now (or soon) will require Kokkos >= 4, it should be safe to remove this line, at least conditionally

Yes, since Legion doesn't have any explicit Kokkos code paths, we don't have any direct dependency to Kokkos anymore except through Realm. So that should go away.

[mpokorny]

I did notice that, but since the issue I mentioned is in the Legion package, I figured that I ought to mention it. (I still have to use the Legion package here in order to build a recent Legion, modified by some of my own patches.)

[mpokorny]

Almost...you're forgetting about the examples, which do have Kokkos code.