feat(default.nix): Init in /compiler+runtime #94
Conversation
|
Hey @jeaye, In 7585782, An error is occured about LLVM supported version. However github.com/llvm/llvm-project has no version, named as CMake Error at cmake/llvm.cmake:78 (message):
Found unsupported version: LLVM 19.1.1;
Please set llvm_dir pointing to the llvm version 19.0 to 19.0.x build or
installation folder
Call Stack (most recent call first):
CMakeLists.txt:113 (include) |
haruki7049
changed the title
|
Hey! I'd say, in order to get things compiling, we can just patch that CMake check to allow your 19.1.1 version. Everyone working on jank, right now, compiles Clang/LLVM from source and it's roughly 19.0.0 but it has a particular fix which hasn't made it into the 19.x release yet. I have a ticket for that here: llvm/llvm-project#111068 By using 19.1.1, we may run into some hard crashes when we run jank, but the main task here is getting it compiling with Nix. Using a different Clang version after we get things compiling should be much easier, especially if I can get that commit backported to 19.x. |
|
I went through our
Hopefully this is helpful. |
|
Tasks:
|
|
Memo: What is the /build/source/folly/logging/LoggerDB.cpp: In constructor 'folly::LoggerDB::LoggerDB()':
/build/source/folly/logging/LoggerDB.cpp:78:3: error: 'DCHECK' was not declared in this scope
78 | DCHECK(ret.second);
| ^~~~~~
/build/source/folly/logging/LoggerDB.cpp: In member function 'folly::LogCategory* folly::LoggerDB::createCategoryLocked(LoggerNameMap&, folly::StringPiece, folly::LogCategory*)':
/build/source/folly/logging/LoggerDB.cpp:473:3: error: 'DCHECK' was not declared in this scope
473 | DCHECK(ret.second);
| ^~~~~~
make[2]: *** [CMakeFiles/folly_base.dir/build.make:3366: CMakeFiles/folly_base.dir/folly/logging/LoggerDB.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:141: CMakeFiles/folly_base.dir/all] Error 2
make: *** [Makefile:136: all] Error 2 |
|
Hmm, so jank builds what it needs from folly as part of a custom target: https://github.com/jank-lang/jank/blob/main/compiler%2Bruntime/CMakeLists.txt#L323 Folly itself is a pain in the ass to build, so I've stripped out just the bits that jank needs and patched them to compile more easily. Are you building folly on its own, or using jank's CMake target for it? |
|
I was building folly only, thinking that I could just build folly as is. it is quite hard to use |
I think getting Nix to clone recursively (or just work with submodules in some way) would be beneficial for two reasons:
You know Nix better than I do, for sure. What I know, though, is that:
Are we able to work with that? |
|
Now my local repository was not cloned using Sure, if we have those files locally, we can reference them. Make But I'd like to say no, because I believe that building in Nix should be possible with or without submodules. |
Initializing submodules (either with Submodules are pure and reproducible, like any git commit. A single git commit in jank represents not only a snapshot of jank's source but also all of its submodules. Hopefully that's a good enough point. Thanks for working with me here. 🙂 すみません |
|
That may indeed be true. Ok, I'll take the method of using local submodules on Nix. And, no need to apologize. It's okay. NO PROBLEM!! |
|
Wait, The folly submodule should contain files other than those listed below. If you do not write the file exclusion settings in the submodule, you will end up writing the source file exclusion in Nix as well. set(
folly_sources
third-party/folly/folly/ScopeGuard.cpp
third-party/folly/folly/SharedMutex.cpp
third-party/folly/folly/concurrency/CacheLocality.cpp
third-party/folly/folly/synchronization/ParkingLot.cpp
third-party/folly/folly/synchronization/SanitizeThread.cpp
third-party/folly/folly/lang/SafeAssert.cpp
third-party/folly/folly/lang/ToAscii.cpp
third-party/folly/folly/system/ThreadId.cpp
third-party/folly/folly/system/AtFork.cpp
third-party/folly/folly/detail/Futex.cpp
) |
The folly submodule contains more files, yes. The ones listed there are just the files we're compiling, but we still need other files, such as all of the headers that those source files include. I don't think we should be excluding any files using Nix. |
|
Then, How should I do for #94 (comment) ? |
|
So, I think you shouldn't need to do anything for folly. We already have a submodule for it and it already builds as part of jank. There shouldn't be anything you need to do there, unless you run into errors compiling the folly which is part of jank. But I believe the error you ran into was when you were compiling a different version of folly. Correct me if I'm wrong. |
|
Ah, you are referring to a submodule that is local. I finally understand. Thanks, Jeaye!! |
When you started this PR, jank was using an unreleased version of LLVM. That's why we had to compile it from source. I told you not to worry about it, since the issues would show up as hard crashes at runtime but we should focus on getting things compiling first. Since then, 19.1.2 has been released with the necessary fixes, so we can go forward using that. |
|
@jeaye Can I close the pull request once I have found it very difficult to write up the |
|
How about I remove vcpkg (and add submodules) and the build-pch script? That should make things easier for you.
…On October 29, 2024 12:20:31 AM PDT, haruki7049 ***@***.***> wrote:
@jeaye Can I close the pull request once I have found it very difficult to write up the `default.nix` for this project? Instead, I am going to add some more `shell.nix`.
--
Reply to this email directly or view it on GitHub:
#94 (comment)
You are receiving this because you were mentioned.
Message ID: ***@***.***>
|
|
If those could be done, it would be quite easy.... But are they feasible? |
|
Yeah. Give me a few days and I'll report back.
…On October 29, 2024 4:29:27 PM PDT, haruki7049 ***@***.***> wrote:
If those could be done, it would be quite easy.... But are they feasible?
--
Reply to this email directly or view it on GitHub:
#94 (comment)
You are receiving this because you were mentioned.
Message ID: ***@***.***>
|
I've moved the incremental PCH compilation out of the script and into cmake as a custom target in 319f0b2 Hopefully that helps. I'll work on removing vcpkg next. That'll take longer. |
|
Commit hash: 3f60ab5 This error is generated. What is this?
|
The error happens when building folly. From what I gather, the issue is because you're using libc++ instead of libstdc++. We can see that in the errors here: jank needs to be built with libstdc++. Generally, clang will use this by default on Linux, but Nix must be doing something weird. |
|
vcpkg is gone: #122 We rely on boost and readline globally, while the rest has been brought in via submodules. |
haruki7049
force-pushed
the
feat/setup-nix-build-compiler
branch
from
ed864ba to
c221ce9
Compare
This comment was marked as outdated.
This comment was marked as outdated.
|
@jeaye Commit hash: c221ce9 System: $ nix-shell -p nix-info --run "nix-info -m"
- system: `"aarch64-linux"`
- host os: `Linux 6.6.31, NixOS, 24.11 (Vicuna), 24.11pre703931.4aa36568d413`
- multi-user?: `yes`
- sandbox: `yes`
- version: `nix-env (Nix) 2.24.10`
- channels(root): `"nixos, nixos-hardware"`
- nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos` |
Looks like you may not be on the latest main. Your compilation line there is missing the include path to |
|
Hey @jeaye, Are you know this error's meaning? https://gist.github.com/haruki7049/b04d1772c570f6019bd02808a0fc3647#file-err-log Commit hash: 3e3a3bd |
The error is saying that the incremental PCH we're loading was compiled with different flags than the jank binary. So when the jank binary tries to load the incremental PCH at runtime, clang raises this error. I saw this before and I think I fixed it by ensuring all the same compilation flags are used for both jank and the incremental.pch. Though, I think I've only seen it when using a prebuilt clang/llvm for building jank. I've never been able to fully get that working, for one reason or another. |
|
If that incremental.pch is the only thing getting in the way, perhaps we can try without it. There would be a severe performance penalty for startup time, though. That's why I'm hopeful that we can get it loaded normally. |
@haruki7049 following up to say the |
TODO |
|
Hey @jeaye !! Now the |
@haruki7049! Happy new year! Very odd to see those not linked. All of the Firstly, check if those shared objects are in your If not, try this:
You should be able to see cmake compiling fmt and libzippp, too. |
|
@haruki7049! We just merged a new So excited to get jank into nixpkgs. :D |
Adding Nix-build support for
/compiler+runtimedirectory.