Migrate to all-in-one CMake-based build configuration

[zhihaoy]

I started a thread in the mailing list at http://jsoftware.com/pipermail/source/2020-November/001353.html.

If accepted, this PR will

• build j, jconsole, jnative, and tsdll in Release and Debug
• run unit tests

It is tested on Windows and Linux, inside and outside Visual Studio 2019, with Clang and GCC.
It should work on Mac as well.

How to use:

I suggest check out my cmake-new branch and try the following. Because when the change is applied on top of master, jconsole crashes on Windows when exiting, which implies an issue in master.

Visual Studio 2019

To build, select "Open" -> "CMake..." or "Folder..." from the menu in Visual Studio 2019.
To debug, select "jconsole.exe (jsrc\jconsole)" as the debug target and run.
To run unit tests, "Test" -> "Run CTests for j."

Linux

Please install cmake >= 3.17 and ninja. If you cannot find a recent software repository, you can open an Anaconda environment and install them there. And then, execute

cmake -G "Ninja Multi-Config" -B build

under the jsource root directory. The last argument, "build," is for specifying the build directory. It can be anything, but I will use "build" for the rest of the article.

The above command generates a Ninja multi-configuration build. Now running

ninja -C build

will build j, jconsole in Debug, and prepare for unit tests. If we found JDK, we will also build jnative.

The executable is now usable with

./build/jsrc/Debug/jconsole

It loads jlibrary under your jsource repository.

T run unit tests against the Debug executable and libraries:

ninja -C build test

Do the Release build:

ninja -C build -f build-Release.ninja

Run unit tests against the Release executable and libraries:

ninja -C build -f build-Release.ninja test

If you find this command being too verbose, you can use the ctest tool:

cd build
ctest -C Release

Note that the Release artifacts and Debug artifacts can coexist. The Release executable is at

./build/jsrc/Release/jconsole

under the jsource root directory.

If you made changes to any source file, running ninja will do an incremental build. If you made changes to CMakeLists.txt, running ninja will incrementally reconfigure. If you totally messed up, rm -fr build.

You can switch compilers to Clang with

cmake -B build -G "Ninja Multi-Config" -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++

Developer Command Prompt for VS 2019

If clang-cl is in your PATH, you can specify the compilers with

cmake -B build -G "Ninja Multi-Config" -DCMAKE_C_COMPILER=clang-cl -DCMAKE_CXX_COMPILER=clang-cl

Discussion

Rationale, details, plan, questions are all in the email I sent to the mailing list. Here is a quick summary I brought up for discussion:

1. Should we support only avx2 with emulation on x64?
2. Can we drop 32-bit builds on all architectures?
3. Should we support only >= arm64-v8a on ARM?
4. Are we okay with always building with OpenMP if available?
5. Are we okay with always turning on READLINE?
6. Do we agree with my plan to get rid of all the other existing build configurations after we finished working on a result of 1-5?

[zhihaoy]

Ping. Please give us a timeline when this change is desired.

Here is some users' story with this PR.

An NVIDIA engineer who works on CUDA libraries for C++, also a fan of APL and J, decided to start a LiveSteam working on converting the J codebase to C++20. He was able to figure out how the CMake-based build works without prior-knowledge of CMake or Ninja (and didn't read my PR comment either, and wasted some time because of that). In the end, he recreated his jsource repo by branching off my PR rather than jsoftware upstream 🤷

I did not get involved in those live streams and was aware of that only after YouTube pushed me a recommendation.

Now that repository has 11 contributors and nobody seemed to complain about the build system. They iterate on it, improved how unit tests run, and even added documentation generation as a part of the build process.

[eiverson]

Zhihaoy, The idea of moving to cmake is interesting. But perhaps not of immediate general interest to jsource users. I will get back to you directly (outside the forum) to see if there is a way forward that is of benefit to everyone. /Eric

…

On Mon, Feb 8, 2021 at 1:03 PM zhihaoy ***@***.***> wrote: Ping. Please give us a timeline when this change is desired. Here is some users' story with this PR. An NVIDIA engineer who works on CUDA libraries for C++, also a fan of APL and J, decided to start a LiveSteam <https://www.youtube.com/playlist?list=PLVFrD1dmDdvfVhYLU_iKkV67X9XqCJLWe> working on converting the J codebase to C++20. He was able to figure out <https://youtu.be/afico3BW-Is> how the CMake-based build works without prior-knowledge of CMake or Ninja (and didn't read my PR comment either, and wasted some time because of that). In the end, he recreated his jsource repo <https://github.com/codereport/jsource> by branching off my PR rather than jsoftware upstream 🤷 I did not get involved in those live streams and was aware of that only after YouTube pushed me a recommendation. Now that repository has 11 contributors and nobody seemed to complain about the build system. They iterate on it, improved how unit tests run, and even added documentation generation as a part of the build process. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#20 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEBRCHWXBGRWVKUSBVXU6WLS6ARPJANCNFSM4TQ5XJXA> .

[zhihaoy]

Rebased on top of master, and switch to the official OpenMP .cmake script.

I'm now maintaining jruntime, a Python shared object module that allows you to run J code from Python, using Python and NumPy datatypes. It distributed in binary on pypi and Anaconda. The source code is at https://github.com/zhihaoy/jsource/blob/jruntime-pyd/jsrc/jruntime.cc. CMake is being used for building this extension: https://github.com/zhihaoy/jsource/blob/jruntime-pyd/jsrc/CMakeLists.txt#L277-L289