Skip to content

[P0] Verify the clang++ version used in the test #18 #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yuqisun wants to merge 22 commits into
base: main
Choose a base branch
from
Open

[P0] Verify the clang++ version used in the test #18 #19

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
85125f1
[P0] Verify the clang++ version used in the test #18
yuqisun May 22, 2025
8b3dfd9
Update main.yml
yuqisun May 23, 2025
09b8363
Merge branch 'coredac:main' into 18-p0-verify-the-clang-version-used-…
yuqisun May 23, 2025
98044c2
Test with apt install llvm instead of source code.
yuqisun May 23, 2025
7b872b6
Revert to install from source code, install from apt doesn't include …
yuqisun May 23, 2025
8a4e8b5
Upgrade to llvm-19.
yuqisun May 23, 2025
b4ab42a
Clone the latest llvm repo code only to speed up.
yuqisun May 23, 2025
e4e18e1
Specify branch for cloning LLVM repository in workflow
yuqisun May 23, 2025
9e43f7b
Add Dockerfile for automated build and test environment setup
yuqisun May 23, 2025
2bb49ff
Update LLVM checkout to specific commit cd70802 in Dockerfile and wor…
yuqisun May 24, 2025
5e0e577
Fix checkout order for LLVM setup in workflow
yuqisun May 24, 2025
24b88ce
Clone includes history commit tree.
yuqisun May 24, 2025
fd95c0e
Update LLVM checkout to full commit hash in Dockerfile and workflow
yuqisun May 24, 2025
61949a5
Shallow clone LLVM source code for faster checkout.
yuqisun May 24, 2025
db3a1af
Refine LLVM build instructions in README.md for clarity and stability
yuqisun May 24, 2025
accef66
Update Dockerfile and workflow to install additional dependencies and…
yuqisun May 26, 2025
6d3dd27
DCMAKE_EXPORT_COMPILE_COMMANDS=ON for VS Code users (to avoid warning…
yuqisun May 26, 2025
b934374
Merge remote-tracking branch 'origin/main' into 18-p0-verify-the-clan…
yuqisun May 26, 2025
389e3d1
Move docker path.
yuqisun May 26, 2025
6b56c40
Refactor LLVM build setup by removing clang installation and adjustin…
yuqisun May 28, 2025
2f06d19
Check clang version instead of installing it in the workflow
yuqisun May 28, 2025
6da365e
Update README and test documentation for clang version consistency
yuqisun May 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
FROM ubuntu:22.04

SHELL ["/bin/bash", "-c"]

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update \
&& apt install software-properties-common -y \
&& add-apt-repository ppa:deadsnakes/ppa -y \
&& apt update \
&& apt install python3.12 -y \
&& apt-get install ninja-build -y \
&& apt-get install ccache -y \
&& apt-get install lld -y \
&& apt-get -y install cmake -y \
&& apt-get install -y clang \
&& apt-get -y install git \
&& mkdir /cgra \
&& cd /cgra \
&& git clone https://github.com/yuqisun/dataflow.git \
&& cd dataflow \
&& until git clone --depth 1 --branch release/19.x https://github.com/llvm/llvm-project.git; do sleep 5; done \
&& cd llvm-project \
&& mkdir build && cd build \
&& cmake -G Ninja ../llvm \
-DLLVM_ENABLE_PROJECTS="mlir" \
-DLLVM_BUILD_EXAMPLES=OFF \
-DLLVM_TARGETS_TO_BUILD="Native" \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DCMAKE_CXX_STANDARD=17 \
-DCMAKE_CXX_FLAGS="-std=c++17 -frtti" \
-DLLVM_ENABLE_LLD=ON \
-DMLIR_INSTALL_AGGREGATE_OBJECTS=ON \
-DLLVM_ENABLE_RTTI=ON \
-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
&& cmake --build . --target check-mlir \
&& dpkg -l | grep llvm \
&& cd /cgra/dataflow \
&& mkdir build && cd build \
&& cmake -G Ninja .. \
-DLLVM_DIR=/cgra/dataflow/llvm-project/build/lib/cmake/llvm \
-DMLIR_DIR=/cgra/dataflow/llvm-project/build/lib/cmake/mlir \
-DMLIR_SOURCE_DIR=/cgra/dataflow/llvm-project/mlir \
-DMLIR_BINARY_DIR=/cgra/dataflow/llvm-project/build \
-DCMAKE_CXX_FLAGS="-std=c++17" \
&& ninja \
&& cd /cgra/dataflow/test \
&& /cgra/dataflow/llvm-project/build/bin/llvm-lit * -v

CMD ["tail", "-f", "/dev/null"]


12 changes: 10 additions & 2 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,16 @@ jobs:
sudo apt-get install ccache
sudo apt-get install lld

# install clang for LLVM build
- name: install clang
run: sudo apt-get install -y clang
Copy link
Copy Markdown
Contributor

@tancheng tancheng May 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may already ask, but what is this clang version?

Copy link
Copy Markdown
Collaborator Author

@yuqisun yuqisun May 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default install version is 14.


# setup LLVM
- name: install a specific version of LLVM
working-directory: ${{github.workspace}}
run: |
git clone https://github.com/llvm/llvm-project.git
git clone --depth 1 --branch release/19.x https://github.com/llvm/llvm-project.git
cd llvm-project
git checkout cd70802
mkdir build && cd build
cmake -G Ninja ../llvm \
-DLLVM_ENABLE_PROJECTS="mlir" \
Expand All @@ -55,10 +58,15 @@ jobs:
-DLLVM_ENABLE_LLD=ON \
-DMLIR_INSTALL_AGGREGATE_OBJECTS=ON \
-DLLVM_ENABLE_RTTI=ON \
-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache

cmake --build . --target check-mlir
dpkg -l | grep llvm
sudo apt-get purge llvm-13
sudo apt-get purge llvm-14
sudo apt-get purge llvm-15

# setup mlir-cgra
- name: setup dataflow tool-chain
Expand Down
25 changes: 19 additions & 6 deletions README.md
View file
Open in desktop
Copy link
Copy Markdown
Contributor

@tancheng tancheng May 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also mention we prefer the clang is also in llvm-19 version?

I am wondering, when we build LLVM-19.x from source, it is compiled using the built-in clang, which may not be llvm-19, right? Then if user install clang in llvm-19, should we update this https://github.com/coredac/dataflow/blob/ff914bc44d0c2cb5589ca962ff32cbe372ea9871/test/c2llvm2mlir/test.mlir#L5 to distinguish the system clang and the user-installed clang?

Copy link
Copy Markdown
Collaborator Author

@yuqisun yuqisun May 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about now: Compile both clang and mlir and add note at the beginning of test.mlir to recommend user to add llvm-19 clang to PATH.

-DLLVM_ENABLE_PROJECTS="clang;mlir" --- To compile clang in llvm-19 but take more time.

Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,13 @@

Build LLVM & Neura
--------------------------------------------------------
- Clone llvm-project.

- Clone this repo.

- Clone llvm-project (we prefer llvm-19).
```sh
$ git clone --depth 1 --branch release/19.x https://github.com/llvm/llvm-project.git
```
- Build LLVM:
- Check out to commit `cd70802` (a stable version randomly picked, will sync to the latest version).
- Build:
```sh
$ cd llvm-project
$ mkdir build && cd build
# May need install ccache and lld.
$ cmake -G Ninja ../llvm \
Expand All @@ -25,6 +24,7 @@ Build LLVM & Neura
-DLLVM_ENABLE_LLD=ON \
-DMLIR_INSTALL_AGGREGATE_OBJECTS=ON \
-DLLVM_ENABLE_RTTI=ON \
-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
Copy link
Copy Markdown
Contributor

@tancheng tancheng May 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for?

Copy link
Copy Markdown
Collaborator Author

@yuqisun yuqisun May 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this because Ninja cannot auto fix RPATH:

CMake Error at cmake/modules/AddLLVM.cmake:1029 (add_executable):
The install of the llvm-tblgen target requires changing an RPATH from the
build tree, but this is not supported with the Ninja generator unless on an
ELF-based or XCOFF-based platform. The CMAKE_BUILD_WITH_INSTALL_RPATH
variable may be set to avoid this relinking step.
Call Stack (most recent call first):
cmake/modules/TableGen.cmake:181 (add_llvm_executable)
utils/TableGen/CMakeLists.txt:32 (add_tablegen)

Copy link
Copy Markdown
Contributor

@tancheng tancheng May 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also have DCMAKE_EXPORT_COMPILE_COMMANDS=ON for VS Code users (to avoid warning in the IDE for include paths), I just set it up with copilot.

Copy link
Copy Markdown
Collaborator Author

@yuqisun yuqisun May 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added.

-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
$ cmake --build . --target check-mlir
Expand Down Expand Up @@ -54,3 +54,16 @@ Build LLVM & Neura
$ /workspace/llvm-project/build/bin/llvm-lit *
```

Docker-based Environment Setup
--------------------------------------------------------
You can quickly set up the build and test environment using the provided Dockerfile:

```sh
# In the root directory of this repo:
$ cd docker
$ docker build -t neura:v1 .
$ docker run --name myneura -it neura:v1
```

This will automatically clone, build, and test LLVM/MLIR and Neura inside a container, ensuring a consistent environment.