add github action.

.github/workflows/linux-cpu.yml

@@ -0,0 +1,28 @@
+name: linux-cpu
+on:
+  push:
+    branches:
+     - master
+     - 'feature/**'
+    paths:
+      - 'src/**'
+      - '.github/workflows/linux-cpu.yml'
+  pull_request:
+    branches: [master]
+    paths:
+      - 'src/**'
+      - '.github/workflows/linux-cpu.yml'
+
+concurrency:
+  group: linux-${{ github.ref }}
+  cancel-in-progress: true
+permissions:
+  contents: read
+
+jobs:
+  linux_buil_test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: build
+        run: ./script/linux_build.sh

.github/workflows/macos-cpu.yml

@@ -0,0 +1,28 @@
+name: macos-cpu
+on:
+  push:
+    branches:
+     - master
+     - 'feature/**'
+    paths:
+      - 'src/**'
+      - '.github/workflows/macos-cpu.yml'
+  pull_request:
+    branches: [master]
+    paths:
+      - 'src/**'
+      - '.github/workflows/macos-cpu.yml'
+
+concurrency:
+  group: macos-${{ github.ref }}
+  cancel-in-progress: true
+permissions:
+  contents: read
+
+jobs:
+  macos_buil_test:
+    runs-on: macos-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: build
+        run: ./script/macos_build.sh

.github/workflows/windows-cpu.yml

@@ -0,0 +1,29 @@
+name: windows-cpu
+on:
+  push:
+    branches:
+     - master
+     - 'feature/**'
+    paths:
+      - 'src/**'
+      - '.github/workflows/windows-cpu.yml'
+  pull_request:
+    branches: [master]
+    paths:
+      - 'src/**'
+      - '.github/workflows/windows-cpu.yml'
+
+concurrency:
+  group: windows-${{ github.ref }}
+  cancel-in-progress: true
+permissions:
+  contents: read
+
+jobs:
+  windows_build_test:
+    runs-on: windows-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: ilammy/msvc-dev-cmd@v1
+      - name: build
+        run: powershell .\script\windows_build.ps1

README.md

@@ -14,151 +14,44 @@
 | Baichuan2-7B-Chat | [onnx](https://github.com/wangzhaode/llm-export/releases/tag/baichuan2-7b-chat-onnx) | [mnn](https://github.com/wangzhaode/mnn-llm/releases/tag/untagged-6798382d6309a35e20d0) |
 
 
-## 用法
-### 0. 模型导出与转换
-可以使用[LLMExporter](https://github.com/wangzhaode/LLMExporter)将模型导出为`onnx`格式，然后使用`mnnconvert`转换为`mnn`模型。
+## 构建
 
-### 1. 下载本项目
-```bash
-git clone https://github.com/wangzhaode/mnn-llm.git
-```
-### 2. 编译MNN库
-- 克隆MNN项目，最新正式版是2.5.0
-```bash
-git clone https://github.com/alibaba/MNN.git -b 2.5.0
-```
-
-- 进入MNN项目, 并构建一个Build目录准备编译
-```bash
-cd MNN
-mkdir build && cd build
-```
-
-- 正式编译，可选CPU/CUDA/OpenCL三种，推荐有英伟达显卡的选择CUDA，没显卡的选CPU,有AMD显卡的选择OpenCL
-
-```bash
-# CPU only（Suport Linux/Mac/Windows）
-cmake -DCMAKE_BUILD_TYPE=Release ..
-
-# using CUDA(Support Linux)
-cmake -DCMAKE_BUILD_TYPE=Release -DMNN_CUDA=ON ..
-
-# using OPENCL
-cmake -DCMAKE_BUILD_TYPE=Release -DMNN_OPENCL=ON -DMNN_USE_SYSTEM_LIB=ON -DMNN_SEP_BUILD=OFF ..
-
-# start build(support Linux/Mac)
-make -j$(nproc)
-
-# start build(support Windows)
-cmake --build . -- /m:8
-
-```
-
-- 回到mnn-llm
-
-```bash
-cd ../..
-```
-
-- 将MNN库的编译结果拷贝到`mnn-llm/libs`目录下
-```bash
-# for Linux/Mac
-cp -r MNN/include/MNN include
-cp MNN/build/libMNN.so libs/
-cp MNN/build/express/*.so  libs/
-
-# for windows
-cp -r MNN/include/MNN include
-cp MNN/build/Debug/MNN.dll libs/
-cp MNN/build/Debug/MNN.lib libs/
+### CPU-Only
 ```
+# linux
+./script/linux_build.sh
 
-- 对于Windows，还需要下载一下第三方库pthread，下载[地址](https://gigenet.dl.sourceforge.net/project/pthreads4w/pthreads-w32-2-9-1-release.zip),下载后解压，打开Pre-built.2\lib\x64， 将pthreadVC2.lib文件拷贝到ChatGLM-MNN的libs文件夹。打开Pre-built.2\include,将下面三个.h文件都放到ChatGLM-MNN的include文件夹。对于windows，项目的最终文件结构如下：
-```bash
-├───libs
-│   ├───MNN.dll
-│   ├───MNN.lib
-│   └───pthreadVC2.lib
-├───include
-│   ├───cppjieba
-│   ├───limonp
-│   ├───MNN
-│   ├───chat.hpp
-│   ├───httplib.h
-│   ├───pthread.h
-│   ├───sched.h
-│   └───semaphore.h
-```
-
-
-### 3. Download Models
-从 `github release` 下载模型文件到 `/path/to/ChatGLM-MNN/resource/models`， 如下：
-- 对于Linux/Mac
-```bash
-cd resource/models
-# 下载fp16权值模型, 几乎没有精度损失
-./download_models.sh fp16
-# 对于中国用户，可以使用第三方服务加速下载fp16模型
-./download_models.sh fp16 proxy
-
-# 下载int8权值模型，极少精度损失，推荐使用
-./download_models.sh int8
-# 对于中国用户，可以使用第三方服务加速下载int8模型
-./download_models.sh int8 proxy
+# macos
+./script/macos_build.sh
 
-# 下载int4权值模型，有一定精度损失
-./download_models.sh int4 
-# 对于中国用户，可以使用第三方服务加速下载int4模型
-./download_models.sh int4 proxy
-```
-- 对于windows,将上面的`xxx.sh`替换为`xxx.ps1`文件即可，例如：
-```powershell
-cd resource/models
+# windows msvc
+./script/windows_build.ps1
 
-# 下载fp16权值模型, 几乎没有精度损失
-./download_models.ps1 fp16
-# 对于中国用户，可以使用第三方服务加速下载fp16模型
-./download_models.ps1 fp16 proxy
+# android
+./script/android_build.sh
 ```
+### CUDA/OPENCL
+`TODO`
 
-### 4. Build and Run
+### 4. 执行
 
-##### Mac/Linux/Windows:
 ```bash
-mkdir build && cd build
-# for CPU
-cmake ..
-# for GPU
-cmake -D WITH_CUDA=on ..
-# for mini memory device
-cmake -D BUILD_MINI_MEM_MODE=on ..
-
-# start build(support Linux/Mac)
-make -j$(nproc)
-
-# start build(support Windows)
-cmake --build . -- /m:8
-
-# run (for Linux/Mac)
+# linux/macos
 ./cli_demo # cli demo
 ./web_demo # web ui demo
 
-# run (for Windows)
+# windows
 .\Debug\cli_demo.exe
 .\Debug\web_demo.exe
 ```
 
 
-##### Android:
-```
-mkdir build
-cd build
-../android_build.sh
-make -j8
-```
-
 ## Reference
-- [ChatGLM-6B](https://huggingface.co/THUDM/chatglm-6b)
+- [chatglm-6b](https://huggingface.co/THUDM/chatglm-6b)
+- [chatglm2-6b](https://huggingface.co/THUDM/chatglm2-6b)
+- [codegeex2-6b](https://huggingface.co/THUDM/codegeex2-6b)
+- [Baichuan2-7B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat)
+- [Qwen-7B-Chat](https://huggingface.co/tangger/Qwen-7B-Chat)
 - [cpp-httplib](https://github.com/yhirose/cpp-httplib)
 - [chatgpt-web](https://github.com/xqdoo00o/chatgpt-web)
 - [cppjieba](https://github.com/yanyiwu/cppjieba)

script/android_build.sh

@@ -0,0 +1,26 @@
+# 1. clone MNN
+git clone https://github.com/alibaba/MNN.git -b 2.6.3 --depth=1
+
+# 2. build MNN
+cd MNN/project/android
+mkdir build
+cd build
+../build_64.sh
+cd ../../../..
+
+# 3. copy headers and libs
+cp -r MNN/include/MNN include
+cp MNN/project/android/build/libMNN.so MNN/project/android/build/libMNN_Express.so libs
+
+# 4. build mnn-llm android
+mkdir android_build
+cd android_build
+cmake .. \
+-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
+-DANDROID_STL=c++_static \
+-DANDROID_ABI="arm64-v8a" \
+-DANDROID_NATIVE_API_LEVEL=android-21  \
+-DCMAKE_BUILD_TYPE=Release \
+-DBUILD_FOR_ANDROID=ON
+make -j4
+cd ..

script/download_model.sh

@@ -0,0 +1,11 @@
+model=$1
+mkdir $model
+cd $model
+# download models
+wget -c https://github.com/wangzhaode/mnn-llm/releases/download/$model-mnn/embedding.mnn
+wget -c https://github.com/wangzhaode/mnn-llm/releases/download/$model-mnn/lm.mnn
+for i in `seq 0 27`
+do
+    wget -c https://github.com/wangzhaode/mnn-llm/releases/download/$model-mnn/block_$i.mnn
+done
+cd ..

script/linux_build.sh

@@ -0,0 +1,21 @@
+# 1. clone MNN
+git clone https://github.com/alibaba/MNN.git -b 2.6.3 --depth=1
+
+# 2. build MNN
+cd MNN
+mkdir build
+cd build
+cmake -DMNN_LOW_MEMORY=ON ..
+make -j4
+cd ../..
+
+# 3. copy headers and libs
+cp -r MNN/include/MNN include
+cp MNN/build/libMNN.so MNN/build/express/libMNN_Express.so libs
+
+# 4. build mnn-llm
+mkdir build
+cd build
+cmake ..
+make -j4
+cd ..

script/macos_build.sh

@@ -0,0 +1,21 @@
+# 1. clone MNN
+git clone https://github.com/alibaba/MNN.git -b 2.6.3 --depth=1
+
+# 2. build MNN
+cd MNN
+mkdir build
+cd build
+cmake -DMNN_LOW_MEMORY=ON ..
+make -j4
+cd ../..
+
+# 3. copy headers and libs
+cp -r MNN/include/MNN include
+cp MNN/build/libMNN.dylib MNN/build/express/libMNN_Express.dylib libs
+
+# 4. build mnn-llm
+mkdir build
+cd build
+cmake ..
+make -j4
+cd ..

script/windows_build.ps1

@@ -0,0 +1,28 @@
+# 1. clone MNN
+git clone https://github.com/alibaba/MNN.git -b 2.6.3 --depth=1
+
+# 2. build MNN
+cd MNN
+mkdir build
+cd build
+# cmake -DMNN_LOW_MEMORY=ON ..
+cmake ..
+cmake --build . -j 4
+cd ../..
+
+# 3. copy headers and libs
+cp -r MNN/include/MNN include
+mv MNN/build/libMNN.lib libs
+
+# 4. download pthread
+wget -Uri https://gigenet.dl.sourceforge.net/project/pthreads4w/pthreads-w32-2-9-1-release.zip -OutFile "pthreads.zip"
+Expand-Archive .\pthreads.zip
+mv .\pthreads\Pre-built.2\lib\x64\pthreadVC2.lib libs
+cp .\pthreads\Pre-built.2\include\*.h .\include\
+
+# 5. build mnn-llm
+mkdir build
+cd build
+cmake ..
+cmake --build .  -j 4
+cd ..