Skip to content

Commit

Permalink
publish to huggingface and modelscope
Browse files Browse the repository at this point in the history
  • Loading branch information
csukuangfj committed Oct 10, 2024
1 parent 246f257 commit 1c46a56
Show file tree
Hide file tree
Showing 10 changed files with 178 additions and 8 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/wasm-simd-hf-space-de-tts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: Install emsdk
uses: mymindstorm/setup-emsdk@v14
with:
version: 3.1.51
version: 3.1.53
actions-cache-folder: 'emsdk-cache'

- name: View emsdk version
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/wasm-simd-hf-space-en-asr-zipformer.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
- name: Install emsdk
uses: mymindstorm/setup-emsdk@v14
with:
version: 3.1.51
version: 3.1.53
actions-cache-folder: 'emsdk-cache'

- name: View emsdk version
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/wasm-simd-hf-space-en-tts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: Install emsdk
uses: mymindstorm/setup-emsdk@v14
with:
version: 3.1.51
version: 3.1.53
actions-cache-folder: 'emsdk-cache'

- name: View emsdk version
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/wasm-simd-hf-space-silero-vad.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: Install emsdk
uses: mymindstorm/setup-emsdk@v14
with:
version: 3.1.51
version: 3.1.53
actions-cache-folder: 'emsdk-cache'

- name: View emsdk version
Expand Down
167 changes: 167 additions & 0 deletions .github/workflows/wasm-simd-hf-space-speaker-diarization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
name: wasm-simd-hf-space-speaker-diarization

on:
push:
branches:
- wasm
- wasm-speaker-diarization
tags:
- 'v[0-9]+.[0-9]+.[0-9]+*'

workflow_dispatch:

concurrency:
group: wasm-simd-hf-space-speaker-diarization-${{ github.ref }}
cancel-in-progress: true

jobs:
wasm-simd-hf-space-speaker-diarization:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Install emsdk
uses: mymindstorm/setup-emsdk@v14
with:
version: 3.1.53
actions-cache-folder: 'emsdk-cache'

- name: View emsdk version
shell: bash
run: |
emcc -v
echo "--------------------"
emcc --check
- name: Download model files
shell: bash
run: |
cd wasm/speaker-diarization/assets/
ls -lh
echo "----------"
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/speaker-segmentation-models/sherpa-onnx-pyannote-segmentation-3-0.tar.bz2
tar xvf sherpa-onnx-pyannote-segmentation-3-0.tar.bz2
rm sherpa-onnx-pyannote-segmentation-3-0.tar.bz2
mv sherpa-onnx-pyannote-segmentation-3-0/model.onnx ./segmentation.onnx
rm -rf sherpa-onnx-pyannote-segmentation-3-0
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/speaker-recongition-models/3dspeaker_speech_eres2net_base_sv_zh-cn_3dspeaker_16k.onnx
mv 3dspeaker_speech_eres2net_base_sv_zh-cn_3dspeaker_16k.onnx ./embedding.onnx
echo "----------"
ls -lh
- name: Build sherpa-onnx for WebAssembly
shell: bash
run: |
./build-wasm-simd-speaker-diarization.sh
- name: collect files
shell: bash
run: |
SHERPA_ONNX_VERSION=v$(grep "SHERPA_ONNX_VERSION" ./CMakeLists.txt | cut -d " " -f 2 | cut -d '"' -f 2)
dst=sherpa-onnx-wasm-simd-${SHERPA_ONNX_VERSION}-speaker-diarization
mv build-wasm-simd-speaker-diarization/install/bin/wasm/speaker-diarization $dst
ls -lh $dst
tar cjfv $dst.tar.bz2 ./$dst
- name: Upload wasm files
uses: actions/upload-artifact@v4
with:
name: sherpa-onnx-wasm-simd-speaker-diarization
path: ./sherpa-onnx-wasm-simd-*.tar.bz2

- name: Release
if: (github.repository_owner == 'csukuangfj' || github.repository_owner == 'k2-fsa') && github.event_name == 'push' && contains(github.ref, 'refs/tags/')
uses: svenstaro/upload-release-action@v2
with:
file_glob: true
overwrite: true
file: ./*.tar.bz2

- name: Publish to ModelScope
# if: false
env:
MS_TOKEN: ${{ secrets.MODEL_SCOPE_GIT_TOKEN }}
uses: nick-fields/retry@v2
with:
max_attempts: 20
timeout_seconds: 200
shell: bash
command: |
SHERPA_ONNX_VERSION=v$(grep "SHERPA_ONNX_VERSION" ./CMakeLists.txt | cut -d " " -f 2 | cut -d '"' -f 2)
git config --global user.email "[email protected]"
git config --global user.name "Fangjun Kuang"
rm -rf ms
export GIT_LFS_SKIP_SMUDGE=1
export GIT_CLONE_PROTECTION_ACTIVE=false
git clone https://www.modelscope.cn/studios/csukuangfj/web-assembly-speaker-diarization-sherpa-onnx.git ms
cd ms
rm -fv *.js
rm -fv *.data
git fetch
git pull
git merge -m "merge remote" --ff origin main
cp -v ../sherpa-onnx-wasm-simd-${SHERPA_ONNX_VERSION}-*/* .
git status
git lfs track "*.data"
git lfs track "*.wasm"
ls -lh
git add .
git commit -m "update model"
git push https://oauth2:${MS_TOKEN}@www.modelscope.cn/studios/csukuangfj/web-assembly-speaker-diarization-sherpa-onnx.git
- name: Publish to huggingface
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
uses: nick-fields/retry@v2
with:
max_attempts: 20
timeout_seconds: 200
shell: bash
command: |
SHERPA_ONNX_VERSION=v$(grep "SHERPA_ONNX_VERSION" ./CMakeLists.txt | cut -d " " -f 2 | cut -d '"' -f 2)
git config --global user.email "[email protected]"
git config --global user.name "Fangjun Kuang"
rm -rf huggingface
export GIT_LFS_SKIP_SMUDGE=1
export GIT_CLONE_PROTECTION_ACTIVE=false
git clone https://csukuangfj:[email protected]/spaces/k2-fsa/web-assembly-speaker-diarization-sherpa-onnx huggingface
ls -lh
cd huggingface
rm -fv *.js
rm -fv *.data
git fetch
git pull
git merge -m "merge remote" --ff origin main
cp -v ../sherpa-onnx-wasm-simd-${SHERPA_ONNX_VERSION}-*/* .
git status
git lfs track "*.data"
git lfs track "*.wasm"
ls -lh
git add .
git commit -m "update model"
git push https://csukuangfj:[email protected]/spaces/k2-fsa/web-assembly-speaker-diarization-sherpa-onnx main
2 changes: 1 addition & 1 deletion .github/workflows/wasm-simd-hf-space-vad-asr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
- name: Install emsdk
uses: mymindstorm/setup-emsdk@v14
with:
version: 3.1.51
version: 3.1.53
actions-cache-folder: 'emsdk-cache'

- name: View emsdk version
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: Install emsdk
uses: mymindstorm/setup-emsdk@v14
with:
version: 3.1.51
version: 3.1.53
actions-cache-folder: 'emsdk-cache'

- name: View emsdk version
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: Install emsdk
uses: mymindstorm/setup-emsdk@v14
with:
version: 3.1.51
version: 3.1.53
actions-cache-folder: 'emsdk-cache'

- name: View emsdk version
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: Install emsdk
uses: mymindstorm/setup-emsdk@v14
with:
version: 3.1.51
version: 3.1.53
actions-cache-folder: 'emsdk-cache'

- name: View emsdk version
Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ We also have spaces built using WebAssembly. They are listed below:
|VAD + speech recognition (English + Chinese, 及多种中文方言) with Paraformer-small |[Click me][wasm-hf-vad-asr-zh-en-paraformer-small]| [地址][wasm-ms-vad-asr-zh-en-paraformer-small]|
|Speech synthesis (English) |[Click me][wasm-hf-tts-piper-en]| [地址][wasm-ms-tts-piper-en]|
|Speech synthesis (German) |[Click me][wasm-hf-tts-piper-de]| [地址][wasm-ms-tts-piper-de]|
|Speaker diarization |[Click me][wasm-hf-speaker-diarization]|[地址][wasm-ms-speaker-diarization]|

### Links for pre-built Android APKs

Expand Down Expand Up @@ -262,6 +263,8 @@ Video demo in Chinese: [爆了!炫神教你开打字挂!真正影响胜率
[wasm-ms-tts-piper-en]: https://modelscope.cn/studios/k2-fsa/web-assembly-tts-sherpa-onnx-en
[wasm-hf-tts-piper-de]: https://huggingface.co/spaces/k2-fsa/web-assembly-tts-sherpa-onnx-de
[wasm-ms-tts-piper-de]: https://modelscope.cn/studios/k2-fsa/web-assembly-tts-sherpa-onnx-de
[wasm-hf-speaker-diarization]: https://huggingface.co/spaces/k2-fsa/web-assembly-speaker-diarization-sherpa-onnx
[wasm-ms-speaker-diarization]: https://www.modelscope.cn/studios/csukuangfj/web-assembly-speaker-diarization-sherpa-onnx
[apk-streaming-asr]: https://k2-fsa.github.io/sherpa/onnx/android/apk.html
[apk-streaming-asr-cn]: https://k2-fsa.github.io/sherpa/onnx/android/apk-cn.html
[apk-tts]: https://k2-fsa.github.io/sherpa/onnx/tts/apk-engine.html
Expand Down

0 comments on commit 1c46a56

Please sign in to comment.