Initial commit

Author: TerrorJack

.github/workflows/build.yml

@@ -0,0 +1,66 @@
+name: build
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+  merge_group:
+  workflow_dispatch:
+
+jobs:
+  build:
+    name: build
+    runs-on: ubuntu-24.04
+    permissions:
+      pages: write
+      id-token: write
+    steps:
+      - name: ghc-wasm-meta
+        run: |
+          pushd "$(mktemp -d)"
+          curl -L https://gitlab.haskell.org/haskell-wasm/ghc-wasm-meta/-/archive/master/ghc-wasm-meta-master.tar.gz | tar xz --strip-components=1
+          PREFIX=/tmp/.ghc-wasm SKIP_GHC=1 ./setup.sh
+          /tmp/.ghc-wasm/add_to_github_path.sh
+          echo "WASI_SDK_PREFIX=/tmp/.ghc-wasm/wasi-sdk" >> $GITHUB_ENV
+          popd
+
+      - name: build-zstd
+        run: |
+          pushd "$(mktemp -d)"
+          curl -L https://github.com/haskell-wasm/zstd/archive/refs/heads/release.tar.gz | tar xz --strip-components=1
+          ./install-wasi.sh
+          popd
+
+      - name: build-libarchive
+        run: |
+          pushd "$(mktemp -d)"
+          curl -L https://github.com/haskell-wasm/libarchive/archive/refs/heads/patch/3.8.tar.gz | tar xz --strip-components=1
+          ./install-wasi.sh
+          popd
+
+      - name: test-playground
+        run: |
+          mkdir ghc-in-browser
+          curl -L https://github.com/haskell-wasm/ghc-in-browser/archive/refs/heads/gh-pages.tar.gz | tar xz --strip-components=1 -C ghc-in-browser
+          pushd ghc-in-browser
+
+          cp /tmp/wasi/bin/bsdtar.wasm .
+          sed -i -e "s|https://haskell-wasm.github.io/bsdtar-wasm/bsdtar.wasm|./bsdtar.wasm|" index.html
+          curl -L https://gitlab.haskell.org/haskell-wasm/ghc/-/raw/ghc-9.14/.gitlab/hello.hs -O
+          curl -L https://gitlab.haskell.org/haskell-wasm/ghc/-/raw/ghc-9.14/testsuite/tests/ghc-api-browser/playground001.js -o test.js
+          sed -i -e 's|".mjs": "application/javascript",|".mjs": "application/javascript", ".wasm": "application/wasm"|' test.js
+
+          chmod +x test.js
+          ./test.js '{"browser":"chrome","executablePath":"/usr/bin/google-chrome-stable","args":["--no-sandbox"]}' | grep -F 'main = putStrLn "hello world"'
+          popd
+
+      - name: upload-pages-artifact
+        uses: actions/upload-pages-artifact@v4
+        with:
+          path: /tmp/wasi/bin
+          retention-days: 90
+
+      - name: deploy-pages
+        uses: actions/deploy-pages@v4
+        if: github.event_name == 'push' && github.ref_name == 'master'

LICENSE

@@ -0,0 +1,23 @@
+Copyright (c) 2025 Cheng Shao
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+1. Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer
+   in this position and unchanged.
+2. Redistributions in binary form must reproduce the above copyright
+   notice, this list of conditions and the following disclaimer in the
+   documentation and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR
+IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT,
+INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

@@ -0,0 +1,59 @@
+# `bsdtar-wasm`
+
+[![Chat on Matrix](https://matrix.to/img/matrix-badge.svg)](https://matrix.to/#/#haskell.wasm:matrix.org)
+
+This repo hosts a `wasm32-wasi` build of the `bsdtar` CLI from the
+[libarchive](https://libarchive.org) project, with
+[`zstd`](https://facebook.github.io/zstd) support. It can be used to
+extract a `.tar.zst` archive in the wasi vfs, useful when:
+
+- You need to download a large binary artifact, and the default
+  gzip/brotli compression provided by the CDN is not good enough
+- You need to set up a vfs with many files, and issuing one `fetch`
+  per file can become a bottleneck
+
+## How to use
+
+### The wasm modules
+
+There are two `wasm32-wasi` modules available here:
+
+- [`bsdtar.wasm`](https://haskell-wasm.github.io/bsdtar-wasm/bsdtar.wasm)
+- [`zstd.wasm`](https://haskell-wasm.github.io/bsdtar-wasm/zstd.wasm)
+
+They are command modules without any custom imports/exports, fully
+single-threaded, compiled with LLVM ThinLTO and optimized by
+`wasm-opt`, with `simd128` feature enabled. `bsdtar.wasm` only
+supports `zstd`, doesn't support other filters like `xz` or `gzip`.
+
+Simply run them and pass the right CLI arguments as well as the right
+input (either stdin or a file in the wasi vfs), then check the exit
+code and use the output.
+
+Prefer `zstd.wasm` when only decompressing a single file, otherwise
+use `bsdtar.wasm`.
+
+### The wasi implementation
+
+The wasm modules work fine with `wasmtime` and `node`/`uvwasi`. But
+for browsers, the only wasi implementation that works right now is
+[`haskell-wasm/browser_wasi_shim`](https://github.com/haskell-wasm/browser_wasi_shim),
+which can be directly imported from this
+[URL](https://esm.sh/gh/haskell-wasm/browser_wasi_shim). Upstream npm
+release lacks some important bugfixes at the moment.
+
+## Building
+
+The wasm-enabled forks of the relevant projects are available at:
+
+- [`haskell-wasm/zstd`](https://github.com/haskell-wasm/zstd)
+- [`haskell-wasm/libarchive`](https://github.com/haskell-wasm/libarchive)
+
+Start from `install-wasi.sh` in the above repos.
+
+The wasm patches are based on upstream release branches instead of
+development branches, and there's currently no plan to upstream them
+due to limited time. That being said, the heavylifting is done by
+upstream and the wasm patches are just bandaids to get things working
+for my use-case; feel free to use them as starting points for building
+whatever you have in mind.