Investigate Alternative JS Engine

[xu-cheng]

This crate requires an embedded JS engine to execute katex. Currently, this is done using quickjs and its rust binding. However, it has certain shortages:

• No support on native windows using MSVC. (MSVC Toolchain Support theduke/quickjs-rs#72)
• No support on WASM environment. (Support wasm32-unknown-unknown Target theduke/quickjs-rs#73)
• The development activity of quickjs is low.

As such, we may want to find an alternative JS engine. Since this crate never exposes the types from the underlying JS engine, changing it will not create any breaking changes. Ideally, the alternatives need supporting all the current used features, addressing the above problems, being small to be embedded, and being well maintained.

Some candidates:

• boa, a JS engine written in pure Rust. But currently, it is experimental with many basic JS features not being implemented.
• rusty_v8, Rust bindings to V8 engine. This engine may be too big to be embedded.
• rust-mozjs, JS engine from servo. It doesn’t seem to be well maintained. Its crate hasn’t been updated for 2 years.

[xu-cheng]

With v0.4.0 of KaTeX, we start to abstract the JS engine behind a common trait. Help is needed to add more JS engine backend in addition to the QuickJS.

[chipsenkbeil]

@xu-cheng, from the candidates you described, it doesn't seem like there's a good option as an alternative in the near future, right? The wasm issue also mentions https://github.com/theduke/slimjs, but that seems more like an experiment than anything with a future.

I was looking at incorporating your crate (chipsenkbeil/vimwiki-rs#109) to support server-side rendering alternative to MathJax.

I've also got vimwiki-wasm to enable parsing the vimwiki language (wiki dialect with similar purpose to markdown) and searching it directly in the browser. It's not a dealbreaker to me that this won't work with wasm right now, but curious what your thoughts are for the future.

[xu-cheng]

Just as I mentioned above, my thought is to provide multiple JS backends in this crate. Therefore, the users can choose the suitable backend to their needs by assigning the corresponding feature to this crate.

I am planning to add two new backends for now: rusty_v8 and wasm-binding (i.e., using the JS from the browser directly). But I don't have too much free time for now. So PR is welcome. Also, it is worth mentioning that the common JS engine trait is not fixed and can be changed when we add new JS backend.

In long term, I hope we can add boa and slimjs support.

[chipsenkbeil]

@xu-cheng, I wonder if it's possible to embed wasmer into this crate and use it to load a QuickJS wasm file. In wasmer readme's quickstart, it specifically shows loading a QuickJS wasm file and using it via the cli. If you can load that, wondering if that would get around the need for quickjs-rs support for a WASM target. I haven't read much into wasmer myself.

[l0calh05t]

@xu-cheng, I wonder if it's possible to embed wasmer into this crate and use it to load a QuickJS wasm file. In wasmer readme's quickstart, it specifically shows loading a QuickJS wasm file and using it via the cli. If you can load that, wondering if that would get around the need for quickjs-rs support for a WASM target. I haven't read much into wasmer myself.

This seems like a very viable option, considering wasmers quickstart-example runs a pre-build quickjs

[l0calh05t]

The prebuilt quickjs module is a WASM version of the quickjs CLI, so not particularly useful. Would https://duktape.org/ be worth looking into? EDIT: I doubt duktape would work, seems to support only a tiny subset of ES6

[l0calh05t]

QuickJS on wasmer is definitely viable. Here's a proof of concept: https://github.com/l0calh05t/wasmer-katex-poc/ However, it doesn't fit the current Engine-API well

[xu-cheng]

Hi, sorry for the late response. Unfortunately, there are multiple issues with quickjs wasm.

• The biggest issue is the wasm itself is not well maintained and not official project of quickjs. I also have some hesitation to distribute an unofficial (i.e., untrusted) binary file.
• As you may already find out, the quickjs wasm is designed to be a CLI rather than a library. This wouldn't work well with this library, especially we want to only initiate katex js once.

I still think the best (maintainable) approach is to add existing well maintained backends like v8 and wasm-bindings. And if someone can maintain a rust version of quickjs (like slimjs), it would solve all our problems.

[l0calh05t]

The biggest issue is the wasm itself is not well maintained and not official project of quickjs. I also have some hesitation to distribute an unofficial (i.e., untrusted) binary file.

Agreed, the lack of maintenance isn't great. However, the source is available: https://github.com/saghul/wasi-lab/tree/master/qjs-wasi and while an untrusted binary isn't great, it is running in a sandboxed environment.

As you may already find out, the quickjs wasm is designed to be a CLI rather than a library. This wouldn't work well with this library, especially we want to only initiate katex js once.

Yes, that's what I meant when I wrote that it doesn't fit the Engine-API well (and is probably very inefficient)

I still think the best (maintainable) approach is to add existing well maintained backends like v8 and wasm-bindings. And if someone can maintain a rust version of quickjs (like slimjs), it would solve all our problems.

The problem with quickjs is the lack of Windows/MSVC support. There are forks with Windows/MSVC support, but the official version isn't viable for the main Rust on Windows platform. v8 is probably the only real option right now

[Chaostheorie]

Might also be worth looking into rust-mozjs, the rust binding to spidermonkey. These binding also include an example of using it with a string for evaluating as a script. Since spidermonkey is a relatively well tested JS engine and has support for most OS's, this might a viable alternative.

[xu-cheng]

I have now released katex 0.4.0. It now supports two new JS backends, i.e., Duktape and wasm-bindgen. If you want to support more JS backends, PR is welcome.

There are some unsolved issues.

• The default feature selection is suboptimal. Currently, the default backend is QuickJS. To enable other backend, the crate users have to disable the default feature first. Ideally, the default feature should automatically select backend based on the platform. However, this is currently impossible due to the restriction of the Cargo.
• The wasm-bindgen backend only supports browser for now. In Node.js, as JS codes are imported through eval, it reports Cannot find module error. We need to find a way to tweak the JS code to be correctly interpreted by Node.js. Help is needed. It has now been fixed by katex 0.4.1.

[joseluis]

Could deno be another possible candidate?

[xu-cheng]

@joseluis deno uses v8 as its js engine. PR is welcome to add v8 or other js engines support to this crate.

[SichangHe]

It might be more desirable to AOT compile the JS.
https://github.com/manuel-serrano/hop

Their documentation is rather poor, though.

[SichangHe]

Maybe compile JS to WASM using javy and then to C using wasm2c.

[SichangHe]

Never mind, javy uses QuickJs under the hood.

[SichangHe]

It might be more desirable to AOT compile the JS. https://github.com/manuel-serrano/hop

Their documentation is rather poor, though.

I don't think Hop supports Windows so it defeats the purpose…

[SichangHe]

Is Webview a good idea?

[SichangHe]

An alive webview library would instead be Wry used by Tauri.

[SichangHe]

I think quickjs_es_runtime may replace quickjs-rs as a more maintained wrapper.

[l0calh05t]

I think quickjs_es_runtime may replace quickjs-rs as a more maintained wrapper.

Does this change that quickjs only works in Linux? wry looks like a more attractive option, especially since it uses the "native" webview instead of embedding a gigantic engine (although I don't know if it works without a visible window)

[SichangHe]

Wry won't work unless we make a big change… The webviews can only be spawned on the main thread.

tauri-apps/wry#906

[hhstore]

you could try rquickjs:

https://github.com/DelSkayn/rquickjs

[xu-cheng]

@hhstore a PR is welcome.

[l0calh05t]

you could try rquickjs:

https://github.com/DelSkayn/rquickjs

What does this solve? quickjs itself is an issue as outlined in the original post

[syockit]

rquickjs solves the issue by applying to quickjs the patches that has yet to be accepted upstream. It requires that you have the patch binary (e.g. from msys2) in your path though.

[charlyisidore]

I am trying to implement the backend using rquickjs ( https://github.com/charlyisidore/katex-rs/tree/rquickjs ).

It compiles successfully.

However, it passes all the tests except test_macros and test_throw_on_error (*).

thread 'tests::test_macros' panicked at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rquickjs-core-0.4.0/src/safe_ref.rs:35:20:
already borrowed: BorrowMutError

To whom might be interested in this implementation, any help would be appreciated.

---

(*) I slightly modified the test test_throw_on_error to detect the text parse error instead of ParseError, because rquickjs did not include the error name when converted to a string.

[andrieshiemstra]

fyi, quickjs_es_runtime can now also be used with quickjs-ng which is currently actively maintained.