Excellent library

[cbh778899]

This npm library node-llama-cpp can be installed easily, it will download the pre-compiled llama.cpp executable which is only around 150mb and invoke GPU automatically, you can run GPU inference using its CLI without any setting, I'll try to implement it in our projects to make it useful to normal users instead of developers only.

To test, as long as you have node environment installed, just install the package using command npm install node-llama-cpp and run using npx node-llama-cpp chat models/model-name.gguf and it will setup everything else for you.

cc @Aisuko @Micost