[DRAFT] POC to add WebAssembly Based Plugin System to OpenBB Terminal

[neuronicnobody]

This PR is for discussion purposes only, and not intended to be merged

Overview

Experimental PR to add a WebAssembly (Wasm) based plugin system to the OpenBB Terminal. Uses Extism, a Wasm framework, to speed up the development process and to improve the developer experience of using Wasm.

This PR focuses on the use case of creating various trading strategies as Extism plugins that can be dynamically loaded into the OpenBB terminal and executed using the Backtester. Strategies can be created in any language supported by Extism. WebAssembly provides sandboxed execution of plugins so the host process is protected from the inherent dangers of running arbitrary code.

Other possible use cases for plugins could include additional data source integrations, screeners, scripts, enhancements to existing commands, etc.

Also could be interesting to explore compiling the entire OpenBB SDK itself as an Extism Plugin. This would enable it to be used as a package in any host language supported by Extism. However, this would require the completion of a Python PDK that supports native dependencies. See note on Current Limitations below.

Goals

• Demonstrate the use of plugins as a viable mechanism for third parties to extend and customize OpenBB’s feature set without modifying the core application code.
• Explore the feasibility of embedding additional languages (e.g., Rust, Go, etc.) directly into the OpenBB’s Python codebase

Implementation Details

• Extism has been added as a new submenu under Backtesting
  • Added a new Model, View and Controller to manage the Extism plugin system
• An example plugin-based trading strategy is dynamically loaded into the terminal from a module hosting service. The plugin is written in Rust using the Extism Rust PDK. Source code can be found here
  • Note: The hardcoded link is for demo purposes. In real practice, there could be a command for users to install modules from the terminal itself (ie. install plugin <url>)
• Uses Host Function concept to enable the plugin to make calls back into the Host (ie. the OpenBB terminal) to offload some of the work such as calculating technical indicators (e.g., exponential moving average).
• The plugin takes as input, a series of prices, and returns a series of signals based on its particular trading strategy using a combination of technical indicators.

Demo

https://www.loom.com/share/615861dc013c49c09be436220bc100dc?sid=1bbcc744-ff0c-4cc2-855f-0a1c0421426f

Setup

Follow instructions here on the Python tab to install the required package

Current Limitations

• Creation of Python-based plugins that have native dependencies (e.g., Numpy, Pandas). This is currently a work in progress

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[piiq]

hey @neuronicnobody , this is a nice integration. we've released the v4 of the platform last week and maybe we can be looking into creating a backtesting v4 extension that would work with extism
have a look at the v4 docs here please https://docs.openbb.co/platform and let me know your thoughts

[neuronicnobody]

Hey @piiq! The new core + extensions framework for OpenBB Platform looks incredible. Very clean and scalable approach.

As you suggested, a v4 extension that is specifically tailored to backtesting functionality could work well here. The extension could delegate the backtesting logic to an Extism plugin using a defined interface. This would allow for many different implementations of the backtester (ie. extension in an extension 😄), and for those implementations to be developed in any language supported by Extism PDKs, which currently are Rust, C/C++, Go, JavaScript, C#, F#, Haskell, and Zig.

This current PR is focused more from the angle of strategies being written as plugins, while the backtesting logic lives in OpenBB (the host) itself, but I could also see a situation where both the strategies and backtester are extensions/plugins. There could be an v4 extension for each. Would just need to put some thought into the API and how that orchestration might work.

I'm also still very curious about the use case where OpenBB Core itself is compiled to Wasm! Would allow for it to be pulled into a large number of other host languages. There's still some work to be done polishing up the Python -> Wasm story but would love to collab on that if you find it interesting.