Editor Plugins

[Speak2Erase]

Is your feature request related to a problem? Please describe.
People tend to build features that extend RPG Maker, (such as custom event properties) and they're usually implemented with hacks. Sound emitters in Fading Memory are a great example of this- events have a magic comment denoting that they are an emitter. From what I've seen as well the use of the Notes section in VX Ace is commonly used to add extra fields to data types.

We can cover the majority of simple cases by adding custom event commands (I am working on that, plugins could likely integrate really well into the system I have in mind) and providing a simple way to specify extra fields on types from the luminol_data crate. However, that won't scale to anything more complex than editing a string or two.

Describe the solution you'd like
A plugin system (preferably in a scripting language) would allow users to extend Luminol with custom features, such as adding new ui, tools specific to their game (such as save editors) and would greatly help to improve the longevity of Luminol.

Describe alternatives you've considered

Lua

Lua seems to be the best option for writing plugins. There are already some amazing bindings out there and Lua is pretty easy to build. It's designed to be integrated as a scripting system as well, which is exactly what we'd want!

Rust

Luminol is made in Rust, which at a glance means we don't have to write a lot of custom bindings. However Rust isn't ABI stable! This is made worse because Luminol only compiles with nightly at the moment. There's also the hassle of having to distribute platform specific builds of plugins as well.

Javascript

We wouldn't need to bundle an entire Javascript runtime on web builds as it's built into the browser already. Unfortunately Javascript runtimes are pretty heavyweight and there don't appear to be many rust bindings available, as far as I can tell. I don't think we want to turn Luminol into a browser!

Ruby

It's what most people using Luminol will be familiar with, and there are decent bindings out there (magnus and rb-sys). That's about the only thing going for Ruby, unfortunately. Ruby is extremely heavyweight- and stubborn to build to say the least. Webassembly builds appear to be experimental as well.
It'd also railroad us into using mingw on Windows, which is a whole other mess.

Python

I don't have enough experience working with Python to make any solid comments, but I'd imagine Python would have the same problems as Ruby, without the benefit of most RPG Maker users being familiar with it.

Additional context
Unless we go with Rust we will have to write egui bindings. I'm also not sure how to even integrate plugins in an immediate mode UI without having tons of hooks everywhere.

This is likely not something that should be worked on until we have a solid idea of how plugins should work and the editor is more finished. Discussion is encouraged!

[zimberzimber]

I believe custom commands and event/page metadata could be implemented without scripting.

The command interfaces could be defined via schema, making them fairly generic.
If all the commands are built via schemas, it will not only cover all but the edgiest of use cases, but would also allow the user to modify existing commands, without the need to hack the extra functionality in or duplicate commands just to have the extended functionality. (Like I had to do for FM to implemented conditional dialogue choices)
What MIGHT require scripting on the user's end, is if they'd want to add a new type of picker.

Example for "Control Variables":

[
  {
    "name": "Variable",
    "controls": [
      {
        "name": "Single",
        "controls": "VariablePicker"
      },
      {
        "name": "Batch",
        "controls": "NumberRange"
      }
    ]
  },
  {
    "name": "Operation",
    "controls": "NumericOperationPicker"
  },
  {
    "name": "Operand",
    "controls": [
      {
        "name": "Constant",
        "controls": "IntegerPicker"
      },
      {
        "name": "Variable",
        "controls": "VariablePicker"
      },
      {
        "name": "Random",
        "controls": "NumberRange"
      },
      {
        "name": "Item",
        "controls": "ItemPicker"
      },
      {
        "name": "Actor",
        "controls": "ActorStatPicker"
      },
      {
        "name": "Enemy",
        "controls": "EnemyStatPicker"
      },
      {
        "name": "Character",
        "controls": "EventDataPicker"
      },
      {
        "name": "Other",
        "controls": "MiscDataPicker"
      }
    ]
  },
]

Metadata such as sound emission for events/pages could be a simple key:value table.

[Speak2Erase]

Oh, that's actually what I'm planning to do! It'll cover most basic use cases but will fall flat if the user needs to add new pickers as you mentioned.

[white-axe]

I also think Lua is the best option not only because it's lightweight and easy to build but also because it's very easy to sandbox Lua (the native version of Luminol would probably benefit from this). This one here is a 200-line Lua script: https://github.com/kikito/lua-sandbox

[white-axe]

Another option, actually, would be to use WebAssembly for the plugins. We can use either Wasmtime or Wasmer (both written in Rust!) to run WebAssembly modules natively. The downside of this approach is that people would have to compile their plugins, and the support for compiling to WebAssembly from most scripting languages is quite poor.

[zimberzimber]

If longevity is the goal, then the bigger issue with Rust/WASM is the bar of entry being way higher.
Lua and Python are likely the best in that regard, although Ruby and JS have a place because people might already be familiar with them through RPG Maker.

[Speak2Erase]

It's also worth noting that mlua provides functionality to convert Lua functions to futures, allowing us to schedule them on something like a Tokio runtime

[Speak2Erase]

I've been thinking on this lately, and I've realized that webassembly might be a better choice? We'd have to use WASI specifically though, as that seems to be what most languages are targeting. That'd mean polyfilling some functions like filesystem access, which we'd probably want to do anyway to ensure plugins can't access outside of the project folder.

C# and Java can compile for it, Ruby can too, Lua can, Python can, C/C++ can, and of course Rust can compile for WASI as well.

I'm not sure how plugins would be able to override behaviour yet, but I do have an idea for how plugins would be able to call luminol code.
We'd provide a C API, essentially, and pass things like rust enums (Option, Result, etc) and complex structures (HashMap, Vec, etc) as opaque types with vtables to work with them.

typedef struct i32Vec {
  void* data;
  struct {
    void (*vec_push)(void*, int);
    int (*vec_pop)(void*);
  } *vtable;
};

[white-axe]

I'm fine with WebAssembly too. There's already a language for describing WebAssembly APIs, WIT, that would be better suited for describing how plugins call Luminol's code than C though.

As for overriding behaviour, WebAssembly has built-in support for exporting functions as well, not just importing them, so we can specify some functions in the WIT language that the plugin may or may not implement, and Luminol will attempt to call those to determine how the plugin wants to override something.

[Speak2Erase]

Wasmer looks like the runtime we should use as it supports running in the browser

[white-axe]

Wasmer has no support for autogenerating Rust bindings from WIT though, which would be helpful since then we'd only have to write the plugin API once as a bunch of WIT files instead of also having to manually write all the Rust code to convert Rust types to WebAssembly types.

As an alternative I would suggest using wasm-bridge, which uses Wasmtime in native builds and reimplements Wasmtime's API using the browser's WebAssembly APIs in web builds. Wasmer also just defers to the browser APIs in web builds, so I don't see any downsides to this.

Wasmtime has the wasmtime::component::bindgen macro for generating bindings from WIT files. wasm-bridge in turn has wasm_bridge::component::bindgen since it copies Wasmtime's API.

[Speak2Erase]

We could always use wit-bindgen?

[white-axe]

wit-bindgen generates bindings for use in WebAssembly binaries, i.e. this is one tool plugins could use to generate bindings for Luminol's API. wit-bindgen does not, however, generate bindings on the side of the WebAssembly runtime; that has to be implemented by the runtime or done manually.