POC, event processors using go-plugin

[fredsod]

Hi,

I needed a way to dynamically add tags to Prometheus metrics and here is a POC using hashicorp go-plugin to build event processor plugins. In my case I would like to add metadata from our inventory to the metrics, but there could be many other use cases.

gNMIc loads plugins from './plugins/' and the name of the binary is also the plugin name, it then communicates with the plugin over RPC. Each plugin needs to implement the EventProcessor interface, due to the RPC the logging setup is a bit different in the plugin.

How to test the POC: Compile the examples/plugin/main.go to ./plugins/event-add-device_function and add 'event-add-device_function' as a processor for an output.

This is my first Golang project and I'm not a developer, but maybe we can get this up to shape and give gNMIc support for plugins together.

A couple of outstanding questions:

• Is there any better way to initialize the plugin_manager, now it uses init() and import in all.go.
• Add support for configuration of the plugin_manager, like debug log true/false and configurable plugin directory.
• Is there a way to send the logger object over RPC?

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[karimra]

This is great, thanks for giving it a go.

• Is there any better way to initialize the plugin_manager, now it uses init() and import in all.go.

That's the way

• Add support for configuration of the plugin_manager, like debug log true/false and configurable plugin directory.

This will be required yes, I can look into it.

• Is there a way to send the logger object over RPC?

I'm not sure that's possible, but what can be sent is the logger level (info, debug, etc...) and let the plugin decide where/how to log

[karimra]

@fredsod can you allow edits on this PR for maintainers so I can fix the conflicts ? Thanks

[fredsod]

@fredsod can you allow edits on this PR for maintainers so I can fix the conflicts ? Thanks

Edits are allowed, you should be able to add commits to this PR

[karimra]

It looks like you resolved the conflicts, thanks.

[fredsod]

It looks like you resolved the conflicts, thanks.

Have you had time to look into how to add support for configuration to the plugin_.manager?

[karimra]

@fredsod: Yes, check #341
I modified a few parts of the plugin manager to make it embeddable in the root app.
I also moved the plugin initialization to be part of the event-plugin init, that way the same plugin can be used under different processors of the same type. Each processor will get a separate running plugin.
The cleanup of the running plugins was not always happening, fixed that too.
Have a look at it and let me know what you think.