feat(plugins): Implement plugin API key management and authentication middleware #746

AnthonyMichaelTDM · 2025-07-18T06:30:52Z

Motivation

Currently, Zoraxy renders plugins in a webframe and allows various ways of deciding which kinds of requests should be handled by a plugin.

A big limitation of the plugin system is that while plugins can make requests to their own backend, they cannot interract with the zoraxy api in any meaningful capacity.

I'm implementing a crowdsec bouncer for zoraxy, ideally I'd be able to adjust the zoraxy blacklist directly but due to the previously mentioned limitations I had to implement the bouncer as a dynamic capture plugin which is less then ideal for a variety of reasons.

Purpose

The purpose of this PR is to give plugins scoped access to internal APIs over https.

Overview

Plugins can declare a list of PermittedAPIEndpoint's in their IntroSpect, each must include the endpoint, the http method (e.g. GET, PUSH), and the reason.
Plugins that declare such a list will receive an API key as part of their ConfigureSpec

These API keys are generated for each applicable plugin when they start, and are invalidated when the plugin stops.

When making requests to the Zoraxy API, the api key should be included as Bearer <API_KEY> in the Authorization header.

The authRouter has been modified so that handlers for pluginAccessible endpoints are wrapped in an additional layer of middleware that will look for the Auth Bearer header, and if its present will validate that the provided API key has access to the endpoint. If the key has access, the normal auth mechanism can be bypassed. If the key is invalid or does not have access, the request fails with a 403 Unauthorized message.

Requests to non-pluginAccessible endpoints redirect to the login page as they did before.
Requests to pluginAccessible endpoints that are missing the Auth Bearer header are treated as they were before.

Status

feature implemented
example plugin to demonstrate using the feature
feature integrated into webUI

Pictures

Where PermittedAPIEndpoints are displayed in the web UI

UI of the example plugin

This UI demonstrates that:

valid requests to permitted endpoints succeed
invalid (wrong key) requests to permitted endpoints are unauthorized
valid requests to unpermitted endpoints are unauthorized
requests to endpoints that aren't plugin accessible fail (you get the login page (normal auth behavior))

Future work

more carefully examine which API are exposed, current implementation is fairly conservative about which endpoints are accessible but may very well be too restrictive
add methods to make it easier for plugin authors to make calls to the endpoint (specifically, to automate setting the Auth Bearer header)
use a more "industry-standard" method of generating the tokens, currently the token is generated by randomly generating 32 bytes, then hashing that with sha256
maybe don't pass the api key in plain text over HTTP? probably doesn't matter since it's all happening over the loopback interface (localhost) but like .. probably still bad practice

… middleware The purpose of this is to allow plugins to access certain internal APIs via - Added PluginAPIKey and APIKeyManager for managing API keys associated with plugins. - Introduced PluginAuthMiddleware to handle API key validation for plugin requests. - Updated RouterDef to support plugin accessible endpoints with authentication. - Modified various API registration functions to include plugin accessibility checks. - Enhanced plugin lifecycle management to generate and revoke API keys as needed. - Updated plugin specifications to include permitted API endpoints for access control.

…ed endpoint

AnthonyMichaelTDM · 2025-07-18T06:35:58Z

It's a pretty big PR, sorry about that, but about 2/3rds of the additions (1100 lines) are from the example plugin so it looks worse than it is

AnthonyMichaelTDM · 2025-07-18T06:57:45Z

regarding

@tobychui #738 (reply in thread)
In fact I kinda want to work on a restful API sub system for Zoraxy (for automation and testing), maybe these two features can share the same token db?

That should be possible, currently a given API key lives only as long as the plugin is running, no attempt is made at persisting the token DB since tokens are regenerated at runtime.
What this does mean is that a restful API sub-system could be very nicely implemented as a plugin with the only modifications to the source-code being to make more endpoints plugin-accessible.

Note: the distinction between pluginAccessible and non pluginAccessible endpoints is motivated by the assumption that some endpoints should only ever be accessible to the actual admin (e.g. changing passwords, (un)installing plugins (this one especially since if there was an exploit in the plugin-auth system, being able to install and run plugins could be a (wormable?) RCE vulnerability), etc.)

AnthonyMichaelTDM

This "review" is just a code tour, I do this with big PRs to make them easier to digest.

Feel free to resolve things as you go to reduce the clutter

example/plugins/api-call-example/ui.go

src/api.go

src/mod/auth/auth.go

src/mod/auth/plugin_apikey_manager.go

src/mod/auth/plugin_middleware.go

src/mod/auth/router.go

src/mod/plugins/lifecycle.go

src/mod/plugins/zoraxy_plugin/zoraxy_plugin.go

AnthonyMichaelTDM · 2025-07-18T21:32:37Z

regarding

@tobychui #738 (reply in thread)
In fact I kinda want to work on a restful API sub system for Zoraxy (for automation and testing), maybe these two features can share the same token db?

That should be possible, currently a given API key lives only as long as the plugin is running, no attempt is made at persisting the token DB since tokens are regenerated at runtime. What this does mean is that a restful API sub-system could be very nicely implemented as a plugin with the only modifications to the source-code being to make more endpoints plugin-accessible.

Note: the distinction between pluginAccessible and non pluginAccessible endpoints is motivated by the assumption that some endpoints should only ever be accessible to the actual admin (e.g. changing passwords, (un)installing plugins (this one especially since if there was an exploit in the plugin-auth system, being able to install and run plugins could be a (wormable?) RCE vulnerability), etc.)

Actually, expanding on this further, I wonder how viable it would be to have the example plugin read a list of endpoints to register and endpoints to hit from a config file and then dynamically create that "report" in the UI based on that list

Thinking about it more, yeah, I could definitely do that. The real question then is whether to include that in this PR or wait for this to be merged and do it in a new PR

The format could be something like:

permitted-endpoints:
    - endpoint: /api/access/list
      method: GET
      reason: used as an example of a permitted and pluginAccessible endpoint
    - endpoint: /api/cert/tls
      method: GET
      reason: used as an example of a "permitted" but not pluginAccessible 

endpoints-to-hit:
    - endpoint: /api/access/list
      expected-result: success
    - endpoint: /api/access/list
      api-key-override: invalid-key
      expected-result: redirect
    - endpoint: /api/cert/tls
      expected-result: redirect
    - endpoint: /api/access/list
      method-override: HEAD
      expected-result: unauthorized
    - endpoint: /api/acme/listExpiredDomains
      expected-result: unauthorized
    - endpoint: /api/proxy/list
      expected-result: unauthorized

which could allow the plugin to be used as a kind of test-suite for the rest API

The webUI of the plugin would generate those blocks automatically based on the response (green for success, red for redirect, yellow for unauthorized, purple for other) and report whether the response matched the expectation

tobychui

Great work and high quality code!

I have left some comments in how the handleFunc and plugin API should be two separated handlers instead of using the same handler as auth.handleFunc. Maybe you can restore the api.go and create a new plugin_api.go with a new handler instead?

That way, we can keep the behavior of old auth.handleFunc and make it more easy for future expansion regarding the token store which automatic testing script can utilize.

src/mod/auth/router.go

tobychui · 2025-07-20T02:30:49Z

src/mod/auth/router.go

 }

-func (router *RouterDef) HandleFunc(endpoint string, handler func(http.ResponseWriter, *http.Request)) error {
+func (router *RouterDef) HandleFunc(endpoint string, handler func(http.ResponseWriter, *http.Request), pluginAccessible bool) error {


Adding this will break the compatibility with the native net.http HandleFunc function. I guess it would be better to implement it as alternative API instead (maybe adding a new handle function instead?)

oh I didn't even think about that, great point

src/api.go

AnthonyMichaelTDM · 2025-07-20T04:22:46Z

Sure, those comments make a lot of sense.
What path should I put the new endpoints at? I'm thinking something like /plugin-api/... or /plugin/api/... or maybe /rest/...

Also, on that point, which endpoints should be included there? Should I just start with the ones I have marked as pluginAccessible and leave the rest as future work?

I'll probably get started on this tomorrow

AnthonyMichaelTDM · 2025-07-20T05:26:29Z

I'll probably get started on this tomorrow

nvm, it was easier than I thought, commits incoming

…le endpoints

AnthonyMichaelTDM · 2025-07-20T05:33:04Z

@tobychui
I've implemented your requested changes if you'd like to take a look
(still need to update the example plugin though)

tobychui · 2025-07-20T05:33:30Z

@AnthonyMichaelTDM Cool! Let me release v3.2.5 first and I will fork a v3.2.6 branch and merge this into the new branch. This should give us more time testing it before it got released. Will wait for your example update, let me know when you are ready :)

AnthonyMichaelTDM · 2025-07-20T05:36:35Z

~~ready when you are~~
~~scratch that, somethings wrong. gimme a second~~
okay, fixed. actually ready now

AnthonyMichaelTDM · 2025-07-20T05:52:17Z

Well that was a goofy mistake, all good now

tobychui

LGTM, I few minor things that I might wanna change but I will merge this first.

AnthonyMichaelTDM · 2025-07-20T07:47:35Z

LGTM, I few minor things that I might wanna change but I will merge this first.

Sounds good, do you want to open an issue for those minor things?

tobychui · 2025-07-20T07:49:45Z

Sounds good, do you want to open an issue for those minor things?

No, I think I will directly push it into the v3.2.6 branch.

AnthonyMichaelTDM · 2025-07-20T07:57:50Z

Fair enough, in that case I'm going to sign off for the night.

Hope to work on either: resolve the issues causing the CI script I'm adding in #752 to fail (pending a decision on whether you'd prefer those changes done in that PR or in a new one), or creating the example plugin for #753 tomorrow.

AnthonyMichaelTDM added 6 commits July 17, 2025 22:20

feat(plugins): Add API Call Example Plugin

5a38c1d

feat: Add .gitignore for Docker to exclude example and src directories

2d43890

feat(webui/plugininfo): Add section for permitted API endpoints

46cfc02

fix(plugin-auth): check both endpoint and method

ed8f9b7

feat(plugins): add an example api call to an accessible but unpermitt…

d187c32

…ed endpoint

AnthonyMichaelTDM mentioned this pull request Jul 18, 2025

feat(plugins): Implement event system w/ POC events #747

Closed

6 tasks

AnthonyMichaelTDM commented Jul 18, 2025

View reviewed changes

tobychui requested changes Jul 20, 2025

View reviewed changes

AnthonyMichaelTDM added 2 commits July 19, 2025 22:23

refactor: partial revert of dd93f9a

39b5da3

remove unused functions

f9e51bf

AnthonyMichaelTDM added 2 commits July 19, 2025 22:29

refactor: reuse PluginAuthMiddleware as AuthAgent for plugin accessib…

be5f631

…le endpoints

feat: add the plugin accessible endpoints

e3e31d9

fix: update example plugin

40f915f

fix(example plugin): update PermittedAPIEndpoints

ad2519d

AnthonyMichaelTDM requested a review from tobychui July 20, 2025 05:54

tobychui changed the base branch from v3.2.5 to v3.2.6 July 20, 2025 06:48

AnthonyMichaelTDM mentioned this pull request Jul 20, 2025

feat(plugins): Implement event system w/ POC events #753

Merged

6 tasks

tobychui approved these changes Jul 20, 2025

View reviewed changes

tobychui merged commit 381184c into tobychui:v3.2.6 Jul 20, 2025

AnthonyMichaelTDM deleted the plugin-improvements-api-keys branch July 20, 2025 07:58

AnthonyMichaelTDM mentioned this pull request Jul 20, 2025

[tracker] Plugin System Improvements #754

Open

12 tasks

AnthonyMichaelTDM mentioned this pull request Aug 21, 2025

[ENHANCEMENTS] Enable HTTP/2 over cleartext TCP (h2c) support #527

Closed

This was referenced Sep 7, 2025

feat(event system): Flesh out EventPayload interface #806

Closed

[BUG] Plugins cannot make post requests #819

Closed

feat(plugins): Implement plugin API key management and authentication middleware #746

feat(plugins): Implement plugin API key management and authentication middleware #746

Uh oh!

Conversation

AnthonyMichaelTDM commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Purpose

Overview

Status

Pictures

Where PermittedAPIEndpoints are displayed in the web UI

UI of the example plugin

Future work

Uh oh!

AnthonyMichaelTDM commented Jul 18, 2025

Uh oh!

AnthonyMichaelTDM commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AnthonyMichaelTDM left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AnthonyMichaelTDM commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tobychui left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tobychui Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

AnthonyMichaelTDM Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AnthonyMichaelTDM commented Jul 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AnthonyMichaelTDM commented Jul 20, 2025

Uh oh!

AnthonyMichaelTDM commented Jul 20, 2025

Uh oh!

tobychui commented Jul 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AnthonyMichaelTDM commented Jul 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AnthonyMichaelTDM commented Jul 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tobychui left a comment

Choose a reason for hiding this comment

Uh oh!

AnthonyMichaelTDM commented Jul 20, 2025

Uh oh!

tobychui commented Jul 20, 2025

Uh oh!

AnthonyMichaelTDM commented Jul 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

AnthonyMichaelTDM commented Jul 18, 2025 •

edited

Loading

AnthonyMichaelTDM commented Jul 18, 2025 •

edited

Loading

AnthonyMichaelTDM left a comment •

edited

Loading

AnthonyMichaelTDM commented Jul 18, 2025 •

edited

Loading

AnthonyMichaelTDM commented Jul 20, 2025 •

edited

Loading

tobychui commented Jul 20, 2025 •

edited

Loading

AnthonyMichaelTDM commented Jul 20, 2025 •

edited

Loading

AnthonyMichaelTDM commented Jul 20, 2025 •

edited

Loading