feat: mqtt wildcards support #6043

lupaulus · 2025-08-07T21:18:13Z

📋 Overview

What problem does this pull request address?
- Add support MQTT Topics Wildcard (+/#)
- https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/
Resolves [mqtt] Add Monitoring of Wildcard Topics like "/device/#" #5041 [mqtt] Add Monitoring of Wildcard Topics like "/device/#" #5041

I used an LLM for the contributions Assisted by Copilot but need to readapt

One potential Issue:

With wildcards, many simultaneous messages arriving to the client, can slow the process of handling and impact.
The added regex check doesn't help. I am not good enough in JS to produce some enhancement for performance

🛠️ Type of change

🐛 Bugfix (a non-breaking change that resolves an issue)
✨ New feature (a non-breaking change that adds new functionality)
⚠️ Breaking change (a fix or feature that alters existing functionality in a way that could cause issues)
[] 🎨 User Interface (UI) updates
📄 New Documentation (addition of new documentation)
📄 Documentation Update (modification of existing documentation)
📄 Documentation Update Required (the change requires updates to related documentation)
🔧 Other (please specify): Test Regex MQTT

📄 Checklist

CommanderStorm

Mostly looks good.

There are a few parts regarding formating, where CI is not happy. Those should be striaght forward enough to fix.

CommanderStorm · 2025-08-07T22:03:31Z

server/monitor-types/mqtt.js

+    static mqttTopicRegex(subcribedTopic) {
+        subcribedTopic = subcribedTopic.replace(/([$.|?*{}()\[\]\\])/g, '\\$1'); // Escape special regex chars except + and #
+        subcribedTopic = subcribedTopic.replace(/\+/g, '[^/]+'); // Replace + with regex for one or more characters except slash
+        subcribedTopic = subcribedTopic.replace(/#/g, '.*'); // Replace # with regex for zero or more characters until next slash


I think \# would also be affected under the current implementation.
Is this intentional?

For me, i don't know to much of the js regex engine but # is not considered as special char

I can add a test to check it

CommanderStorm · 2025-08-07T22:03:35Z

server/monitor-types/mqtt.js

 const { log, UP } = require("../../src/util");
 const mqtt = require("mqtt");
 const jsonata = require("jsonata");
+const { regex } = require("nostr-tools/nip30");


MQtt and nostr don't seem really connected.
Can you add a comment explaining this?

Wrong import, auto import from vs-code ...

CommanderStorm · 2025-08-07T22:06:14Z

test/backend-test/test-mqtt.js

+        assert.ok(regex.test('sensor.pomme/temperature') === true);
+        assert.ok(regex.test('sensor.pomme/humidity') === false);
+    });
+


please add a testcase specifically for the escaping of #, + and the other regex characters.

CommanderStorm · 2025-08-07T22:11:24Z

server/monitor-types/mqtt.js

+     */
+    static mqttTopicRegex(subcribedTopic) {
+        subcribedTopic = subcribedTopic.replace(/([$.|?*{}()\[\]\\])/g, '\\$1'); // Escape special regex chars except + and #
+        subcribedTopic = subcribedTopic.replace(/\+/g, '[^/]+'); // Replace + with regex for one or more characters except slash


I don't think this quite works when the start of the string is "+", or am I misunderstanding the syntax here wrong?

This would meann that \+ => \[^/]+ which likely won't work as you want it to.

Copilot

Pull Request Overview

This pull request adds MQTT wildcard support (+ and #) to the MQTT monitor type, allowing subscription to multiple topics using MQTT wildcard patterns. The implementation converts MQTT wildcard patterns into regular expressions for topic matching.

Implements MQTT wildcard support by adding regex-based topic matching
Replaces exact string matching with regex pattern matching for topic subscription
Adds comprehensive test coverage for wildcard functionality

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File	Description
`server/monitor-types/mqtt.js`	Adds `mqttTopicRegex()` method and replaces exact topic matching with regex-based matching
`test/backend-test/test-mqtt.js`	Adds comprehensive test cases for wildcard matching scenarios

server/monitor-types/mqtt.js

Copilot · 2025-08-09T09:53:38Z

server/monitor-types/mqtt.js

                clientId: "uptime-kuma_" + Math.random().toString(16).substr(2, 8)
            });

+            var regexTopic;


Use 'let' instead of 'var' for block-scoped variable declaration following modern JavaScript best practices.

Suggested change

var regexTopic;

let regexTopic;

NOPE here explicit usage of var for global scope of the instance

server/monitor-types/mqtt.js

lupaulus · 2025-08-10T20:17:58Z

need to check and test with regex engine

lupaulus · 2025-08-13T13:37:39Z

For me they are wrong positive from Eslint on 101regex the pattern are not correct recognized without escaping chars!

Don't merge until my go, I will do real tests with the app next week :)

lupaulus · 2025-08-20T19:00:00Z

Still in progress, please don't close

CommanderStorm · 2025-08-20T19:32:39Z

We generally only close PRs which are irrevocably stuck.
Those obviously also can be reopened.
This one is not stuck, it just needs a testcase and maybe a bugfix depending on said testcase.

feat: mqtt wildcards support

44e36dd

CommanderStorm requested changes Aug 7, 2025

View reviewed changes

CommanderStorm added the pr:please address review comments this PR needs a bit more work to be mergable label Aug 7, 2025

CommanderStorm reviewed Aug 7, 2025

View reviewed changes

CommanderStorm requested a review from Copilot August 9, 2025 09:52

Copilot AI reviewed Aug 9, 2025

View reviewed changes

lupaulus added 2 commits August 10, 2025 21:58

chore: mqtt.js adaptions

9460663

chore: renaming review linter

cf52d47

lupaulus added 2 commits August 13, 2025 13:42

ci : adaptation

cda7e86

ci: eslint disable for regex

67cc26e

lupaulus force-pushed the mqtt-wildcard branch from 0679efa to 67cc26e Compare August 13, 2025 14:07

ci: eslint disable for regex

2ba8568

lupaulus force-pushed the mqtt-wildcard branch from adf384c to 2ba8568 Compare August 13, 2025 14:11

CommanderStorm marked this pull request as draft August 27, 2025 01:21

lupaulus added 4 commits September 11, 2025 20:33

fix: regex check pattern without string quotes

67c70e0

test: add test to check escaped

3abbba4

fix: linting

81cdc6f

Merge branch 'master' into mqtt-wildcard

1979f13

lupaulus marked this pull request as ready for review September 11, 2025 20:48

CommanderStorm added pr:needs review this PR needs a review by maintainers or other community members and removed pr:please address review comments this PR needs a bit more work to be mergable labels Sep 12, 2025

Uh oh!

feat: mqtt wildcards support #6043

Are you sure you want to change the base?

feat: mqtt wildcards support #6043

Conversation

lupaulus commented Aug 7, 2025

📋 Overview

🛠️ Type of change

📄 Checklist

Uh oh!

CommanderStorm left a comment

Choose a reason for hiding this comment

Uh oh!

CommanderStorm Aug 7, 2025

Choose a reason for hiding this comment

Uh oh!

lupaulus Aug 10, 2025

Choose a reason for hiding this comment

Uh oh!

CommanderStorm Aug 7, 2025

Choose a reason for hiding this comment

Uh oh!

lupaulus Aug 10, 2025

Choose a reason for hiding this comment

Uh oh!

CommanderStorm Aug 7, 2025

Choose a reason for hiding this comment

Uh oh!

CommanderStorm Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Aug 9, 2025

Choose a reason for hiding this comment

Uh oh!

lupaulus Aug 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lupaulus commented Aug 10, 2025

Uh oh!

lupaulus commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lupaulus commented Aug 20, 2025

Uh oh!

CommanderStorm commented Aug 20, 2025

Uh oh!

Uh oh!

CommanderStorm Aug 7, 2025 •

edited

Loading

lupaulus commented Aug 13, 2025 •

edited

Loading