Skip to content

Add ability to set pipeline topics at runtime #1750

@eaton-coreymutter

Description

@eaton-coreymutter

🚀 Feature Request

Relevant Package [REQUIRED]

This feature request is for... App Functions SDK

Description [REQUIRED]

I have been working on some app services where the service is interested in different devices/topic trees depending on what happens at runtime. E.g. in SSE we want to process the topics the web browser has asked about, or in an analytics service we might want to configure, in the service, which devices will be watched.

Currently, to do this we need to have the app service subscribe to everything, then in our pipeline, ignore most events. There is a nontrivial amount of work involved to decode/unmarshal the messages before the app service gets the callback, even if we are asking for the raw message body.

It would be nice to have event messages ignored as early as possible.

Describe the solution you'd like

I have a pull request ready to push once this issue is submitted: using AddFunctionsPipelineForTopics() with only a dummy topic at startup, then adding SetFunctionsPipelineTopics() to update the topic list associated with the pipeline as needed.

Testing with SSE on a busy system, this saves about 25% of the CPU usage of the app service. I think this is from not having to instantiate the context or runtime for messages we are not interested in.

Implementation is straightforward, and already has the locking needed to prevent message loss/duplication. It doesn't present security issues since everything is still bounded by the underlying message bus subscription.

Describe alternatives you've considered

Updating the actual message bus subscription would be even more efficient, but seems more complicated. We would want to be able to bound the allowable topics, and trying to prevent message loss/duplication during changes would be difficult. Maybe that's a future feature request.

If this is architecturally inadvisable, let me know.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions