From 86b3203af20816ef8831c9872514f73c4822841a Mon Sep 17 00:00:00 2001 From: jiangpengcheng Date: Wed, 28 Feb 2024 11:19:57 +0800 Subject: [PATCH] [improve][pip] PIP-339: Introducing the --log-topic Option for Pulsar Sinks and Sources (#22071) --- pip/pip-339.md | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 pip/pip-339.md diff --git a/pip/pip-339.md b/pip/pip-339.md new file mode 100644 index 0000000000000..a710ad453ea28 --- /dev/null +++ b/pip/pip-339.md @@ -0,0 +1,62 @@ +# PIP-339: Introducing the --log-topic Option for Pulsar Sinks and Sources + +# Motivation + +The `--log-topic` option already exists in Pulsar Functions, enabling users to direct function logs to a specified +"log topic". This feature is useful for debugging and analysis. However, Pulsar Sinks and Sources currently lack this +option, resulting in inconsistent log management across Pulsar Functions and Connectors. + +# Goals + +## In Scope + +The primary objective of this proposal is to integrate the `--log-topic` option into the **create**, **update**, and +**localrun** sub-commands for Pulsar Sinks and Sources. + +# Detailed Design + +## Design & Implementation Details + +1. Integrate the `--log-topic` option into `SinkDetailsCommand` and `SourceDetailsCommand`: + + ```java + @Parameter(names = "--log-topic", description = "The topic to which the logs of a Pulsar Sink/Source are produced") + protected String logTopic; + ``` + +2. Pass this option to `functionDetailsBuilder` when creating, updating, or locally running Pulsar Sinks and Sources: + ```java + if (sinkConfig.getLogTopic() != null) { + functionDetailsBuilder.setLogTopic(sinkConfig.getLogTopic()); + } + ``` + + ```java + if (sourceConfig.getLogTopic() != null) { + functionDetailsBuilder.setLogTopic(sourceConfig.getLogTopic()); + } + ``` + +3. Return the "log topic" when getting Pulsar Sinks and Sources + + ```java + if (!isEmpty(functionDetails.getLogTopic())) { + sinkConfig.setLogTopic(functionDetails.getLogTopic()); + } + ``` + + ```java + if (!isEmpty(functionDetails.getLogTopic())) { + sourceConfig.setLogTopic(functionDetails.getLogTopic()); + } + ``` + +# General Notes + +Upon successful implementation of this proposal, the **create**, **update**, and **localrun** sub-commands for Pulsar +Sinks and Sources will include the --log-topic option. + +# Links + +* Mailing List discussion thread: https://lists.apache.org/thread/8h6f8jcgs0cvvj96318zvcr18zs9513t +* Mailing List voting thread: https://lists.apache.org/thread/00682h05r4mh1plk10s6qq90p2s2xo74