Skip to content
This repository has been archived by the owner on Apr 1, 2024. It is now read-only.

Commit

Permalink
[improve][pip] PIP-339: Introducing the --log-topic Option for Pulsar…
Browse files Browse the repository at this point in the history
… Sinks and Sources (apache#22071)
  • Loading branch information
jiangpengcheng authored Feb 28, 2024
1 parent 56eb758 commit 86b3203
Showing 1 changed file with 62 additions and 0 deletions.
62 changes: 62 additions & 0 deletions pip/pip-339.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# PIP-339: Introducing the --log-topic Option for Pulsar Sinks and Sources

# Motivation

The `--log-topic` option already exists in Pulsar Functions, enabling users to direct function logs to a specified
"log topic". This feature is useful for debugging and analysis. However, Pulsar Sinks and Sources currently lack this
option, resulting in inconsistent log management across Pulsar Functions and Connectors.

# Goals

## In Scope

The primary objective of this proposal is to integrate the `--log-topic` option into the **create**, **update**, and
**localrun** sub-commands for Pulsar Sinks and Sources.

# Detailed Design

## Design & Implementation Details

1. Integrate the `--log-topic` option into `SinkDetailsCommand` and `SourceDetailsCommand`:

```java
@Parameter(names = "--log-topic", description = "The topic to which the logs of a Pulsar Sink/Source are produced")
protected String logTopic;
```

2. Pass this option to `functionDetailsBuilder` when creating, updating, or locally running Pulsar Sinks and Sources:
```java
if (sinkConfig.getLogTopic() != null) {
functionDetailsBuilder.setLogTopic(sinkConfig.getLogTopic());
}
```

```java
if (sourceConfig.getLogTopic() != null) {
functionDetailsBuilder.setLogTopic(sourceConfig.getLogTopic());
}
```

3. Return the "log topic" when getting Pulsar Sinks and Sources

```java
if (!isEmpty(functionDetails.getLogTopic())) {
sinkConfig.setLogTopic(functionDetails.getLogTopic());
}
```

```java
if (!isEmpty(functionDetails.getLogTopic())) {
sourceConfig.setLogTopic(functionDetails.getLogTopic());
}
```

# General Notes

Upon successful implementation of this proposal, the **create**, **update**, and **localrun** sub-commands for Pulsar
Sinks and Sources will include the --log-topic option.

# Links

* Mailing List discussion thread: https://lists.apache.org/thread/8h6f8jcgs0cvvj96318zvcr18zs9513t
* Mailing List voting thread: https://lists.apache.org/thread/00682h05r4mh1plk10s6qq90p2s2xo74

0 comments on commit 86b3203

Please sign in to comment.