Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][pip] PIP-396: Align WindowFunction's WindowContext with BaseContext #23659

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions pip/pip-396.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# PIP-396: Align WindowFunction's WindowContext with BaseContext

# Background knowledge

Pulsar Functions and Connectors provide a `BaseContext` object, enabling users to access the current execution environment, including logs, metrics, states, secrets, and other context-specific features.
The `WindowFunction`, designed to process multiple messages at once, offers a `WindowContext` object with similar functionality. However, `WindowContext` lacks certain features available in `BaseContext`.

# Motivation

Currently, the `WindowContext` interface does not include some critical methods provided by `BaseContext`, such as `getSecret` and `deleteState`. This limitation makes Window Functions feel less capable compared to standard Functions.

# Goals

## In Scope

- Enhance `WindowContext` by adding missing features from `BaseContext`.

## Out of Scope

# High-Level Design

The solution involves aligning `WindowContext` with `BaseContext` by extending the `WindowContext` interface and implementing missing methods in `WindowContextImpl`.

# Detailed Design

## Design & Implementation Details

Refer to implementation PR: https://github.com/apache/pulsar/pull/23628

The implementation strategy is straightforward:

- Update the `WindowContext` interface to extend the `BaseContext` interface.
- Remove duplicate method definitions in `WindowContext` already covered by `BaseContext`.
- Implement the following missing methods in the `WindowContextImpl` class:

```java
@Override
public String getSecret(String secretName) {
return this.context.getSecret(secretName);
}

@Override
public CompletableFuture<Void> incrCounterAsync(String key, long amount) {
return this.context.incrCounterAsync(key, amount);
}

@Override
public CompletableFuture<Long> getCounterAsync(String key) {
return this.context.getCounterAsync(key);
}

@Override
public CompletableFuture<Void> putStateAsync(String key, ByteBuffer value) {
return this.context.putStateAsync(key, value);
}

@Override
public CompletableFuture<ByteBuffer> getStateAsync(String key) {
return this.context.getStateAsync(key);
}

@Override
public void deleteState(String key) {
this.context.deleteState(key);
}

@Override
public CompletableFuture<Void> deleteStateAsync(String key) {
return this.context.deleteStateAsync(key);
}

@Override
public void fatal(Throwable t) {
this.context.fatal(t);
}
```

## Public-facing Changes

With this update, developers working on Window Functions can leverage the following new methods from WindowContext:

- getSecret
- incrCounterAsync
- getCounterAsync
- putStateAsync
- getStateAsync
- deleteState
- deleteStateAsync
- fatal

### Configuration

# Backward & Forward Compatibility

This update is fully backward-compatible. No existing methods in WindowContext are modified or removed, ensuring seamless integration with current implementations.