Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] deadLetterPolicy attribute in the ConsumerConfigurationData class marked as transient #23704

Open
3 tasks done
AnuragReddy2000 opened this issue Dec 10, 2024 · 3 comments · May be fixed by AnuragReddy2000/pulsar#1 or #23718
Open
3 tasks done
Labels
type/bug The PR fixed a bug or issue reported a bug

Comments

@AnuragReddy2000
Copy link

AnuragReddy2000 commented Dec 10, 2024

Search before asking

  • I searched in the issues and found nothing similar.

Read release policy

  • I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

Version

All Java clients of version v2.8.0 and above

Minimal reproduce step

Create an instance of ConsumerConfigurationData class and set a deadLetterPolicy on it. Serialize it and de-serialize it back to get a new instance of the same. The deadLetterPolicy previously set will not be present after deserialization.

What did you expect to see?

Ideally the deadLetterPolicy should have been preserved through serialization-deserialization.

What did you see instead?

null

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@AnuragReddy2000 AnuragReddy2000 added the type/bug The PR fixed a bug or issue reported a bug label Dec 10, 2024
@lhotari
Copy link
Member

lhotari commented Dec 10, 2024

btw. It would be useful context to explain the issue that you ran into with pulsar-storm. That seems to require serialization support?

@lhotari
Copy link
Member

lhotari commented Dec 10, 2024

Please also explain the context around #9630 which you explained in the GitHub Discussions question.

@AnuragReddy2000
Copy link
Author

AnuragReddy2000 commented Dec 10, 2024

btw. It would be useful context to explain the issue that you ran into with pulsar-storm. That seems to require serialization support?

Sure.

We use pulsar-storm adaptor to consume messages in Storm topologies. In a Storm cluster, there is a leader Nimbus node (analogous to a master / controller node) which controls the lifecycle of any topology.

Whenever a new instance of a topology needs to be created, the topology initialization code executes on the Nimbus. Once the topology components are all initialized, the Nimbus serializes them and sends them to the worker nodes, where the execution continues.

Now the PulsarSpout provided by the pulsar-storm adaptor is one such topology component that gets initialized on the Nimbus and later gets transferred to some workers. The deadLetterPolicy is one of the many configurations that we provide during the initialization of the PulsarSpout. But due to it being marked transient, the resultant instances of the configuration object on the workers do not contain any deadLetterPolicy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
2 participants