Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Allow config for client reconnect in connector #138

Merged
merged 1 commit into from
Sep 19, 2024

Conversation

Joel-hanson
Copy link
Contributor

Contributes to: event-integration/eventstreams-planning#12716

Signed-off-by: Joel Hanson [email protected]
Signed-off-by: Joel Hanson [email protected]

Description

Queue instances are not duplicated across queue managers in a uniform cluster.
(In the same way that messages on a kafka topic are not duplicated across partitions - partition 0 does not have the same messages as partition 1, etc.)

This means:

Connect task starts working with a source queue on QM1.
Connect task stores it's state in the state queue on QM1.
Queue manager instructs the Connect task to rebalance to QM2.
Connect task disconnects from QM1 and reconnects to QM2
Connect task retrieves it's current state from the state queue on QM2 - which is empty, and doesn't reflect the Connect task's actual state which is stored on the state queue on QM1

Fixes https://github.ibm.com/event-integration/eventstreams-planning/issues/12716

Added a new configuration option for mq source connector to configure mq client reconnect option. Also to avoid exactly once to work with having client reconnect option as QMGR I have added in a new validation.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

Contributes to: event-integration/eventstreams-planning#12716

Signed-off-by: Joel Hanson <[email protected]>
Signed-off-by: Joel Hanson <[email protected]>
Copy link
Contributor

@dalelane dalelane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code changes look good to me

(I couldn't tell how the empty message new test cases in MQSourceTaskIT.java were relevant to this particular PR, but I don't object to more tests!)

@Joel-hanson Joel-hanson merged commit 7e9d940 into ibm-messaging:main Sep 19, 2024
2 checks passed
@Joel-hanson Joel-hanson deleted the feat-12716 branch September 19, 2024 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants