Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Documented CDC example fails #23662

Open
2 of 3 tasks
AlvaroStream opened this issue Nov 29, 2024 · 0 comments
Open
2 of 3 tasks

[Bug] Documented CDC example fails #23662

AlvaroStream opened this issue Nov 29, 2024 · 0 comments
Labels
type/bug The PR fixed a bug or issue reported a bug

Comments

@AlvaroStream
Copy link
Contributor

Search before asking

  • I searched in the issues and found nothing similar.

Read release policy

  • I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

Version

MAC OS 14.7 - Apache 4.0.0 standalone - pulsar-io-debezium-postgres-4.0.0.nar

Minimal reproduce step

As documented here:

Started the container:

ocker run -d -it --rm \                
    --name pulsar-postgres \
    -p 5432:5432 \
    -e POSTGRES_PASSWORD=changeme \
    postgres:13.3 -c wal_level=logical

Started the standalone

./bin/pulsar standalone 

Launched the local run, copied previously the nar in connector directory.


bin/pulsar-admin source localrun \  
    --archive $PWD/connectors/pulsar-io-debezium-postgres-4.0.0.nar \
    --name debezium-postgres-source \
    --tenant public \
    --namespace default \
    --source-config '{"database.hostname": "localhost","database.port": "5432","database.user": "postgres","database.password": "changeme","database.dbname": "postgres","database.server.name": "dbserver1","plugin.name": "pgoutput","schema.whitelist": "public","table.whitelist": "public.users","pulsar.service.url": "pulsar://127.0.0.1:6650"}'

Execute the changes

UPDATE users SET hash_firstname='maxim' WHERE id=1;

and I can see this in the logs of the connection working:

2024-11-29T10:06:53,848+0100 [debezium-postgresconnector-dbserver1-change-event-source-coordinator] INFO  io.debezium.connector.postgresql.connection.AbstractMessageDecoder - Streaming requested from LSN LSN{0/15F8A00}, received LSN LSN{0/15F8A00} identified as already p
rocessed                                                                                                                                                                                                                                                                       
2024-11-29T10:06:53,848+0100 [debezium-postgresconnector-dbserver1-change-event-source-coordinator] INFO  io.debezium.connector.postgresql.connection.WalPositionLocator - Message with LSN 'LSN{0/15F8AA8}' arrived, switching off the filtering                              
2024-11-29T10:07:37,861+0100 [pulsar-client-io-1-3] INFO  org.apache.pulsar.io.kafka.connect.AbstractKafkaConnectSource - Finished flushing offsets to storage                                                 

But the subscription never receives any message:

bin/pulsar-client consume -s "sub-users" public/default/dbserver1.public.users -n 0 -p Earliest  

What did you expect to see?

A message in the topic

What did you see instead?

Nothing, seems the connector is connecting properly to the database

Anything else?

I'm not sure how to debug it more.

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@AlvaroStream AlvaroStream added the type/bug The PR fixed a bug or issue reported a bug label Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Development

No branches or pull requests

1 participant