Search before reporting

• I searched in the issues and found nothing similar.

Read release policy

• I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

User environment

Broker version: 4.2.0-SNAPSHOT
Broker Operating system and hardware type: Linux x86_64 GNU/Linux
Broker Java version: 17.0.16

Issue Description

Each of the tests below were written with the assumption that the json key-value pairs would have a deterministic order (by comparing json results vs hard coded strings). The order of key-value pairs is not guaranteed in JSON files or JSON objects, though. As a result, the ordering can change due to different environments producing the contents in different orders despite the logical contents being the same. Since each of the tests below compare the raw strings/trees "as-is", harmless re-ordering could flip the test from pass to fail despite the data being semantically the same.

• org.apache.pulsar.common.policies.data.NamespaceOwnershipStatusTest#testSerialization
• org.apache.pulsar.common.policies.impl.NamespaceIsolationPoliciesTest#testJsonSerialization
• org.apache.pulsar.functions.utils.FunctionConfigUtilsTest#testConvertBackFidelity
• org.apache.pulsar.functions.utils.SinkConfigUtilsTest#testConvertBackFidelity
• org.apache.pulsar.functions.utils.SourceConfigUtilsTest#testBatchConfigMergeEqual
• org.apache.pulsar.io.elasticsearch.ElasticSearchExtractTest#testGenericRecord
• org.apache.pulsar.io.kinesis.UtilsTest#testKeyValueSerializeNoValue.

I discovered the problem with the NonDex tool. NonDex systematically detects incorrect tests that rely on non-deterministic behaviors in Java APIs—like assuming order of name/value pairs in json files — by exploring all specification-allowed outcomes. It does this by instrumenting undetermined APIs and randomizing the returned order/behavior within what the spec allows. Failures it exposes reliably indicate flawed assumptions in order that were never promised and should be fixed.

Note: I created a single issue at the advice here: #24808

Error messages

https://gist.github.com/LucasEby/0d9dc227f3b835527078245672f31efa

Reproducing the issue

You can replicate the problem with the NonDex tool commands:

mvn -pl pulsar-common -Dtest=org.apache.pulsar.common.policies.data.NamespaceOwnershipStatusTest#testSerialization -DforkCount=1 -DnondexSeed=1016066 -DnondexRuns=1 -DreuseForks=false edu.illinois:nondex-maven-plugin:2.2.1:nondex

mvn -pl pulsar-common -Dtest=org.apache.pulsar.common.policies.impl.NamespaceIsolationPoliciesTest#testJsonSerialization -DforkCount=1 -DnondexSeed=1057510 -DnondexRuns=1 -DreuseForks=false edu.illinois:nondex-maven-plugin:2.2.1:nondex

mvn -pl pulsar-functions/utils -Dtest=org.apache.pulsar.functions.utils.FunctionConfigUtilsTest#testConvertBackFidelity -DforkCount=1 -DnondexSeed=1016066 -DnondexRuns=1 -DreuseForks=false edu.illinois:nondex-maven-plugin:2.2.1:nondex

mvn -pl pulsar-functions/utils -Dtest=org.apache.pulsar.functions.utils.SinkConfigUtilsTest#testConvertBackFidelity -DforkCount=1 -DnondexSeed=974622 -DnondexRuns=1 -DreuseForks=false edu.illinois:nondex-maven-plugin:2.2.1:nondex

mvn -pl pulsar-functions/utils -Dtest=org.apache.pulsar.functions.utils.SourceConfigUtilsTest#testBatchConfigMergeEqual -DforkCount=1 -DnondexSeed=974622 -DnondexRuns=1 -DreuseForks=false edu.illinois:nondex-maven-plugin:2.2.1:nondex

mvn -pl pulsar-io/elastic-search -Dtest=org.apache.pulsar.io.elasticsearch.ElasticSearchExtractTest#testGenericRecord -DforkCount=1 -DnondexSeed=974622 -DnondexRuns=1 -DreuseForks=false edu.illinois:nondex-maven-plugin:2.2.1:nondex

mvn -pl pulsar-io/kinesis -Dtest=org.apache.pulsar.io.kinesis.UtilsTest#testKeyValueSerializeNoValue -DforkCount=1 -DnondexSeed=974622 -DnondexRuns=1 -DreuseForks=false edu.illinois:nondex-maven-plugin:2.2.1:nondex

Additional information

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!