Skip to content

Conversation

anthonyx24
Copy link
Contributor

@anthonyx24 anthonyx24 commented Oct 1, 2025

Issue: The method createDefaultPublishModel() in TestInvocationPublishModelFactory compares JSONs to verify that the default object is properly created. However, since JSONs are unordered collections, when converted to strings the parameter order may not be preserved. This test uses a hard-coded JSON string as the expected correct value, but the test could fail if the parameters from the test object are stringified in a different order than the expected string. For another example of this issue being addressed, see this previous merged PR.

This test was flagged via the NonDex tool, which detects potentially unreliable tests due to underlying Java API assumptions. To see the Nondex output for this test, you can run:

mvn -pl metrics/metrics-core edu.illinois:nondex-maven-plugin:2.1.7:nondex -Dtest="org.apache.servicecomb.metrics.core.publish.TestInvocationPublishModelFactory#createDefaultPublishModel"

Fix: The fix I implemented uses JSONAssert's assertEquals() method, using non-strict checking (which ignores parameter ordering). This compares the JSON strings without enforcing parameter ordering, which removes the potentially unreliable nature of these tests. This library is already included within the spring-boot dependency, which is already used in the project, so there's no need to update the pom file. Rerunning Nondex shows a passing result.

I believe this is the simplest fix, but it seems someone else has already fixed a similar test in a different way: #4633 using JsonNode trees. If it's better to keep code consistency, I can also implement the same methodology here.

One additional change I had to make was to add the throws Exception clause onto the test method signature since JSONAssert.assertEquals() can throw a JSONException. Please let me know if this is an issue; this clause already exists in the same method in line 48 of TestThreadPoolPublishModelFactory, and the integration tests from mvn clean install -Pit pass. An alternative would be to use a try-catch block and Assertions.fail() the test if the exception is thrown.

PR Checklist:

  • Github Issue: [BUG] - Nondeterminism in unit test TestInvocationPublishModelFactory due to JSON parameter ordering #4946
  • Each commit in the pull request should have a meaningful subject line and body.
  • Format the pull request title like [SCB-XXX] Fixes bug in ApproximateQuantiles, where you replace SCB-XXX with the appropriate JIRA issue.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Run mvn clean install -Pit to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.

@anthonyx24
Copy link
Contributor Author

Apologies, I accidentally closed the PR by deleting the feature branch when trying to clean up some other branches. I've restored the branch and reopened the PR, feel free to take a look when possible. Thank you!

As a reference, two of my other PRs addressing the exact same issue have been merged:
#4971
#4969

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant