-
Notifications
You must be signed in to change notification settings - Fork 14.8k
KAFKA-19174 Gradle version upgrade 8 -->> 9 #19513
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
A label of 'needs-attention' was automatically added to this PR in order to raise the |
|
This PR is being marked as stale since it has not had any activity in 90 days. If you If you are having difficulty finding a reviewer, please reach out on the [mailing list](https://kafka.apache.org/contact). If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed. |
|
A label of 'needs-attention' was automatically added to this PR in order to raise the |
|
Update: project compilation works as expected (when powered by Gradle 9) but binary release creation fails. |
|
A label of 'needs-attention' was automatically added to this PR in order to raise the |
|
@dejan2609 any update? |
|
A label of 'needs-attention' was automatically added to this PR in order to raise the |
|
@chia7712 |
|
@dejan2609 thanks! |
|
Pending: I will also explain second problem (specific to Gradle 9, that is). |
|
A label of 'needs-attention' was automatically added to this PR in order to raise the |
|
@omkreddy The same goes for a current trunk state (no errors whatsoever): |
All in all: I see no errors here @omkreddy |
|
@dejan2609 I think the command/task (unitTest) is not running the tests. Normally we should see test execution output like On trunk: |
@omkreddy Are you referring to the command(s) that I used, i.e. See here for more details: https://docs.gradle.org/9.1.0/userguide/command_line_interface.html#sec:command_line_logging Now, even your output posted above shows no errors:
📢 Edit/update (after few days or so): I was wrong, @omkreddy was right: I was searching for errors and while it's true that there are none of them tests are still skipped |
|
@omkreddy Thanks for your report. We will take a look and test other commands to ensure everything works correctly |
Can we add a CI task that validates these commands? I'd assume that the goal should be to avoid regressions for these moving forward A separate workflow would be fine, I think |
|
I will file a PR to fix the task
there are many custom tasks in kafka, but yes it is worth having the CI - I have opened https://issues.apache.org/jira/browse/KAFKA-19780 to discuss that later. |
|
Another task, |
The simple fix is to add |
|
I noticed another error that was introduced by #12280 and it unrelated to Gradle9 It can be fixed by changing the |
…ded with a `--no-parallel` switch (#20683) Prologue: #19513 (comment) Also related to: #20652 @chia7712 I double-checked custom gradle commands in `README.md` and AFAIK we should be fine now. Reviewers: Chia-Ping Tsai <[email protected]>
…-->> 9 (#20684) from: #19513 (comment) 1. Fix the task `unitTest` and `integrationTest`; 2. Change the `name` to a method call `name()` for `KeepAliveMode`. Reviewers: Ken Huang <[email protected]>, dejan2609 <[email protected]>, Chia-Ping Tsai <[email protected]>
Extends from: #19513 details: - scala-logging_2.13: Updated from 3.9.5 to 3.9.6 - org.owasp.dependencycheck: Updated from 12.1.3 to 12.1.8 - org.scoverage: Updated from 8.0.3 to 8.1 - com.diffplug.spotless: Updated from 7.2.1 to 8.0.0 - scalafmt: Updated from 3.7.14 to 3.10.0 - grgit: Updated from 4.1.1 to 5.3.3 - jacoco: Updated from 0.8.13 to 0.8.14 - scoverage: Updated from 2.0.11 to 2.3.0 Reviewers: Chia-Ping Tsai <[email protected]>
According to the [discussion](#19513 (comment)) in #19513 , add a new checksum verification process to determine whether a new wrapper JAR needs to be downloaded. This prevents developers from running into incompatibility issues when using an outdated wrapper JAR after a Gradle upgrade. Reviewers: Chia-Ping Tsai <[email protected]>
According to the [discussion](apache#19513 (comment)) in apache#19513 , add a new checksum verification process to determine whether a new wrapper JAR needs to be downloaded. This prevents developers from running into incompatibility issues when using an outdated wrapper JAR after a Gradle upgrade. Reviewers: Chia-Ping Tsai <[email protected]>
Related discussion:#20561 (comment) This PR primarily upgrades release tooling to JDK 25, and includes two fixes identified while dry-running `release.py` locally. 1. JDK upgrade: Switches `release.py` to run under JDK 25. 2. Pre‑req check false negative in git clean‑state verification task * When running `release.py` with a clean working tree, the script incorrectly failed the pre‑requisite check. ```bash $ git status On branch MINOR-1014 nothing to commit, working tree clean $ python release.py ... FAILURE: Pre-requisite not met: Verifying that you have no unstaged git changes. ``` * Root cause: the checks used `has_unstaged_changes()` / `has_staged_changes()` in a way that treated success as “has changes”. The fix inverts the logic to assert **no** changes. 3. Gradle 9 aggregatedJavadoc failure * With **Gradle 9**, `aggregatedJavadoc` currently fails in parallel mode. ``` FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':aggregatedJavadoc'. > Resolution of the configuration ':connect:compileClasspath' was attempted without an exclusive lock. This is unsafe and not allowed. ``` * The change adds `--no-parallel` for the aggregated Javadoc step to avoid build failures (discussion: #19513 (comment)) Reviewers: Mickael Maison <[email protected]>, Ming-Yen Chung <[email protected]>, kuoche1712003 <[email protected]>, Chia-Ping Tsai <[email protected]>
Related discussion:apache#20561 (comment) This PR primarily upgrades release tooling to JDK 25, and includes two fixes identified while dry-running `release.py` locally. 1. JDK upgrade: Switches `release.py` to run under JDK 25. 2. Pre‑req check false negative in git clean‑state verification task * When running `release.py` with a clean working tree, the script incorrectly failed the pre‑requisite check. ```bash $ git status On branch MINOR-1014 nothing to commit, working tree clean $ python release.py ... FAILURE: Pre-requisite not met: Verifying that you have no unstaged git changes. ``` * Root cause: the checks used `has_unstaged_changes()` / `has_staged_changes()` in a way that treated success as “has changes”. The fix inverts the logic to assert **no** changes. 3. Gradle 9 aggregatedJavadoc failure * With **Gradle 9**, `aggregatedJavadoc` currently fails in parallel mode. ``` FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':aggregatedJavadoc'. > Resolution of the configuration ':connect:compileClasspath' was attempted without an exclusive lock. This is unsafe and not allowed. ``` * The change adds `--no-parallel` for the aggregated Javadoc step to avoid build failures (discussion: apache#19513 (comment)) Reviewers: Mickael Maison <[email protected]>, Ming-Yen Chung <[email protected]>, kuoche1712003 <[email protected]>, Chia-Ping Tsai <[email protected]>
List of changes: - prerequisite Jira ticket: - [KAFKA-19591](https://issues.apache.org/jira/browse/KAFKA-19591) - mandatory version upgrades: - Gradle version: 8.14.3 -->> 9.1.0 - Gradle Shadow plugin: 8.3.6 -->> 8.3.9 - Gradle dependencycheck plugin: 8.2.1 -->> 12.1.3 - Gradle spotbugs plugin: 6.2.3 -->> 6.2.5 - Gradle spotless plugin: 6.25.0 -->> 7.2.1 - build logic will be refactored to accommodate Gradle 9 breaking changes - (optional): a dozen of Gradle plugins versions will also be upgraded - other JIRA tickets that had to be solved all along: - [KAFKA-16801](https://issues.apache.org/jira/browse/KAFKA-16801) - [KAFKA-19654](https://issues.apache.org/jira/browse/KAFKA-19654) **Related links:** - https://gradle.org/whats-new/gradle-9 - https://github.com/gradle/gradle/releases/tag/v9.0.0 - https://docs.gradle.org/9.0.0/release-notes.html - https://docs.gradle.org/9.0.0/userguide/upgrading_major_version_9.html - https://docs.gradle.org/9.1.0/release-notes.html Notes: - new Gradle version brings up some breaking changes, as always 😃 - Kafka build with Gradle 9 has same issues as other projects: - redhat-developer/vscode-java#4018 - gradle/gradle#32597 Reviewers: Chia-Ping Tsai <[email protected]>
…ome issues with Gradle 9+ are solved) (apache#20652) Extends from: apache#19513 Note: in Gradle 9+ we have to add a switch like this: ``` ./gradlew dependencyUpdates --no-parallel ``` Related link: https://github.com/ben-manes/gradle-versions-plugin/releases/tag/v0.53.0 Reviewers: Chia-Ping Tsai <[email protected]>
…ded with a `--no-parallel` switch (apache#20683) Prologue: apache#19513 (comment) Also related to: apache#20652 @chia7712 I double-checked custom gradle commands in `README.md` and AFAIK we should be fine now. Reviewers: Chia-Ping Tsai <[email protected]>
…-->> 9 (apache#20684) from: apache#19513 (comment) 1. Fix the task `unitTest` and `integrationTest`; 2. Change the `name` to a method call `name()` for `KeepAliveMode`. Reviewers: Ken Huang <[email protected]>, dejan2609 <[email protected]>, Chia-Ping Tsai <[email protected]>
Extends from: apache#19513 details: - scala-logging_2.13: Updated from 3.9.5 to 3.9.6 - org.owasp.dependencycheck: Updated from 12.1.3 to 12.1.8 - org.scoverage: Updated from 8.0.3 to 8.1 - com.diffplug.spotless: Updated from 7.2.1 to 8.0.0 - scalafmt: Updated from 3.7.14 to 3.10.0 - grgit: Updated from 4.1.1 to 5.3.3 - jacoco: Updated from 0.8.13 to 0.8.14 - scoverage: Updated from 2.0.11 to 2.3.0 Reviewers: Chia-Ping Tsai <[email protected]>
According to the [discussion](apache#19513 (comment)) in apache#19513 , add a new checksum verification process to determine whether a new wrapper JAR needs to be downloaded. This prevents developers from running into incompatibility issues when using an outdated wrapper JAR after a Gradle upgrade. Reviewers: Chia-Ping Tsai <[email protected]>
Related discussion:apache#20561 (comment) This PR primarily upgrades release tooling to JDK 25, and includes two fixes identified while dry-running `release.py` locally. 1. JDK upgrade: Switches `release.py` to run under JDK 25. 2. Pre‑req check false negative in git clean‑state verification task * When running `release.py` with a clean working tree, the script incorrectly failed the pre‑requisite check. ```bash $ git status On branch MINOR-1014 nothing to commit, working tree clean $ python release.py ... FAILURE: Pre-requisite not met: Verifying that you have no unstaged git changes. ``` * Root cause: the checks used `has_unstaged_changes()` / `has_staged_changes()` in a way that treated success as “has changes”. The fix inverts the logic to assert **no** changes. 3. Gradle 9 aggregatedJavadoc failure * With **Gradle 9**, `aggregatedJavadoc` currently fails in parallel mode. ``` FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':aggregatedJavadoc'. > Resolution of the configuration ':connect:compileClasspath' was attempted without an exclusive lock. This is unsafe and not allowed. ``` * The change adds `--no-parallel` for the aggregated Javadoc step to avoid build failures (discussion: apache#19513 (comment)) Reviewers: Mickael Maison <[email protected]>, Ming-Yen Chung <[email protected]>, kuoche1712003 <[email protected]>, Chia-Ping Tsai <[email protected]>

Related JIRA ticket: https://issues.apache.org/jira/browse/KAFKA-19174
List of changes:
9.0.09.1.09.0.28.3.9changes
Related links:
Notes:
Reviewers: Chia-Ping Tsai [email protected]