[920] Address the review comments on the 0.4.0 Spark runtime docs - #922
Open
rangareddy wants to merge 1 commit into
Open
[920] Address the review comments on the 0.4.0 Spark runtime docs#922rangareddy wants to merge 1 commit into
rangareddy wants to merge 1 commit into
Conversation
PR apache#913 was merged before the review comments left on it were addressed. This applies them. website/docs/how-to-spark-runtime.md - Lead with "Running a sync as its own job" and move the in-job section after it. The page previously opened on a spark-submit that referenced an application jar nothing had introduced yet. - Split the in-job section into writing, building and running the job, so the code appears before the command that submits it, and document the provided-scope dependency needed to compile against the runtime classes. - Add a Prerequisites section pointing at the Quickstart for creating the source table every example on the page syncs. - Give XTableSparkSync's two modes a subsection each, single table from command line options and several tables from a --datasetconfig file. - Use the Quickstart's local file:///tmp/hudi-dataset/people paths rather than s3://example-warehouse/db/orders, so both pages can be followed with the same setup. - Promote the paragraph on which Delta implementation is used to a note. - State the supported Spark versions as 3.4.x and 3.5.x. "3.5.x and newer" read as a promise of Spark 4 support: the 0.4.0 build sets spark.version 3.4.2 with hudi-spark3.4-bundle and iceberg-spark-runtime-3.4, and ITXTableSparkRuntimeBundle covers the 3.4 and 3.5 lines only. website/releases/downloads.mdx - Link each release heading to its release notes page. - Move Downloads to the top of the Releases sidebar. It rendered below every release entry because those use negative sidebar_position values (-4, -3, -2) while downloads.mdx used 1. website/releases/release-0.4.0-incubating.mdx - Call out Spark 3.4.x and 3.5.x support instead of noting it mid-sentence.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #920.
What is this?
#913 added the 0.4.0 release notes and the Spark runtime quickstart. It was merged at
2026-08-25T15:20:10Z, about ten hours after a round of review comments was left on it, and thosecomments were never applied. This PR applies them. No new content beyond what the review asked for.
Changes
website/docs/how-to-spark-runtime.mdorders-job.jarappears without contextspark-submitthat submits it. Building the job documents theprovided-scope dependency needed to compile against the runtime classesfile:///tmp/hudi-dataset/people, the table the Quickstart createsXTableSparkSyncnow has a subsection per mode: "Syncing a single table" and "Syncing several tables from a config file":::note Which Delta implementation you get3.4.xand3.5.xexplicitly (see Verification below)demo/spark-runtimelink 404smainat review time; merging #913 created it and the link now resolveswebsite/releases/downloads.mdx### Release x.y.z-incubatingheading now links to its release notes page.pages use negative
sidebar_positionvalues (-4,-3,-2) whiledownloads.mdxused1.The existing "Release Notes:" bullet under each heading is left in place; it is now somewhat
redundant with the linked heading, and I am happy to drop it if reviewers prefer.
website/releases/release-0.4.0-incubating.mdx:::notecallout.Verification
Facts were read from the
0.4.0-incubatingtag rather thanmain, becausextable-spark-runtimeexists only on the 0.4.x release line (
branch-0.4and the release branches);main'spom.xmldoes not list the module.
The Spark version claim specifically:
pom.xmlon the tag setsspark.version3.4.2andspark.version.prefix3.4, with no3.5anywhere in the file and no Spark profile;xtable-spark-runtime/pom.xmldepends onhudi-spark${spark.version.prefix}-bundleandiceberg-spark-runtime-${spark.version.prefix};ITXTableSparkRuntimeBundledocuments the 3.4 and 3.5 lines only.Nothing supports Spark 4, so "3.5.x and newer" was removed.
Site checks, run locally on this branch:
npm run buildsucceeds with no new warnings (the remaining ones are pre-existing blogtruncation-marker and inline-author warnings).
200.docusaurus.config.jssetsonBrokenLinks: 'ignore', so a broken internal link would not failthe build.
block titles, the linked release headings, and the Releases sidebar now ordered Downloads,
0.4.0, 0.3.0, 0.2.0.
Out of scope
While verifying comments 6 and 7 I found two further inaccuracies in the same "Spark version
support" section, which this PR deliberately leaves alone so it stays scoped to the review
comments. Both are filed as #921:
That is true only for the
XTableSparkSyncCLI, which auto-enables it viaisSparkAtLeast35.The in-job API the page leads with has no version detection:
XTableSyncService.syncreadsspec.isUseDeltaKernel(), andTableSyncSpec.useDeltaKerneldefaults tofalse.xtable-spark-runtime/pom.xmldepends on
delta-core,delta-kernel-apianddelta-kernel-defaults. It contains nodelta-standalonedependency.