Feature Request / Improvement
Publish a JSON Schema for datasetConfig.yaml, generated from the DatasetConfig class rather than written by hand.
Why
Writing a valid datasetConfig is the first thing a new user does and the first place they get stuck: sourceFormat, targetFormats, tableBasePath, tableName, namespace, partitionSpec, and the right conversionSourceProviderClass. Today the only description of that shape is prose in the docs, so it drifts from the code silently.
A generated schema gives three things at once:
- Editors and IDEs validate and complete the file.
- Any agent or script can check a config before running a sync, instead of discovering problems from a stack trace.
- The documentation stops being a second source of truth that can go stale.
PR #829 includes a hand-written config-schema.md for exactly this purpose. That is the thing to replace with something generated.
Scope
- Generate a JSON Schema from
DatasetConfig and its nested types at build time.
- Publish it as a release artifact and at a stable URL, so
$schema references keep working.
- Fail the build when the schema is out of date with the model, so it cannot silently drift.
- Reference it from the docs in place of any hand-maintained field list.
Are you willing to submit PR?
Code of Conduct
Feature Request / Improvement
Publish a JSON Schema for
datasetConfig.yaml, generated from theDatasetConfigclass rather than written by hand.Why
Writing a valid
datasetConfigis the first thing a new user does and the first place they get stuck:sourceFormat,targetFormats,tableBasePath,tableName,namespace,partitionSpec, and the rightconversionSourceProviderClass. Today the only description of that shape is prose in the docs, so it drifts from the code silently.A generated schema gives three things at once:
PR #829 includes a hand-written
config-schema.mdfor exactly this purpose. That is the thing to replace with something generated.Scope
DatasetConfigand its nested types at build time.$schemareferences keep working.Are you willing to submit PR?
Code of Conduct