Improve make gen to avoid unnecessary file changes#6920
Open
jerrylin118 wants to merge 1 commit intoflyteorg:masterfrom
Open
Improve make gen to avoid unnecessary file changes#6920jerrylin118 wants to merge 1 commit intoflyteorg:masterfrom
jerrylin118 wants to merge 1 commit intoflyteorg:masterfrom
Conversation
- Gate go mod tidy behind RUN_GO_TIDY in go-gen.sh - Add idempotency check in go_generate workflow - Copy embedded swagger only when content changed - Pin buf image to v1.65.0 in generate_protos.sh - Run generate CI only when relevant paths change
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tracking issue
Related to #6845
Why are the changes needed?
make generatewas updating files such asgo.sum,go.mod,swagger.json, and lockfiles even when no protobuf definitions or generation inputs changed. This resulted in unnecessary CI diffs, noisy PRs, and confusion about what actually changed. These updates ensure generation only modifies files when inputs change and that the process is deterministic and idempotent.What changes were proposed in this pull request?
Gate
go mod tidybehindRUN_GO_TIDY(
boilerplate/flyte/golang_test_targets/go-gen.sh)go mod tidyis no longer run by default in the shared generate script.It runs only when explicitly enabled (e.g.
RUN_GO_TIDY=1 make generate).Prevents incidental
go.mod/go.sumchurn. Tidy remains available viamake go-tidyor the env var.Add idempotency check in CI
(
.github/workflows/go_generate.yml)After
DELTA_CHECK=true make generate, CI runsmake generateagain and fails if the working tree is dirty.Ensures deterministic generation and catches nondeterministic tools or environment differences.
Content-aware swagger copy
(
flyteidl/clients/go/assets/embed.go)The swagger file is copied only if content differs (
cmp -s ... || cp ...).Prevents no-op diffs and timestamp-only changes.
Pin
bufimage(
flyteidl/generate_protos.sh)Updated from implicit
latesttobufbuild/buf:v1.65.0for reproducible proto generation.Add path filters to generate CI
(
.github/workflows/checks.yml,.github/workflows/flyteidl-checks.yml)Generate checks run only when relevant paths change, reducing unnecessary CI runs and unrelated churn.
How was this patch tested?
make generatelocally in affected components and confirmed no unexpected diffs when inputs were unchanged.RUN_GO_TIDY=1 make generatestill runsgo mod tidy.Check all the applicable boxes