Skip to content

Commit 4677e24

Browse files
authored
Merge pull request #191 from tampakrap/theo/docs_examples
docs(examples): fill the outputs of the examples with examples-gen, and add the new examples
2 parents 43714a0 + ef559a0 commit 4677e24

25 files changed

Lines changed: 950 additions & 262 deletions

‎.github/renovate.json5‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,18 @@
6464
datasourceTemplate: 'github-releases',
6565
depNameTemplate: 'golangci/golangci-lint',
6666
},
67+
{
68+
customType: 'regex',
69+
description: 'Bump dyff version in the Earthfile',
70+
managerFilePatterns: [
71+
'/^Earthfile$/',
72+
],
73+
matchStrings: [
74+
'ARG DYFF_VERSION=(?<currentValue>.*?)\\n',
75+
],
76+
datasourceTemplate: 'github-releases',
77+
depNameTemplate: 'homeport/dyff',
78+
},
6779
{
6880
customType: 'regex',
6981
description: 'Bump E2E Crossplane v1 version in Earthfile',

‎CONTRIBUTING.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ If you change example files that affect e2e test output, regenerate the expected
3434
earthly -P +e2e-regen-expected
3535
```
3636

37+
The outputs of the examples in `examples/README.md` are generated too, see [examples-gen](docs/contributing/examples-gen.md) to regenerate them.
38+
3739
## PR Process
3840

3941
1. Rebase on `upstream/main`

‎Earthfile‎

Lines changed: 65 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -246,9 +246,6 @@ e2e-v2:
246246

247247
# e2e-regen-expected runs all three CLI tiers in parallel, merges artifacts, runs cleanup, then exports.
248248
e2e-regen-expected:
249-
BUILD +e2e-regen-expected-v1
250-
BUILD +e2e-regen-expected-v2legacy
251-
BUILD +e2e-regen-expected-v2
252249
FROM alpine:3.24
253250
RUN apk add --no-cache bash
254251
WORKDIR /work
@@ -265,7 +262,6 @@ e2e-regen-expected:
265262
SAVE ARTIFACT expected AS LOCAL tests/e2e/expected
266263

267264
# e2e-regen-expected-v1 runs the regen script for Crossplane v1 only; used in parallel then merged.
268-
# Full target (not BUILD-only) so COPY +e2e-regen-expected-v1/expected works in e2e-regen-expected.
269265
e2e-regen-expected-v1:
270266
ARG TARGETARCH
271267
ARG TARGETOS
@@ -320,3 +316,68 @@ e2e-regen-expected-v2:
320316
END
321317
SAVE ARTIFACT expected
322318

319+
# go-build-examples-gen builds the tool that fills the outputs of the examples (see docs/contributing/examples-gen.md).
320+
go-build-examples-gen:
321+
FROM +go-modules
322+
CACHE --id go-build --sharing shared /root/.cache/go-build
323+
COPY --dir cmd/ internal/ .
324+
RUN CGO_ENABLED=0 go build -o examples-gen ./cmd/examples-gen
325+
SAVE ARTIFACT examples-gen
326+
327+
# dyff-setup is used by other targets to setup the dyff CLI, which the hooks of the examples call.
328+
dyff-setup:
329+
ARG DYFF_VERSION=v1.12.0
330+
ARG NATIVEPLATFORM
331+
FROM --platform=${NATIVEPLATFORM} alpine:3.24
332+
RUN apk add --no-cache bash curl jq
333+
RUN curl -sL https://raw.githubusercontent.com/homeport/dyff/main/scripts/download-latest.sh | bash -s ${DYFF_VERSION}
334+
SAVE ARTIFACT /usr/local/bin/dyff
335+
336+
# regen-examples regenerates examples/README.md outputs, with the Crossplane CLI v2 and v1, and merges them (see docs/contributing/examples-gen.md)
337+
regen-examples:
338+
ARG REGENERATE_ALL
339+
ARG REGENERATE_MATCH
340+
FROM alpine:3.24
341+
RUN apk add --no-cache bash git
342+
WORKDIR /work
343+
COPY examples/README.md README.md
344+
COPY (+regen-examples-v2/README.v2.md --REGENERATE_ALL=$REGENERATE_ALL --REGENERATE_MATCH=$REGENERATE_MATCH) README.v2.md
345+
COPY (+regen-examples-v1/README.v1.md --REGENERATE_ALL=$REGENERATE_ALL --REGENERATE_MATCH=$REGENERATE_MATCH) README.v1.md
346+
COPY cmd/examples-gen/merge-examples.sh .
347+
RUN ./merge-examples.sh
348+
SAVE ARTIFACT README.md AS LOCAL examples/README.md
349+
350+
# regen-examples-v2 regenerates the outputs of the examples/README.md blocks that have no tag, with the current Crossplane CLI
351+
regen-examples-v2:
352+
ARG REGENERATE_ALL
353+
ARG REGENERATE_MATCH
354+
FROM +regen-examples-run --REGENERATE_ALL=$REGENERATE_ALL --REGENERATE_MATCH=$REGENERATE_MATCH --CROSSPLANE_CLI_VERSION=$E2E_CROSSPLANE_CLI --TAG=
355+
SAVE ARTIFACT /Users/myuser/repos/xprin/examples/README.md README.v2.md
356+
357+
# regen-examples-v1 regenerates the outputs of the examples/README.md blocks that are tagged v1, with the Crossplane CLI v1
358+
regen-examples-v1:
359+
ARG REGENERATE_ALL
360+
ARG REGENERATE_MATCH
361+
FROM +regen-examples-run --REGENERATE_ALL=$REGENERATE_ALL --REGENERATE_MATCH=$REGENERATE_MATCH --CROSSPLANE_CLI_VERSION=$E2E_CROSSPLANE_V1 --TAG=v1
362+
SAVE ARTIFACT /Users/myuser/repos/xprin/examples/README.md README.v1.md
363+
364+
# regen-examples-run is used by the regen-examples targets to run examples-gen with a Crossplane CLI version
365+
regen-examples-run:
366+
ARG REGENERATE_ALL
367+
ARG REGENERATE_MATCH
368+
ARG TAG
369+
ARG CROSSPLANE_CLI_VERSION
370+
FROM earthly/dind:alpine-3.20-docker-26.1.5-r0
371+
ENV REGENERATE_ALL=$REGENERATE_ALL
372+
ENV REGENERATE_MATCH=$REGENERATE_MATCH
373+
ENV TAG=$TAG
374+
RUN apk add --no-cache bash yq
375+
COPY (+crossplane-cli-setup/crossplane --CROSSPLANE_CLI_VERSION=$CROSSPLANE_CLI_VERSION) /usr/local/bin/crossplane
376+
COPY +go-build/xprin /usr/local/bin/xprin
377+
COPY +go-build-examples-gen/examples-gen /usr/local/bin/examples-gen
378+
COPY +dyff-setup/dyff /usr/local/bin/dyff
379+
WORKDIR /Users/myuser/repos/xprin
380+
COPY --dir examples/ ./
381+
WITH DOCKER
382+
RUN (xprin test examples/mytests/5_chained_tests/example4_cross-composition-chaining_xprin.yaml > /dev/null || true) && examples-gen $([ "$REGENERATE_ALL" = "true" ] && echo --regenerate-all) ${REGENERATE_MATCH:+"--regenerate-match=$REGENERATE_MATCH"} ${TAG:+"--tag=$TAG"} examples/README.md
383+
END

‎cmd/examples-gen/main.go‎

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
/*
2+
Copyright 2026 The Crossplane Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
// examples-gen fills the outputs of the examples of markdown files, see docs/contributing/examples-gen.md.
18+
package main
19+
20+
import (
21+
"errors"
22+
"fmt"
23+
"os"
24+
"os/exec"
25+
"strings"
26+
27+
"github.com/alecthomas/kong"
28+
)
29+
30+
const (
31+
prompt = "➜ "
32+
placeholder = "<output>"
33+
codeBlock = "```"
34+
tagPrefix = "<!-- examples-gen: "
35+
tagSuffix = " -->"
36+
)
37+
38+
// cli is the command line of the tool.
39+
type cli struct {
40+
Files []string `arg:"" help:"Markdown files whose examples are filled." type:"existingfile"`
41+
RegenerateAll bool `help:"Regenerate the outputs of every command, not only the ones that have a placeholder." name:"regenerate-all" xor:"regenerate"`
42+
RegenerateMatch string `help:"Regenerate the outputs of the commands that contain this text." name:"regenerate-match" xor:"regenerate"`
43+
Tag string `help:"Fill the code blocks that have this tag, instead of the ones that have none." name:"tag"`
44+
}
45+
46+
func main() {
47+
var args cli
48+
49+
kong.Parse(&args,
50+
kong.Name("examples-gen"),
51+
kong.Description("Fill the outputs of the examples of markdown files."),
52+
kong.UsageOnError(),
53+
)
54+
55+
update := func(content string) (string, error) {
56+
// To regenerate an output, the placeholder goes in its place first
57+
if args.RegenerateAll || args.RegenerateMatch != "" {
58+
content = clearOutputs(content, args.RegenerateMatch, args.Tag)
59+
}
60+
61+
return fill(content, args.Tag, runCommand)
62+
}
63+
64+
for _, path := range args.Files {
65+
if err := updateFile(path, update); err != nil {
66+
fmt.Fprintf(os.Stderr, "examples-gen: %s: %v\n", path, err)
67+
os.Exit(1)
68+
}
69+
}
70+
}
71+
72+
// updateFile replaces the content of the file with what update returns for it.
73+
func updateFile(path string, update func(content string) (string, error)) error {
74+
data, err := os.ReadFile(path) //nolint:gosec // G304 - the path is an argument of the maintainer who runs the tool
75+
if err != nil {
76+
return fmt.Errorf("read: %w", err)
77+
}
78+
79+
filled, err := update(string(data))
80+
if err != nil {
81+
return err
82+
}
83+
84+
info, err := os.Stat(path)
85+
if err != nil {
86+
return fmt.Errorf("stat: %w", err)
87+
}
88+
89+
if err := os.WriteFile(path, []byte(filled), info.Mode().Perm()); err != nil {
90+
return fmt.Errorf("write: %w", err)
91+
}
92+
93+
return nil
94+
}
95+
96+
// fill puts the output of run in place of the placeholder after each command, in the code blocks that have the tag.
97+
func fill(content, tag string, run func(command string) (string, error)) (string, error) {
98+
lines := strings.Split(content, "\n")
99+
filled := make([]string, 0, len(lines))
100+
inFence := false
101+
tagged := false
102+
103+
for i := 0; i < len(lines); i++ {
104+
line := lines[i]
105+
filled = append(filled, line)
106+
107+
if strings.HasPrefix(line, codeBlock) {
108+
inFence = !inFence
109+
tagged = inFence && i > 0 && blockTag(lines[i-1]) == tag
110+
111+
continue
112+
}
113+
114+
command, isCommand := strings.CutPrefix(line, prompt)
115+
if !inFence || !tagged || !isCommand || i+1 >= len(lines) || lines[i+1] != placeholder {
116+
continue
117+
}
118+
119+
fmt.Fprintf(os.Stderr, "running: %s\n", command)
120+
121+
output, err := run(command)
122+
if err != nil {
123+
return "", fmt.Errorf("%q: %w", command, err)
124+
}
125+
126+
if output != "" {
127+
filled = append(filled, strings.Split(output, "\n")...)
128+
}
129+
130+
// The placeholder is replaced by the output
131+
i++
132+
}
133+
134+
return strings.Join(filled, "\n"), nil
135+
}
136+
137+
// clearOutputs puts the placeholder in place of the outputs of the commands that contain match, in the tagged blocks.
138+
func clearOutputs(content, match, tag string) string {
139+
lines := strings.Split(content, "\n")
140+
cleared := make([]string, 0, len(lines))
141+
inFence := false
142+
tagged := false
143+
144+
for i := 0; i < len(lines); i++ {
145+
line := lines[i]
146+
cleared = append(cleared, line)
147+
148+
if strings.HasPrefix(line, codeBlock) {
149+
inFence = !inFence
150+
tagged = inFence && i > 0 && blockTag(lines[i-1]) == tag
151+
152+
continue
153+
}
154+
155+
command, isCommand := strings.CutPrefix(line, prompt)
156+
if !inFence || !tagged || !isCommand || !strings.Contains(command, match) {
157+
continue
158+
}
159+
160+
cleared = append(cleared, placeholder)
161+
162+
// The output goes on until the next command, or the end of the block
163+
next := i + 1
164+
for next < len(lines) && !strings.HasPrefix(lines[next], prompt) && !strings.HasPrefix(lines[next], codeBlock) {
165+
next++
166+
}
167+
168+
if next < len(lines) && strings.HasPrefix(lines[next], prompt) && next > i+1 && lines[next-1] == "" {
169+
cleared = append(cleared, "")
170+
}
171+
172+
i = next - 1
173+
}
174+
175+
return strings.Join(cleared, "\n")
176+
}
177+
178+
// blockTag returns the tag of the line before a code block, or an empty string when the line is not a tag.
179+
func blockTag(line string) string {
180+
tag, found := strings.CutPrefix(line, tagPrefix)
181+
if !found {
182+
return ""
183+
}
184+
185+
return strings.TrimSuffix(tag, tagSuffix)
186+
}
187+
188+
// runCommand returns the output of the command in a shell, ignoring its exit status: a failing test is a valid example.
189+
func runCommand(command string) (string, error) {
190+
cmd := exec.Command("sh", "-c", command) //nolint:gosec // The commands are the ones of the examples, written by the maintainers
191+
192+
output, err := cmd.CombinedOutput()
193+
194+
var exitErr *exec.ExitError
195+
if err != nil && !errors.As(err, &exitErr) {
196+
return "", err
197+
}
198+
199+
return strings.TrimRight(string(output), "\n"), nil
200+
}

‎cmd/examples-gen/merge-examples.sh‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
#!/bin/bash
2+
# Merge examples/README.md outputs of the Crossplane CLI v2 and v1 (run inside the container of the
3+
# regen-examples target, see docs/contributing/examples-gen.md)
4+
#
5+
# In the working directory:
6+
# README.md the file before the regeneration
7+
# README.v2.md the file with the untagged blocks regenerated
8+
# README.v1.md the file with the blocks tagged v1 regenerated
9+
#
10+
# Each of them changed other blocks of README.md, so a three-way merge puts the changes together. The result goes to
11+
# README.md, and the script fails when the changes overlap.
12+
13+
set -euo pipefail
14+
15+
merged="$(mktemp)"
16+
17+
if ! git merge-file -p README.v2.md README.md README.v1.md > "${merged}"; then
18+
echo "merge-examples.sh: README.v2.md and README.v1.md changed the same lines of README.md"
19+
exit 1
20+
fi
21+
22+
mv "${merged}" README.md

‎docs/contributing/examples-gen.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# examples-gen
2+
3+
Fills the outputs of the examples in `examples/README.md`.
4+
5+
In a code block, a command starts with `➜ ` and its output goes below it. When the line after a command is `<output>`, the tool runs the command and puts its output (stdout and stderr) in place of the placeholder:
6+
7+
```
8+
➜ xprin test examples/mytests/1_simple_tests/example1_using-xr_xprin.yaml -v
9+
<output>
10+
```
11+
12+
A block without a placeholder is not touched. To regenerate an output, replace it with `<output>`, or use the options below.
13+
14+
Some outputs depend on the Crossplane CLI (for example, the ones of the Crossplane CLI v1). A code block with a tag, in a comment on the line right before it, is filled only when the tool runs with the same `--tag`, and a block without a tag only when it runs without one:
15+
16+
````
17+
<!-- examples-gen: v1 -->
18+
```
19+
➜ xprin check
20+
<output>
21+
```
22+
````
23+
24+
## Run it
25+
26+
From the root of the repository, with `xprin`, the Crossplane CLI, Docker and `yq` available (`xprin` has to be in the `PATH`):
27+
28+
```shell
29+
go build -o /tmp/bin/xprin ./cmd/xprin
30+
PATH=/tmp/bin:$PATH go run ./cmd/examples-gen examples/README.md # fill the placeholders
31+
PATH=/tmp/bin:$PATH go run ./cmd/examples-gen --regenerate-all examples/README.md # regenerate every output
32+
PATH=/tmp/bin:$PATH go run ./cmd/examples-gen --regenerate-match=hooks examples/README.md # the commands that contain "hooks"
33+
```
34+
35+
Add `--tag=v1` to fill the blocks tagged `v1` instead of the untagged ones, with the Crossplane CLI v1 in the `PATH`. `--regenerate-all` and `--regenerate-match` can't be used together. The commands run in the working directory and can change files (for example `update-goldens`), so look at the diff afterwards.
36+
37+
## Run it with Earthly
38+
39+
Docker-in-Docker, with the Crossplane CLIs and `yq`. Only `examples/README.md` is exported:
40+
41+
```shell
42+
earthly --allow-privileged +regen-examples # fill the placeholders
43+
earthly --allow-privileged +regen-examples --REGENERATE_ALL=true # regenerate every output
44+
earthly --allow-privileged +regen-examples --REGENERATE_MATCH=hooks # the commands that contain "hooks"
45+
```
46+
47+
It runs the examples with the Crossplane CLI v2 (the untagged blocks), and then with v1 (the blocks that are tagged).

0 commit comments

Comments
 (0)