Skip to content

Use lychee for docbuilding - #898

Open
AVHopp wants to merge 10 commits into
mainfrom
docs/separate_linkcheck_new
Open

Use lychee for docbuilding#898
AVHopp wants to merge 10 commits into
mainfrom
docs/separate_linkcheck_new

Conversation

@AVHopp

@AVHopp AVHopp commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

This PR separates the linkcheck job from the actual doc building. It does so by creating a new dedicated job for it in the CI and re-organizing the code.

External link checking

  • Replace Sphinx's linkcheck builder with lychee for external URL validation
  • Add a standalone lychee CI job that scans baybe/, docs/, examples/, and top-level markdown files
  • Configure exclusions in lychee.toml (migrated from linkcheck_ignore in docs/conf.py)
  • Remove the now-unused linkcheck_ignore from docs/conf.py

Cross-reference checking

  • Switch the Sphinx build in check_links.py from linkcheck to dummy builder, which validates internal cross-references (:class:, :func:, :meth:, intersphinx) without rendering HTML
  • The dummy builder runs without -n/-W flags because it cannot fully resolve intersphinx references (e.g. ArrayLike), which would cause false positives; the html build with -n -W checks those

Naming cleanup

  • Rename things related to check_links to change_crossrefs

CI changes

  • Add lychee job to ci.yml, gating build-docs on both lint and lychee

Misc

  • The CI job shows all links that have been checked or excluded. Excluded links are the internal ones (as those are handled by sphinx). At the end of that job there is also a short summary (see screenshots)
  • We would also be able use lychee for internal links but since the sphinx html build already does this, there is no further speed-up that we can gain here.
  • Up-to-date version of the fork is available at https://avhopp.github.io/baybe_dev/latest/ (built after my latest commits answering to the first review of @Scienfitz )
image image

Stack created with GitHub Stacks CLIGive Feedback 💬

Copilot AI lite review requested due to automatic review settings August 18, 2026 15:06
Comment thread .github/workflows/ci.yml Outdated
Comment thread CHANGELOG.md Outdated
Comment thread docs/scripts/check_links.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR restructures BayBE’s documentation CI so external link checking is run as a dedicated step/job instead of being embedded in the documentation build script, and it tunes Sphinx linkcheck settings to make the new gating behavior more reliable.

Changes:

  • Adds a dedicated linkcheck tox environment and wires it into GitHub Actions (including a separate CI job).
  • Removes linkcheck logic/flags from docs/scripts/build_documentation.py and enables parallel Sphinx builds.
  • Configures Sphinx linkcheck behavior (ignore patterns + timeout/retries) for more stable CI runs.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tox.ini Adds testenv:linkcheck and updates docs-quickbuild to match the new separation.
docs/scripts/check_links.py Updates linkcheck utility and adds parallel build flag.
docs/scripts/build_documentation.py Removes linkcheck flag/logic and enables parallel Sphinx HTML build.
docs/conf.py Adds linkcheck ignore/timeout/retry settings to reduce flaky failures.
CHANGELOG.md Notes the new documentation linkcheck job.
.github/workflows/docs.yml Runs linkcheck explicitly before building/publishing docs.
.github/workflows/ci.yml Introduces a dedicated linkcheck job and makes docs build depend on it.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread docs/scripts/check_links.py
Comment thread CHANGELOG.md Outdated
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch from 51ac6a5 to be0aaa7 Compare August 19, 2026 07:08
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch from be0aaa7 to 297738c Compare August 19, 2026 07:12
Comment thread .github/workflows/ci.yml
@AVHopp
AVHopp marked this pull request as draft August 20, 2026 08:56
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch from 3e48b15 to 297738c Compare August 20, 2026 09:38
@AVHopp AVHopp closed this Aug 20, 2026
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch from 297738c to 24125fa Compare August 20, 2026 09:49
@AVHopp AVHopp reopened this Aug 20, 2026
@AVHopp
AVHopp changed the base branch from docs/refactor-apidoc_new to docs/parallelize_example_execution August 20, 2026 10:45
@AVHopp
AVHopp marked this pull request as ready for review August 20, 2026 10:46
@AVHopp
AVHopp marked this pull request as draft August 20, 2026 10:49
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch from 56edeec to c4068b9 Compare August 20, 2026 11:01
@AVHopp
AVHopp marked this pull request as ready for review August 20, 2026 11:02
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch from c4068b9 to 9e3a794 Compare August 20, 2026 11:18
@AVHopp
AVHopp marked this pull request as draft August 20, 2026 13:32
@AVHopp AVHopp self-assigned this Aug 20, 2026
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch 2 times, most recently from fa70d27 to 14e9af8 Compare August 31, 2026 08:13
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch from a091454 to d206f85 Compare September 1, 2026 14:42
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch 3 times, most recently from f453363 to 22d45fe Compare September 1, 2026 15:21
@AdrianSosic AdrianSosic added the documentation Improvements or additions to documentation label Sep 7, 2026
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch from 22d45fe to 4622a11 Compare September 11, 2026 10:22
Base automatically changed from docs/parallelize_example_execution to main September 11, 2026 11:05
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch from 4622a11 to 98fd9e4 Compare September 11, 2026 11:05
@AVHopp AVHopp changed the title Separate linkcheck job from doc building Use lychee for docbuilding Sep 11, 2026
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch 2 times, most recently from 6cb1dd7 to 8eaa054 Compare September 11, 2026 12:00
Checking of external links is no longer handled by
the `linkcheck` builder. The `dummy ` builder is used
instead, which does not check external links but still
checks internal links.
Those helpers and tools were previously used for
checking links, but now they focus on verifying
cross-references instead.
@AVHopp
AVHopp force-pushed the docs/separate_linkcheck_new branch from 8eaa054 to b24fd59 Compare September 11, 2026 12:33
@Scienfitz
Scienfitz marked this pull request as ready for review September 11, 2026 13:27
Comment thread lychee.toml
Comment thread .github/workflows/ci.yml

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand the PR desc correctly lychee only checks external links, but could also check internal crossrefs like for code objects too
is that correct?

the PR desc mentions this wouldnt bring speedbut, but the whole point of pre-performed linkcheck is also to enable fail-early, not overall speedup (whichis a benefit on top but not needed)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. The issue is that it could only properly check those links in the fully build documentation (source: Independent querying of ChatGPT and Claude Code, checking their docs manually: "Out of the box lychee supports HTML, Markdown and plain text formats.") So we would need to fully build the html documentation first anyway, and building this already checks for broken cross refs (tested)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Three moderate review findings remain unresolved.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

.github/workflows/ci.yml:90

  • Moving external-link checking out of build_documentation.py also changes the separate .github/workflows/docs.yml publish path: it still runs the build script on main pushes, releases, and manual runs, but it has no lychee step or dependency. The needs: [lint, lychee] here cannot gate that separate workflow, so documentation can still be published without external-link validation; add lychee to the publishing workflow or otherwise make publication depend on the check if that path is in scope.
  build-docs:
    name: "Build Docs"
    runs-on: ubuntu-latest
    needs: [lint, lychee]
  • Files reviewed: 6/6 changed files
  • Comments generated: 2
  • Review effort level: Lite

Comment thread .github/workflows/ci.yml Outdated
Comment thread docs/scripts/check_crossrefs.py
cancel-in-progress: true

jobs:
lychee:

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Scienfitz @AdrianSosic I overlooked this in my first implementation. Added it now as I assume that we also want to have link checking when actually building the documentation, not only in the CI. Feel free to resolve once you have seen this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants