Skip to content

Add broken link checker - #958

Open
Scotchester wants to merge 4 commits into
mainfrom
chore/broken-link-checker
Open

Add broken link checker#958
Scotchester wants to merge 4 commits into
mainfrom
chore/broken-link-checker

Conversation

@Scotchester

@Scotchester Scotchester commented Jul 29, 2026

Copy link
Copy Markdown
Member

Part of #962


  • Check PRs that update the site
  • Run on a schedule to catch external links that change out from under us

@Scotchester
Scotchester requested a review from a team as a code owner July 29, 2026 21:42
@netlify

netlify Bot commented Jul 29, 2026

Copy link
Copy Markdown

Deploy Preview for cal-itp-mobility-marketplace ready!

Name Link
🔨 Latest commit 4791caf
🔍 Latest deploy log https://app.netlify.com/projects/cal-itp-mobility-marketplace/deploys/6a739eed5f00550008abec56
😎 Deploy Preview https://deploy-preview-958--cal-itp-mobility-marketplace.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@jgravois jgravois left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks promising!

i'll hold off on reviewing this one until i see a completed workflow run.

@Scotchester
Scotchester force-pushed the chore/broken-link-checker branch from ab14ac1 to 49df560 Compare August 5, 2026 19:17
@Scotchester

Scotchester commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Thought I had left this in draft, but apparently not. It's now ready for review!

For the PR check, I elected to just have it run like any other PR check and report success or failure (i.e., it doesn't post a comment as I previously suggested doing). You can see it doing this on this very PR.

For the weekly run, the approach I took was to have it open an issue with a copy of the report if there are any failures. If it finds that there is already an issue with the linkchecker label on it, it will add a comment to that issue with the latest results, rather than creating a new issue, so will only be one linkchecker issue open at a time. The idea was to prevent a proliferation of issues reporting the same problems if we can't remediate them all in the space of a week.

You can see an example of how it looks in the test repo: compilerla/actions-testing#166

Feedback welcome!

@jgravois jgravois left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have some feedback that is orthogonal to something i said inline.

personally, i think the cron workflow is substantially higher value here than the action that runs on pull_request. if we aren't catching broken links during manual QA, we have bigger problems.

Comment on lines +6 to +8
- "eleventy.config.js"
- "linkcheckerrc"
- "src/**"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with this many path filters, i'd be inclined to just yank paths altogether.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really prefer not to run unnecessary checks when we can avoid them without too much effort. I don't think we actually have to pay for Actions minutes since our repos are open source, but it's still wasteful and adds noise on PRs where they don't apply.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's why my orthogonal suggestion was to get rid of the workflow altogether! 😆

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a solid point! I'm not opposed to not doing it on PRs of everyone agrees a weekly check is sufficient.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep I'm good with the weekly

jobs:
linkchecker:
runs-on: ubuntu-latest
timeout-minutes: 5 # prevent background server step from running forever if something else goes wrong

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this reminds me that it would be cool to have our new terraform gh actions self-destruct and ensure their associated locks are removed....

- name: Check for an open linkchecker issue
id: open-issue-check
if: ${{ failure() }}
run: echo "OPEN_ISSUE_ID=`gh issue list --label linkchecker --json number --jq '.[0].number'`" >> "$GITHUB_OUTPUT"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dig that you're just using gh for all this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a pretty powerful addition to the built-in GHA toolset!

Comment on lines +45 to +46
--body-file comment.md \
--repo $GITHUB_REPOSITORY

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YMMV, but in peter-evans/create-or-update-comment, the default edit-mode mode appends files to anything existing it finds...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for suggesting this. I have a few reasons for preferring the current approach, though:

  • We have expressed a desire to reduce our allowance on third-party actions.
  • It's not even clear to me from the docs/issues in that repo if that action can update the original issue description, or just comments. Is the description considered a comment that can be acted on in the same way? Would require some deeper research and/or testing.
  • Most importantly (to me), editing the issue description/comment will not create a new notification for those watching the issue, which will make it harder to notice new results that occur after someone has already started working on the prior results (or make it easy to forget how long reported issues have going unaddressed).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry. i didn't intend to suggest that you switch to the github action i linked to.

i was only saying that the underlying gh behavior might be the same. 😇

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah!

I wouldn't expect that to be the case since I'm just telling it to make a new comment and not referencing an exiting comment, but I'll run the workflow in the test repo manually again and confirm what happens.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread linkcheckerrc
Comment on lines +2 to +3
ignore=
/cdn-cgi/l/email-protection

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what are these links pray tell?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out that we have Cloudflare set up to mask email addresses, so the href that linkchecker sees for a mailto link looks like this:

<a href="/cdn-cgi/l/email-protection#79111c151516391a1815100d0957160b1e">
  <span class="__cf_email__" data-cfemail="670f020b0b082704060b0e131749081500">[email&#160;protected]</span>
</a>

And those result in 404s if it just tries to follow them directly.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very cool!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I literally had no idea... 🤔

@thekaveman thekaveman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make it explicit, we are awaiting removal of the PR workflow and aligned on the weekly automated workflow only 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[spike] Link scanner tool

3 participants