Skip to content

Reusable Claude review workflow#8

Merged
squaremo merged 3 commits intomainfrom
reusable-review-workflow
Feb 27, 2026
Merged

Reusable Claude review workflow#8
squaremo merged 3 commits intomainfrom
reusable-review-workflow

Conversation

@squaremo
Copy link
Contributor

@squaremo squaremo commented Feb 5, 2026

This takes the Claude workflow definition as it is in nscaledev/uni-region#381, tweaks it a little to make it more robust when files aren't present, and puts it in a workflow_call trigger.

This means you can call it from another repo, and it'll act locally; and that means we can have centralised prompts (as below), but local guidelines (as in CLAUDE.md), which makes it much easier to evolve our reviews.

Fixes INST-545.

This is to centralise the automated workflow definition, so we don't
have to keep it up to date everywhere. The definition can either be
static, or better, it can refer to guidelines in the individual repos.
@squaremo squaremo requested a review from peterj-ns February 5, 2026 16:44
secrets:
CLAUDE_CODE_OAUTH_TOKEN:
description: A Claude Code access token. Supply this **or** ANTHROPIC_API_KEY.
ANTHROPIC_API_KEY:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used one of these (as created in the Anthropic console), but I note that our repos have a CLAUDE_CODE_OAUTH_TOKEN (I don't know how to get one of those) -- they are not interchangeable, so I've made it possible to supply either.

@squaremo
Copy link
Contributor Author

squaremo commented Feb 6, 2026

I've been using a PR in https://github.com/nscaledev/uni-shared-workflow-tester, which calls the workflow here, to experiment.

Some results (most recent to least):

  • https://github.com/nscaledev/uni-shared-workflow-tester/actions/runs/21760515725 -- this one was run after separating out specific instructions into rules/*.md (and importing them in the CLAUDE.md here). The prompt -- also here -- says to resolve those imports and use the WebFetch tool to get them. This ends up stringing at least a couple of web fetches together (one to get the CLAUDE.md, then more to get individual files that it imports), which amplifies the non-determinism. But it seems pretty reliable at figuring out the URL to fetch, so that's something.
  • https://github.com/nscaledev/uni-shared-workflow-tester/actions/runs/21760004065 this one is interesting because you can see what WebFetch does. The first use gets the CLAUDE.md, which it summarises .. OK .. then you can see it interprets what it wants from the other imports, and chooses which ones. The interpretation exposes the stochastic behaviour -- different runs may or may not decide to fetch guidelines.

@squaremo squaremo force-pushed the reusable-review-workflow branch from 3b8328e to 1ba54de Compare February 10, 2026 18:16
This needs to be suplied by the caller, since it's not (at present at
least) a secret at the org level.

Claude acvtions elsewhere use a Claude API token, but I needed to use
an API key. I have allowed for both by making them optional and
supplying both (as possibly empty values).
This tries to bias the output towards an evaluation of the pull
request as submitted, including the summary.

 - Throw in "team-oriented" and "experienced" as framing (I have not
   much idea how it influences the output to be honest).

 - "dispassionate" too; this seems to dial down the cheerleading.

 - with the built-in prompt there is often a wordy recapitulation of
   the content of the diff, which is needless -- so, this custom
   prompt asks for not that.

 - Ask for Claude to compare the PR summary with the diff, specifically,
   as this doesn't come through with the built-in prompt.

The overall effect (I hope, but we shall see) is for the machine to
treat the pull request as an argument for the patch therein. The
alternative (and default) is to assume the diff is self-evidently
necessary, which is seldom the case.
@squaremo squaremo force-pushed the reusable-review-workflow branch from 1ba54de to 5b54bc5 Compare February 12, 2026 16:52
@squaremo
Copy link
Contributor Author

I wanted this to centralise the guidelines (what would go in CLAUDE.md) but I couldn't get this to work reliably. Locally it's possible to put a CLAUDE.md in another directory and use --add-dir to tell Claude to look there (https://code.claude.com/docs/en/memory#load-memory-from-additional-directories).

I tried implementing this in the workflow by checking out this repo (fine) alongside the repo under review (I couldn't get the checkout action to do this in an acceptable way, either you supply .path for the second checkout and it nests it in the first, or you give .path for both checkouts and it fails because github.workspace is set to the default path.) But even in the nested case, it didn't seem to accept the -add-dir argument. But it is 1. difficult to tell, because LLMs, and 2. always possible I got something wrong.

@squaremo
Copy link
Contributor Author

@peterj-ns After experiments I reduced this to a callable workflow, which at least means we can just call with a secret and keep the (minimal) prompt and other parameters centrally.

@squaremo squaremo requested review from yankcrime and removed request for peterj-ns February 13, 2026 09:54
@squaremo squaremo requested a review from peterj-ns February 27, 2026 11:49
@squaremo squaremo merged commit db2339f into main Feb 27, 2026
@squaremo squaremo deleted the reusable-review-workflow branch February 27, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants