Skip to content

Create a PR on population automation runs #340

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AardWolf merged 1 commit into from
Mar 1, 2026
Merged

Create a PR on population automation runs #340

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 79 additions & 17 deletions .github/workflows/population-scheduled.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,50 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Aggressive cleanup
run: |
df -h
# Remove Java (JDKs)
sudo rm -rf /usr/lib/jvm

# Remove .NET SDKs
sudo rm -rf /usr/share/dotnet

# Remove Swift toolchain
sudo rm -rf /usr/share/swift

# Remove Haskell (GHC)
sudo rm -rf /usr/local/.ghcup

# Remove Julia
sudo rm -rf /usr/local/julia*

# Remove Android SDKs
sudo rm -rf /usr/local/lib/android

# Remove Chromium (optional if not using for browser tests)
sudo rm -rf /usr/local/share/chromium

# Remove Microsoft/Edge and Google Chrome builds
sudo rm -rf /opt/microsoft /opt/google

# Remove Azure CLI
sudo rm -rf /opt/az

# Remove PowerShell
sudo rm -rf /usr/local/share/powershell

# Remove CodeQL and other toolcaches
sudo rm -rf /opt/hostedtoolcache
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cleanup step deletes /opt/hostedtoolcache, which is where GitHub-hosted runners typically keep the Node runtime used to execute JavaScript-based actions (e.g., actions/checkout, actions/setup-node). Removing it at the start of the job can cause subsequent uses: steps to fail. Consider avoiding deletion of /opt/hostedtoolcache (or at least preserving the Node toolcache), or move disk-reclamation to after all uses: actions have run.

Suggested change
sudo rm -rf /opt/hostedtoolcache
# Note: /opt/hostedtoolcache is preserved to keep Node runtimes for GitHub Actions.

Copilot uses AI. Check for mistakes.

sudo docker system prune -af || true
sudo docker builder prune -af || true
df -h

- name: Checkout MHCT db docker repo
uses: actions/checkout@v4
uses: actions/checkout@v5
with:
repository: m-h-c-t/mhct-db-docker
path: mhct-db-docker
Expand All @@ -22,34 +63,51 @@ jobs:
- name: Build MHCT db docker image
run: |
cd mhct-db-docker
docker build -t mhct-db-docker .
docker build -t mhct-db -f Dockerfile.slim .

- name: Run MHCT db docker container
run: |
docker run -d --name mhct-db -p 3306:3306 mhct-db-docker
docker run -d --name mhct-db -v mhct-db-data:/var/lib/mysql -p 3306:3306 mhct-db
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker run now uses a named volume mhct-db-data, but the workflow cleanup only removes the container. This can leave the volume consuming disk for the remainder of the job (and can contribute to disk pressure, which this workflow is trying to mitigate). Either remove the named volume in the cleanup step or avoid using a named volume here if persistence isn’t required.

Copilot uses AI. Check for mistakes.

# Wait for database to be ready with retry loop
echo "Waiting for database to be ready..."
for i in {1..30}; do
if docker exec mhct-db mysqladmin ping -h localhost -psecret --silent; then
echo "Database is ready!"
# Wait for database initialization (up to 2 hours)
echo "Waiting for database initialization to complete..."
for i in {1..120}; do
if docker logs mhct-db 2>&1 | grep -q "MySQL init process done. Ready for start up."; then
echo "Database initialization complete!"
break
fi
echo "Attempt $i: Database not ready yet, waiting..."
sleep 2
echo "Attempt $i: Database still initializing, waiting..."
sleep 60
done

# Final check to ensure database is ready
if ! docker exec mhct-db mysqladmin ping -h localhost -psecret --silent; then
echo "Database failed to become ready after 60 seconds"
# Wait for MySQL to come back up after initialization
if docker logs mhct-db 2>&1 | grep -q "MySQL init process done. Ready for start up."; then
echo "Waiting for MySQL to restart and become ready..."
for i in {1..30}; do
if docker exec mhct-db mysqladmin ping -h localhost -psecret --silent; then
echo "MySQL is ready!"
break
fi
echo "Waiting for MySQL to come back up..."
sleep 2
done

if ! docker exec mhct-db mysqladmin ping -h localhost -psecret --silent; then
echo "MySQL failed to become ready after restart."
docker logs mhct-db
exit 1
fi
else
echo "Database initialization failed to complete after 120 minutes."
docker logs mhct-db
exit 1
fi

- name: Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@v5

- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v5
Comment on lines +107 to +110
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow uses actions/checkout@v5 / actions/setup-node@v5 while the only other workflow in this repo pins major v4 for both. If v5 is intentional, consider aligning versions repo-wide; otherwise, sticking to the same major version helps avoid unexpected behavioral differences.

Copilot uses AI. Check for mistakes.
with:
node-version: 24
cache: npm
Expand All @@ -60,7 +118,7 @@ jobs:
- name: Update population data
run: npm run pop

- name: Commit and push if there are changes
- name: Create a PR if population data changed
run: |
if [ -z "$(git status --porcelain data/pop-csv)" ]; then
echo "No changes in population data."
Expand All @@ -72,9 +130,13 @@ jobs:

git add data/pop-csv/*
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

git add data/pop-csv/* won’t stage deletions (and can miss files if the directory structure changes). To ensure population updates correctly capture removals/renames as well as additions/modifications, prefer staging the directory with an option that includes deletions (e.g., adding the path with -A).

Suggested change
git add data/pop-csv/*
git add -A data/pop-csv

Copilot uses AI. Check for mistakes.

git checkout -b scheduled/population-$(date +%Y%m%d)
git commit -m "Automated population data update" || exit 0
Comment on lines +133 to 134
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The branch name only includes the date, so reruns on the same day (or a manually re-triggered run after a partial failure) can fail at git checkout -b / git push because the branch already exists. Consider including a unique suffix (e.g., run id / timestamp) or using a branch-reset approach (-B) with an appropriate push strategy.

Copilot uses AI. Check for mistakes.
git push
git push origin HEAD

gh pr create --fill || echo "PR already exists"
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gh pr create --fill || echo "PR already exists" will also hide other failures (auth problems, validation errors, rate limits), potentially making the workflow look successful when it didn’t open a PR. Consider explicitly checking whether a PR already exists for the head branch and only suppressing that specific case; otherwise fail the step so problems are visible.

Copilot uses AI. Check for mistakes.
env:
GH_TOKEN: ${{ github.token }}
- name: Cleanup Docker container
if: always()
run: |
Expand Down
14 changes: 0 additions & 14 deletions data/pop-js/balacks-cove.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,20 +81,6 @@ module.exports = {
}
]
},
{
cheese: utils.genVarField("cheese", [
"Vanilla Stilton",
"Vengeful Vanilla Stilton",
]),
phase: utils.genVarField("stage", "High Tide"),
config: [
{
opts: {
include: ["Riptide"]
}
}
]
}
],
/**
*
Expand Down
Loading