Skip to content

feat(helm): add weekly extra MongoDB backup CronJob for history, linker_output, and webpages_text#3145

Merged
yodem merged 2 commits intomasterfrom
feature/sc-42396/mongo-extra-backup-weekly
Mar 11, 2026
Merged

feat(helm): add weekly extra MongoDB backup CronJob for history, linker_output, and webpages_text#3145
yodem merged 2 commits intomasterfrom
feature/sc-42396/mongo-extra-backup-weekly

Conversation

@yodem
Copy link
Collaborator

@yodem yodem commented Mar 11, 2026

Summary

Shortcut: sc-42601 (subtask of sc-42396)

Files changed

File Change
helm-chart/sefaria/templates/cronjob/mongo-backup-extra.yaml New — weekly CronJob with 40Gi emptyDir
helm-chart/sefaria/templates/configmap/create-extra-backup-dumps.yaml New — dumps all 3 collections sequentially
helm-chart/sefaria/templates/configmap/upload-extra-backup-dumps.yaml New — streams each collection to GCS as separate tarball, 3-week rotation
helm-chart/sefaria/values.yaml Add extraBackupEnabled: false
build/ci/production-values.yaml Add extraBackupEnabled: true

Test plan

  • helm template with extraBackupEnabled: true renders all 3 resources (CronJob + 2 ConfigMaps)
  • helm template with default values (flag absent/false) renders 0 extra resources
  • Verify job runs successfully in staging and produces 3 separate tarballs in GCS
  • Confirm rotation deletes old files after 3 weeks

🤖 Generated with Claude Code

yodem and others added 2 commits March 11, 2026 15:35
…er_output, and webpages_text

Consolidates sc-42543 and sc-42544 into a single weekly job (Sunday 2AM UTC).
Dumps all three large collections sequentially and uploads each as its own tarball.
Rotation keeps 3 weeks of snapshots. Controlled by backup.mongo.extraBackupEnabled.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…y loops

Retry loops for linker_output and webpages_text were missing the
rm -rf cleanup before retrying, unlike the history loop. Without this,
a partial dump from a failed attempt could persist on disk and be
silently uploaded as a corrupt backup on a subsequent retry.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@yodem yodem requested a review from BrendanGalloway March 11, 2026 13:46
@yodem yodem added this pull request to the merge queue Mar 11, 2026
@mergify
Copy link

mergify bot commented Mar 11, 2026

🧪 CI Insights

Here's what we observed from your CI run for 17dcde4.

🟢 All jobs passed!

But CI Insights is watching 👀

Merged via the queue into master with commit 23c2845 Mar 11, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants