Add cluster backups guide (etcd + Velero)

[Akini98]

Purpose

Resolves #218. The k8s-cluster examples document how to provision a tenant cluster but not how to back it up. Product teams own their cluster backups to their own S3 buckets and had no guide.

Goals

Provide a self-service backup guide, as a companion to the cluster-provisioning guide, covering both backup layers a tenant needs.

Approach

• Add modules/tenancy/k8s-cluster/examples/BACKUPS.md:
  • etcd S3 snapshots — bucket + IAM (policy JSON), a Rancher S3 rancher2_cloud_credential, the module's etcd_s3 input, verify, restore, and the etcd_snapshot_create drift note.
  • Velero — bucket + IAM (policy JSON incl. multipart actions), rancher2_catalog_v2 + rancher2_app_v2 (chart 8.1.0, plugin v1.9.0, upgradeCRDs: false), schedule, verify (incl. the velero- schedule-name prefix), backup + restore.
  • Scheduling/retention reference and a production-hardening checklist.
• Link it from examples/README.md (TOC + new section 12 "Backups").

Release note

Docs: add a self-service etcd + Velero backup guide for tenant RKE2 clusters.

Documentation

This PR is documentation: modules/tenancy/k8s-cluster/examples/BACKUPS.md.

Automation tests

N/A — documentation only. Validated end-to-end on a live tenant cluster: a non-admin tenant user provisioned an RKE2 cluster, configured etcd S3 snapshots (objects confirmed in S3), and installed Velero + ran a namespace backup and restore.

Security checks

• Followed secure coding standards? yes
• Confirmed no keys/passwords/tokens/secrets committed? yes — generic placeholders only (my-team, AKIAxxx).

Related PRs

A follow-up module PR will add rke_config[0].etcd_snapshot_create to the k8s-cluster ignore_changes (the drift noted in the guide).

Other template sections (Training, Certification, Marketing, Migrations): N/A.

Summary by CodeRabbit

• Documentation
  • Added a comprehensive “Backups” guide covering two recovery layers: control-plane etcd snapshots and namespace/PersistentVolume backups via Velero.
  • Included step-by-step setup for bucket/IAM configuration, Terraform + verification steps, and clear restore procedures.
  • Added retention/scheduling guidance (UTC) and an S3 hardening checklist.
  • Updated the cluster examples README with a new Backups section linking to the full guide.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 8c81e3bb-51fb-4b2f-8ecc-7771ecada03f

📥 Commits

Reviewing files that changed from the base of the PR and between ea3234c and 8eab7df.

📒 Files selected for processing (1)

• modules/tenancy/k8s-cluster/examples/BACKUPS.md

🚧 Files skipped from review as they are similar to previous changes (1)

• modules/tenancy/k8s-cluster/examples/BACKUPS.md

---

📝 Walkthrough

Walkthrough

Adds a new BACKUPS.md guide for tenant cluster backups with etcd snapshots and Velero, then links it from the examples README. The guide covers setup, verification, restore steps, scheduling, retention, and hardening guidance.

Changes

Backup Documentation

Layer / File(s)	Summary
Guide introduction and table of contents modules/tenancy/k8s-cluster/examples/BACKUPS.md	Introduces the two backup layers, prerequisites, and the guide table of contents.
etcd S3 snapshot setup and restore modules/tenancy/k8s-cluster/examples/BACKUPS.md	Documents S3 bucket and IAM setup, Terraform variables and Rancher cloud credential, etcd_s3 module wiring, verification, restore behavior, and drift mitigation for etcd snapshots.
Velero setup, backup, and restore modules/tenancy/k8s-cluster/examples/BACKUPS.md	Documents Velero S3 bucket and IAM setup, Terraform and Rancher app wiring, scheduled backups, verification, CLI backup and restore steps, and DR guidance.
Scheduling reference, hardening checklist, and README linkage modules/tenancy/k8s-cluster/examples/BACKUPS.md, modules/tenancy/k8s-cluster/examples/README.md	Adds scheduling and retention reference material, production hardening guidance, and README table of contents and Backups section linking to the guide.

Estimated code review effort: 2 (Simple) | ~15 minutes

Related issues: #218

Suggested labels: documentation

Suggested reviewers: HiranAdikari

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the main change: a new cluster backups guide for etcd snapshots and Velero.
Description check	✅ Passed	The description covers summary, changes, testing, checklist items, and links the issue, though headings differ from the template.
Linked Issues check	✅ Passed	The PR adds BACKUPS.md, links it from README, and covers etcd S3 snapshots and Velero with setup, verify, and restore steps.
Out of Scope Changes check	✅ Passed	The changes stay within documentation scope and match the backup-guide objectives with no unrelated code additions.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@modules/tenancy/k8s-cluster/examples/BACKUPS.md`:
- Around line 62-63: The backup example currently enables S3 bucket versioning
without handling noncurrent versions, so snapshot retention does not truly cap
stored data. Update the BACKUPS.md example to either remove the versioning step
or add a lifecycle rule for noncurrent object versions in the same flow, and
then revise the later retention note so it matches the chosen behavior. Use the
S3 setup example around the bucket versioning command and the snapshot_retention
guidance to keep the instructions consistent.
- Around line 58-63: The S3 bucket setup example needs to handle the us-east-1
special case in the bucket creation step. Update the create-bucket example in
the BACKUPS documentation so the aws s3api create-bucket command omits
--create-bucket-configuration when REGION is us-east-1, while preserving the
existing LocationConstraint behavior for other regions. Use the bucket example
block in the document to apply the conditional logic clearly.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: de18ebb4-bd3c-4c3a-8f0a-3c8faa0344c7

📥 Commits

Reviewing files that changed from the base of the PR and between bb7138b and ea3234c.

📒 Files selected for processing (2)

• modules/tenancy/k8s-cluster/examples/BACKUPS.md
• modules/tenancy/k8s-cluster/examples/README.md