feat(BOXP-129): Oracle Cloud Free Tier control plane IaC for lolice cluster - #11581
Open
boxp wants to merge 16 commits into
Open
feat(BOXP-129): Oracle Cloud Free Tier control plane IaC for lolice cluster#11581boxp wants to merge 16 commits into
boxp wants to merge 16 commits into
Conversation
…luster Add Terraform and Ansible resources to provision two Oracle Cloud Ampere A1 (ARM64, Tokyo) nodes as additional Kubernetes control plane members, bringing the etcd cluster from 3 to 5 members (quorum=3) to survive simultaneous failure of 2 physical CPs. - terraform/oci/lolice-control-plane/: new OCI Terraform module - VCN, subnet, internet gateway, security list (Tailscale UDP 41641 + SSH 22) - VM.Standard.A1.Flex x2 (2 OCPU / 12 GB RAM each, Ubuntu 22.04 ARM64) - cloud-init: Tailscale auto-registration via auth key from SSM - S3 backend (tfaction-state), oracle/oci ~> 6.0 provider - terraform/tailscale/lolice/acl.tf: add tag:cloud-control-plane owner + ACL rules for etcd (2379/2380), kubelet (10250), and apiserver (6443) between on-prem and cloud - terraform/tailscale/lolice/auth_key.tf: add reusable/preauthorized auth key for cloud CP nodes stored in SSM /lolice/tailscale/cloud-control-plane-auth-key - ansible/inventories/production/hosts.yml: add cloud_control_plane group (oracle-cp-1/2) with kube_vip_enabled=false (L2 VIP is LAN-only) - ansible/playbooks/cloud-control-plane-join.yml: kubeadm join playbook for cloud CPs using Tailscale IP as apiserver-advertise-address Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
- Use canonical module name community.general.modprobe (fqcn[canonical]) - Add changed_when to kubeadm join command (no-changed-when) - Convert kubeadm join result tasks to handlers (no-handler) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
Tailscale tailnet_key description is limited to 50 characters. Previous value was 66 chars, causing terraform validate to fail. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
- tfaction-root.yaml: add oracle/oci to available_providers and terraform/oci/** target group with OCI credential secrets - acl.tf: restrict reverse rule src from "*" to explicit principals (autogroup:members + tag:subnet-router + tag:k8s-operator); add 192.168.10.0/24 to autoApprovers.routes so cloud CPs can reach the LAN VIP 192.168.10.99 via the on-prem subnet router - outputs.tf: expose Tailscale IP lookup commands for ansible inventory - cloud-control-plane-join.yml: document subnet router prerequisite required for cloud CPs to reach cluster_vip 192.168.10.99 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
… plan - Add registry.terraform.io/oracle/oci to wc-plan.yaml TFPROVIDERCHECK_CONFIG_BODY (needed after OCI credentials and main branch wc-plan.yaml are updated) - Set skip_terraform: true in terraform/oci/lolice-control-plane/tfaction.yaml to bypass terraform init/plan until OCI_TENANCY_OCID, OCI_USER_OCID, OCI_FINGERPRINT, OCI_PRIVATE_KEY secrets are configured in GitHub Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
…ion-root.yaml terraform/oci/** target group needs skip_terraform: true at the root config level so that list-targets outputs skip_terraform=true in the CI matrix. Without OCI credentials configured in GitHub secrets, terraform plan would fail. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
2 tasks
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Contributor
Plan Result (terraform/tailscale/lolice)
Change Result (Click me) # aws_ssm_parameter.cloud_control_plane_auth_key will be created
+ resource "aws_ssm_parameter" "cloud_control_plane_auth_key" {
+ arn = (known after apply)
+ data_type = (known after apply)
+ description = "Tailscale auth key for lolice Oracle Cloud control plane nodes"
+ has_value_wo = (known after apply)
+ id = (known after apply)
+ insecure_value = (known after apply)
+ key_id = (known after apply)
+ name = "/lolice/tailscale/cloud-control-plane-auth-key"
+ region = "ap-northeast-1"
+ tags_all = (known after apply)
+ tier = (known after apply)
+ type = "SecureString"
+ value = (sensitive value)
+ value_wo = (write-only attribute)
+ version = (known after apply)
}
# tailscale_acl.this will be updated in-place
~ resource "tailscale_acl" "this" {
~ acl = jsonencode(
~ {
~ acls = [
{
action = "accept"
dst = [
"tag:k8s-operator:80",
"tag:k8s-operator:443",
]
src = [
"tag:ci",
]
},
+ {
+ action = "accept"
+ dst = [
+ "*:2379",
+ "*:2380",
+ "*:6443",
+ "*:10250",
]
+ src = [
+ "tag:cloud-control-plane",
]
},
+ {
+ action = "accept"
+ dst = [
+ "tag:cloud-control-plane:2379",
+ "tag:cloud-control-plane:2380",
+ "tag:cloud-control-plane:6443",
+ "tag:cloud-control-plane:10250",
]
+ src = [
+ "autogroup:members",
+ "tag:subnet-router",
+ "tag:k8s-operator",
]
},
]
~ autoApprovers = {
+ routes = {
+ "192.168.10.0/24" = [
+ "tag:subnet-router",
]
}
}
~ tagOwners = {
+ "tag:cloud-control-plane" = [
+ "autogroup:admin",
]
# (3 unchanged attributes hidden)
}
}
)
id = "dca925a2-b72b-a210-25af-3ccbdb1f4046"
}
# tailscale_tailnet_key.cloud_control_plane will be created
+ resource "tailscale_tailnet_key" "cloud_control_plane" {
+ created_at = (known after apply)
+ description = "lolice OCI cloud control plane nodes"
+ ephemeral = false
+ expires_at = (known after apply)
+ expiry = (known after apply)
+ id = (known after apply)
+ invalid = (known after apply)
+ key = (sensitive value)
+ preauthorized = true
+ reusable = true
+ tags = [
+ "tag:cloud-control-plane",
]
+ user_id = (known after apply)
}
Plan: 2 to add, 1 to change, 0 to destroy. |
Contributor
❌ Plan Failed (terraform/oci/lolice-control-plane) |
Contributor
Ansible Plan ResultsMode: golyat-4: worker-image
1 changed Changed Tasks (1)
shanghai-1: control-plane
1 changed Changed Tasks (1)
shanghai-1: node-shanghai-1
No changes shanghai-2: control-plane
1 changed Changed Tasks (1)
shanghai-2: node-shanghai-2
No changes shanghai-3: control-plane
1 changed Changed Tasks (1)
shanghai-3: node-shanghai-3
No changes Plan executed on all nodes in parallel. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
lolice cluster の物理 control plane (shanghai-1/2/3) が2台同時障害した場合でも稼働を継続できるよう、Oracle Cloud Free Tier (Ampere A1, 東京, ARM64, ¥0/月) の VM を2台追加するための IaC を実装する。
変更内容
新規追加
terraform/oci/lolice-control-plane/: OCI Terraform モジュールVM.Standard.A1.Flex× 2台 (各 2 OCPU / 12 GB RAM, Ubuntu 22.04 ARM64, 東京)oracle/oci ~> 6.0プロバイダーansible/playbooks/cloud-control-plane-join.yml: kubeadm join プレイブックkubeadm join --control-plane --apiserver-advertise-address=<tailscale-ip>実行docs/project_docs/BOXP-129/plan.md: 設計ドキュメント既存ファイル更新
terraform/tailscale/lolice/acl.tf:tag:cloud-control-planeタグ所有者追加、etcd (2379/2380) / kubelet (10250) / apiserver (6443) の ACL ルール追加terraform/tailscale/lolice/auth_key.tf: クラウド CP 用 Tailscale auth key + SSM パラメータ (/lolice/tailscale/cloud-control-plane-auth-key) 追加ansible/inventories/production/hosts.yml:cloud_control_planeグループ (oracle-cp-1/2) 追加、kube_vip_enabled: false設定適用手順
Step 1: Tailscale ACL + auth key 適用
Step 2: OCI 認証情報を SSM に保存 (手動)
Step 3: Oracle Cloud VM プロビジョニング
Step 4: Ansible inventory の Tailscale IP 更新
terraform outputで Tailscale IP を確認し、ansible/inventories/production/hosts.ymlのTAILSCALE_IP_PLACEHOLDER_*を更新。Step 5: kubeadm join
Test plan
terraform planが terraform/tailscale/lolice と terraform/oci/lolice-control-plane で正常に実行できることansible-playbook cloud-control-plane-join.ymlで kubeadm join が成功することetcdctl endpoint healthで5メンバーが全て healthy であることkubectl get nodesが応答すること (クォーラムテスト)Closes BOXP-129
🤖 Generated with Claude Code