Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 45 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,49 @@ jobs:
working-directory: ansible_collections/cozystack/installer
run: ansible-test sanity --color

master-nodes:
name: Multi-master MASTER_NODES
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.14"

- name: Install Ansible and dependencies
run: pip install ansible-core netaddr

- name: Install required collections
run: ansible-galaxy collection install ansible.utils

- name: Test auto-detection from server group (3 nodes)
run: >-
ansible-playbook tests/test-master-nodes.yml
--inventory tests/test-multi-master-inventory.yml

- name: Test single-node auto-detection (IP host key)
run: >-
ansible-playbook tests/test-master-nodes.yml
--inventory tests/test-single-master-inventory.yml

- name: Test single-node explicit override (CI inventory)
run: >-
ansible-playbook tests/test-master-nodes.yml
--inventory tests/ci-inventory.yml

- name: Test hostname host keys are rejected
run: |
if ansible-playbook tests/test-master-nodes.yml \
--inventory tests/test-hostname-inventory.yml 2>&1; then
echo "ERROR: Expected failure for hostname host keys, but playbook succeeded"
exit 1
else
echo "OK: Hostname host keys correctly rejected"
fi
Comment on lines +100 to +108

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Tighten negative test assertion to prevent false positives.

At Lines 102-108, any playbook failure is considered success. This can pass even when failure is unrelated to hostname-key validation.

✅ Proposed fix (assert failure reason)
       - name: Test hostname host keys are rejected
         run: |
-          if ansible-playbook tests/test-master-nodes.yml \
-            --inventory tests/test-hostname-inventory.yml 2>&1; then
+          set +e
+          output="$(ansible-playbook tests/test-master-nodes.yml \
+            --inventory tests/test-hostname-inventory.yml 2>&1)"
+          status=$?
+          set -e
+
+          if [ "$status" -eq 0 ]; then
             echo "ERROR: Expected failure for hostname host keys, but playbook succeeded"
             exit 1
-          else
-            echo "OK: Hostname host keys correctly rejected"
           fi
+
+          if ! grep -q "not a valid IP address in MASTER_NODES" <<<"$output"; then
+            echo "ERROR: Playbook failed, but not due to hostname/IP validation"
+            echo "$output"
+            exit 1
+          fi
+
+          echo "OK: Hostname host keys correctly rejected"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/test.yml around lines 100 - 108, The current negative test
treats any ansible-playbook failure as success; modify the step named "Test
hostname host keys are rejected" to capture the playbook output (run
ansible-playbook tests/test-master-nodes.yml --inventory
tests/test-hostname-inventory.yml) into a variable and assert the failure
contains the specific hostname-key validation message (e.g. grep/check for a
known substring like "host key validation" or the exact error emitted when
hostname keys are rejected) before declaring success; if the playbook fails but
the output does not contain that expected message, fail the job to avoid false
positives.


e2e:
name: E2E
runs-on: ubuntu-latest
Expand All @@ -76,8 +119,8 @@ jobs:
with:
python-version: "3.14"

- name: Install Ansible
run: pip install ansible-core
- name: Install Ansible and dependencies
run: pip install ansible-core netaddr

- name: Build and install collection
run: |
Expand Down
26 changes: 26 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,7 @@ Runs on `server[0]` only.
| `cozystack_pod_gateway` | `10.42.0.1` | Pod gateway |
| `cozystack_svc_cidr` | `10.43.0.0/16` | Service CIDR |
| `cozystack_join_cidr` | `100.64.0.0/16` | Join CIDR |
| `cozystack_master_nodes` | `""` (auto-detect) | Comma-separated control-plane node IPs for kube-ovn RAFT. Empty = auto-detect from `server` group |
| `cozystack_operator_wait_timeout` | `300` | Timeout for operator/CRD readiness (seconds) |

## Using with k3s
Expand All @@ -229,6 +230,31 @@ or `examples/suse/`:

On cloud providers with NAT (OCI, AWS, GCP), nodes have internal IPs different from public IPs. KubeOVN validates the host IP against `NODE_IPS` and crashes if they don't match. Always use the IP visible on the node's network interface.

### Multi-master setup (kube-ovn RAFT)

Kube-ovn requires `MASTER_NODES` — a comma-separated list of all
control-plane node IPs for OVN RAFT consensus. By default, the role
auto-detects these IPs from the `server` inventory group host keys.

This works when host keys are internal IPs (the recommended inventory
pattern):

```yaml
server:
hosts:
10.0.0.10:
ansible_host: 203.0.113.10
10.0.0.11:
ansible_host: 203.0.113.11
```

If your inventory uses hostnames or non-IP host keys, set
`cozystack_master_nodes` explicitly:

```yaml
cozystack_master_nodes: "10.0.0.10,10.0.0.11,10.0.0.12"
```

### Automatic Helm installation

The role installs Helm and the
Expand Down
13 changes: 10 additions & 3 deletions examples/ubuntu/inventory.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,18 @@ cluster:
children:
server:
hosts:
# Internal IP as host key, public IP as ansible_host
# Internal IP as host key, public IP as ansible_host.
# All server host keys are auto-collected as kube-ovn MASTER_NODES.
10.0.0.10:
ansible_host: 203.0.113.10
agent:
hosts:
10.0.0.11:
ansible_host: 203.0.113.11
10.0.0.12:
ansible_host: 203.0.113.12
agent:
hosts:
10.0.0.20:
ansible_host: 203.0.113.20

vars:
ansible_port: 22
Expand All @@ -26,6 +29,10 @@ cluster:
# Cozystack configuration (REQUIRED)
cozystack_api_server_host: "10.0.0.10"

# kube-ovn master nodes (auto-detected from 'server' group host keys).
# Override only if host keys are not internal IPs.
# cozystack_master_nodes: "10.0.0.10,10.0.0.11,10.0.0.12"

# Platform Package configuration
cozystack_root_host: "cozy.example.com"
cozystack_platform_variant: "isp-full-generic"
Expand Down
1 change: 1 addition & 0 deletions galaxy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ homepage: https://cozystack.io
documentation: https://cozystack.io/docs
dependencies:
ansible.posix: ">=1.0.0"
ansible.utils: ">=2.0.0"
kubernetes.core: ">=5.0.0"
tags:
- infrastructure
Expand Down
1 change: 1 addition & 0 deletions requirements.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
collections:
- name: ansible.posix
- name: ansible.utils
- name: community.general
- name: kubernetes.core
4 changes: 4 additions & 0 deletions roles/cozystack/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ cozystack_platform_variant: "isp-full-generic"
# Leave empty to skip publishing config in the Platform Package
cozystack_root_host: ""

# Comma-separated control-plane node IPs for kube-ovn RAFT consensus.
# Empty = auto-detect from 'server' inventory group host keys.
cozystack_master_nodes: ""

# Network CIDRs for the Platform Package (defaults match k3s)
cozystack_pod_cidr: "10.42.0.0/16"
cozystack_pod_gateway: "10.42.0.1"
Expand Down
29 changes: 29 additions & 0 deletions roles/cozystack/tasks/compute-master-nodes.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
# Compute and validate master node IPs for kube-ovn.
# Included from main.yml and tests.
- name: Compute master node IPs for kube-ovn
ansible.builtin.set_fact:
_cozystack_master_nodes: >-
{%- if cozystack_master_nodes | length > 0 -%}
{{ cozystack_master_nodes }}
{%- else -%}
{{ groups['server'] | join(',') }}
Comment on lines +7 to +10

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Guard inventory lookups to avoid pre-assert crashes.

At Line 10, groups['server'] hard-fails if the group is absent, which bypasses your clearer assert message. Use safe defaults in the compute step.

Proposed hardening
     _cozystack_master_nodes: >-
-      {%- if cozystack_master_nodes | length > 0 -%}
+      {%- if (cozystack_master_nodes | default('')) | length > 0 -%}
         {{ cozystack_master_nodes }}
       {%- else -%}
-        {{ groups['server'] | join(',') }}
+        {{ groups.get('server', []) | join(',') }}
       {%- endif -%}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@roles/cozystack/tasks/compute-master-nodes.yml` around lines 7 - 10, The
current lookup uses groups['server'] which will hard-fail when the 'server'
group is absent; change the compute expression to safely default to an empty
list before joining (e.g. use groups.get('server', []) or groups['server'] |
default([])) so in the template block that references cozystack_master_nodes and
groups['server'] you replace the failing join with a guarded join (e.g.
groups.get('server', []) | join(',') or groups['server'] | default([]) |
join(',')), preserving the existing cozystack_master_nodes check and assert
flow.

{%- endif -%}

- name: Validate master node IPs are non-empty
ansible.builtin.assert:
that:
- _cozystack_master_nodes | length > 0
fail_msg: >-
Could not determine master node IPs for kube-ovn.
Set cozystack_master_nodes explicitly in your inventory.

- name: Validate master node IPs are valid addresses
ansible.builtin.assert:
that:
- (item | ansible.utils.ipaddr) != false
Comment thread
coderabbitai[bot] marked this conversation as resolved.
fail_msg: >-
'{{ item }}' is not a valid IP address in MASTER_NODES.
Inventory host keys in the 'server' group must be IPs, not hostnames.
Alternatively, set cozystack_master_nodes explicitly.
loop: "{{ _cozystack_master_nodes.split(',') }}"
4 changes: 4 additions & 0 deletions roles/cozystack/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@
It must be the node's INTERNAL IP
(visible on network interface, not public/NAT IP).

- name: Compute and validate master node IPs for kube-ovn
ansible.builtin.include_tasks: compute-master-nodes.yml
when: cozystack_create_platform_package

- name: Set architecture for Helm download
ansible.builtin.set_fact:
_cozystack_arch: "{{ 'amd64' if ansible_architecture == 'x86_64' else 'arm64' }}"
Expand Down
2 changes: 1 addition & 1 deletion roles/cozystack/templates/platform-package.yml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ spec:
autoMount:
enabled: true
kube-ovn:
MASTER_NODES: {{ cozystack_api_server_host | to_json }}
MASTER_NODES: {{ _cozystack_master_nodes | to_json }}
platform:
values:
networking:
Expand Down
1 change: 1 addition & 0 deletions tests/ci-inventory.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ cluster:

# Cozystack configuration
cozystack_api_server_host: "127.0.0.1"
cozystack_master_nodes: "127.0.0.1"
cozystack_root_host: "cozy.ci-test.local"
cozystack_platform_variant: "isp-full-generic"

Expand Down
16 changes: 16 additions & 0 deletions tests/test-hostname-inventory.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
# Inventory with hostname host keys (NOT IPs).
# Used to verify that IP validation correctly rejects hostnames.
cluster:
children:
server:
hosts:
node1.example.com:
ansible_connection: local
ansible_host: 127.0.0.1
agent:
hosts: {}

vars:
ansible_user: runner
cozystack_api_server_host: "10.0.0.1"
92 changes: 92 additions & 0 deletions tests/test-master-nodes.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
# Verify that MASTER_NODES is correctly derived from the 'server'
# inventory group in both single-master and multi-master setups.
# Uses the actual role tasks (no duplicated Jinja2 logic).
- name: Test MASTER_NODES auto-detection and template rendering
hosts: server[0]
gather_facts: false
pre_tasks:
- name: Set template defaults when not defined by inventory
ansible.builtin.set_fact:
cozystack_master_nodes: "{{ cozystack_master_nodes | default('') }}"
cozystack_api_server_host: "{{ cozystack_api_server_host | default('127.0.0.1') }}"
cozystack_api_server_port: "{{ cozystack_api_server_port | default('6443') }}"
cozystack_namespace: "{{ cozystack_namespace | default('cozy-system') }}"
cozystack_platform_variant: "{{ cozystack_platform_variant | default('isp-full-generic') }}"
cozystack_root_host: "{{ cozystack_root_host | default('') }}"
cozystack_pod_cidr: "{{ cozystack_pod_cidr | default('10.42.0.0/16') }}"
cozystack_pod_gateway: "{{ cozystack_pod_gateway | default('10.42.0.1') }}"
cozystack_svc_cidr: "{{ cozystack_svc_cidr | default('10.43.0.0/16') }}"
cozystack_join_cidr: "{{ cozystack_join_cidr | default('100.64.0.0/16') }}"
tasks:
- name: Include role master-node computation
ansible.builtin.include_tasks:
file: "{{ playbook_dir }}/../roles/cozystack/tasks/compute-master-nodes.yml"

- name: Assert auto-detected MASTER_NODES matches server group
ansible.builtin.assert:
that:
- >-
(_cozystack_master_nodes | split(',') | sort) ==
(groups['server'] | sort)
fail_msg: >-
Expected MASTER_NODES to match server group hosts.
Got '{{ _cozystack_master_nodes }}',
expected '{{ groups['server'] | join(',') }}'.
when: (cozystack_master_nodes | default('')) | length == 0

- name: Construct API server endpoint
ansible.builtin.set_fact:
_cozystack_api_endpoint: "https://{{ cozystack_api_server_host }}:{{ cozystack_api_server_port }}"

- name: Render Platform Package template
ansible.builtin.template:
src: "{{ playbook_dir }}/../roles/cozystack/templates/platform-package.yml.j2"
dest: /tmp/cozystack-test-platform-package.yml
mode: "0600"

- name: Read rendered manifest
ansible.builtin.slurp:
src: /tmp/cozystack-test-platform-package.yml
register: _rendered_manifest

- name: Assert rendered MASTER_NODES contains all server IPs
ansible.builtin.assert:
that:
- "'MASTER_NODES' in _rendered_content"
- "(_cozystack_master_nodes | to_json) in _rendered_content"
fail_msg: >-
Rendered template does not contain expected MASTER_NODES value.
Expected {{ _cozystack_master_nodes | to_json }} in output.
vars:
_rendered_content: "{{ _rendered_manifest.content | b64decode }}"

- name: Show rendered MASTER_NODES (debug)
ansible.builtin.debug:
msg: "MASTER_NODES = {{ _cozystack_master_nodes }}"

- name: Clean up test manifest
ansible.builtin.file:
path: /tmp/cozystack-test-platform-package.yml
state: absent
changed_when: false

- name: Test explicit cozystack_master_nodes override
hosts: server[0]
gather_facts: false
tasks:
- name: Set explicit master nodes override
ansible.builtin.set_fact:
cozystack_master_nodes: "192.168.1.1,192.168.1.2"

- name: Include role master-node computation
ansible.builtin.include_tasks:
file: "{{ playbook_dir }}/../roles/cozystack/tasks/compute-master-nodes.yml"

- name: Assert explicit override is used
ansible.builtin.assert:
that:
- _cozystack_master_nodes == "192.168.1.1,192.168.1.2"
fail_msg: >-
Expected explicit override '192.168.1.1,192.168.1.2',
got '{{ _cozystack_master_nodes }}'.
33 changes: 33 additions & 0 deletions tests/test-multi-master-inventory.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
# Test inventory with three control-plane nodes.
# Used to verify MASTER_NODES auto-detection from the 'server' group.
cluster:
children:
server:
hosts:
10.0.0.10:
ansible_connection: local
ansible_host: 127.0.0.1
10.0.0.11:
ansible_connection: local
ansible_host: 127.0.0.1
10.0.0.12:
ansible_connection: local
ansible_host: 127.0.0.1
agent:
hosts: {}

vars:
ansible_user: runner

# Cozystack configuration
cozystack_api_server_host: "10.0.0.10"
cozystack_api_server_port: "6443"
cozystack_namespace: "cozy-system"
cozystack_platform_variant: "isp-full-generic"
cozystack_create_platform_package: true
cozystack_root_host: ""
cozystack_pod_cidr: "10.42.0.0/16"
cozystack_pod_gateway: "10.42.0.1"
cozystack_svc_cidr: "10.43.0.0/16"
cozystack_join_cidr: "100.64.0.0/16"
15 changes: 15 additions & 0 deletions tests/test-single-master-inventory.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
# Single-master test inventory with IP as host key.
# Tests auto-detection path (cozystack_master_nodes is NOT set).
cluster:
children:
server:
hosts:
127.0.0.1:
ansible_connection: local
agent:
hosts: {}

vars:
ansible_user: runner
cozystack_api_server_host: "127.0.0.1"