diff --git a/docs/migration-guide.md b/docs/migration-guide.md new file mode 100644 index 000000000..acccd3dcb --- /dev/null +++ b/docs/migration-guide.md @@ -0,0 +1,138 @@ +# Migrating from foreman-installer to foremanctl + +## Overview + +When upgrading from foreman-installer to foremanctl, the `foremanctl migrate` command helps convert your existing configuration to the new format. + +This guide explains how to migrate your foreman-installer answer files to foremanctl configuration files. + +## Prerequisites + +Before migrating, ensure the following: + +1. **Foreman deployment using foreman-installer** - You should have an existing Foreman deployment has been installed using foreman-installer and has an answers file to migrate from. + +2. **foremanctl is installed** on the system: + ```bash + # Enable the foremanctl repository + dnf copr enable @theforeman/foremanctl rhel-9-x86_64 + + # Install foremanctl + dnf install foremanctl + ``` + + For more installation options, see the main [README](../README.md#packages). + +## Migration Workflow + +1. **Generate the migrated configuration**: + ```bash + foremanctl migrate --output /var/lib/foremanctl/parameters.yaml + ``` + +2. **Review the output** for any warnings about unmapped parameters + +3. **Use the migrated configuration** with foremanctl: + ```bash + foremanctl deploy + ``` + (foremanctl automatically loads configuration from `/var/lib/foremanctl/parameters.yaml`) + +## Command Usage + +### Basic Migration + +Migrate from the default location (reads the currently active scenario): +```bash +foremanctl migrate --output /var/lib/foremanctl/parameters.yaml +``` + +### Custom Answer File + +Migrate from a specific answer file: +```bash +foremanctl migrate --answer-file /path/to/custom-answers.yaml --output /var/lib/foremanctl/parameters.yaml +``` + +### Output to stdout + +Preview the migrated configuration without writing a file: +```bash +foremanctl migrate +``` + +## Command Options + +- `--answer-file PATH` - Path to the foreman-installer answer file. If not specified, reads the currently active scenario and extracts the answer file path from it. +- `--output PATH` - Path for the migrated configuration (default: stdout) + +> [!NOTE] +> Unlike other `foremanctl` commands, migrate does not persist parameters between runs. Each migration is independent. + +## Parameter Mappings + +The migrate command automatically maps foreman-installer parameters to foremanctl parameters. For a complete list of all parameter mappings, see the [Parameters documentation](parameters.md#mapping). + +## Example + +Below is an example showing how the transformation works: + +### Input (foreman-installer format) + +```yaml +foreman: + db_host: database.example.com + db_port: 5432 + db_database: foreman + db_username: foreman_user + db_password: secret123 + db_manage: true + initial_admin_username: admin + initial_admin_password: changeme +``` + +### Output (foremanctl format) + +```yaml +database_host: database.example.com +database_port: 5432 +database_mode: external +foreman_database_name: foreman +foreman_database_password: secret123 +foreman_database_user: foreman_user +foreman_initial_admin_password: changeme +foreman_initial_admin_username: admin +``` + +## Handling Unmapped Parameters + +When the migration completes, you may see warnings like: + +> [!WARNING] +> The following parameters could not be mapped: +> - katello::enable_ostree +> - foreman::some_other_param + +These parameters need to be manually reviewed and added to the new configuration if needed. Check the [parameters documentation](parameters.md) for equivalent foremanctl parameters. + +## Using the Migrated Configuration + +Once you've generated and reviewed the migrated configuration: + +1. **Save it to the foremanctl parameters file**: + ```bash + # Either generate directly to the parameters file + foremanctl migrate --output /var/lib/foremanctl/parameters.yaml + + # Or copy after review + foremanctl migrate --output /tmp/migrated.yaml + # Review /tmp/migrated.yaml + cp /tmp/migrated.yaml /var/lib/foremanctl/parameters.yaml + ``` + +2. **Deploy using foremanctl**: + ```bash + foremanctl deploy + ``` + + The `foremanctl deploy` command automatically loads configuration from `/var/lib/foremanctl/parameters.yaml`. diff --git a/src/ansible.cfg b/src/ansible.cfg index 2b4e69983..0af18bcd8 100644 --- a/src/ansible.cfg +++ b/src/ansible.cfg @@ -1,6 +1,7 @@ [defaults] host_key_checking = False roles_path = ./roles +library = ./plugins/modules filter_plugins = ./filter_plugins callback_plugins = ./callback_plugins callback_result_format = yaml diff --git a/src/playbooks/migrate/metadata.obsah.yaml b/src/playbooks/migrate/metadata.obsah.yaml new file mode 100644 index 000000000..b937f954e --- /dev/null +++ b/src/playbooks/migrate/metadata.obsah.yaml @@ -0,0 +1,15 @@ +--- +help: | + Migrate foreman-installer answer file to foremanctl configuration format. + + This command reads foreman-installer answer files and converts them to the + new foremanctl configuration format. Unmappable parameters are reported + as warnings but do not cause the command to fail. + +variables: + answer_file: + help: Path to the foreman-installer answer file to migrate. If not specified, attempts to read from the default location. + persist: false + output: + help: Path where the migrated configuration should be written. If not specified, outputs to stdout. + persist: false diff --git a/src/playbooks/migrate/migrate.yaml b/src/playbooks/migrate/migrate.yaml new file mode 100644 index 000000000..6fa2ec577 --- /dev/null +++ b/src/playbooks/migrate/migrate.yaml @@ -0,0 +1,29 @@ +--- +- name: Migrate foreman-installer answer file to foremanctl format + hosts: + - quadlet + gather_facts: false + tasks: + - name: Run migration + migrate_answers: + answer_file: "{{ answer_file | default(omit) }}" + output: "{{ output | default(omit) }}" + working_directory: "{{ lookup('env', 'PWD') }}" + register: migration_result + + - name: Display migrated configuration to stdout + when: migration_result.output_content | default('') != '' + ansible.builtin.debug: + msg: "{{ migration_result.output_content }}" + + - name: Display migration results + ansible.builtin.debug: + msg: + - "Migration completed successfully!" + - "Mapped parameters: {{ migration_result.mapped_count }}" + - "Unmappable parameters: {{ migration_result.unmappable_count }}" + - "{{ _unmappable_warning if migration_result.unmappable | length > 0 else '' }}" + - "{{ _output_file_msg if migration_result.output_file is defined else '' }}" + vars: + _unmappable_warning: "Warning: {{ migration_result.unmappable | length }} parameter(s) could not be mapped - see warnings above" + _output_file_msg: "Output written to: {{ migration_result.output_file }}" diff --git a/src/plugins/modules/migrate_answers.py b/src/plugins/modules/migrate_answers.py new file mode 100755 index 000000000..6eee062e8 --- /dev/null +++ b/src/plugins/modules/migrate_answers.py @@ -0,0 +1,242 @@ +#!/usr/bin/python3 + +import os +import sys +import yaml +from ansible.module_utils.basic import AnsibleModule + + +def cast_database_mode(value): + """Convert db_manage boolean to database_mode string.""" + if isinstance(value, bool): + return 'internal' if value else 'external' + return value + + +PARAMETER_MAP = { + # Database configuration + ('foreman', 'db_host'): 'database_host', + ('foreman', 'db_port'): 'database_port', + ('foreman', 'db_database'): 'foreman_database_name', + ('foreman', 'db_username'): 'foreman_database_user', + ('foreman', 'db_password'): 'foreman_database_password', + ('foreman', 'db_manage'): ('database_mode', cast_database_mode), + ('foreman', 'db_manage_rake'): 'IGNORE', # Not needed in new format + + # Foreman configuration + ('foreman', 'initial_admin_username'): 'foreman_initial_admin_username', + ('foreman', 'initial_admin_password'): 'foreman_initial_admin_password', + + # Certificate configuration + ('foreman', 'server_ssl_cert'): 'server_certificate', + ('foreman', 'server_ssl_key'): 'server_key', + ('foreman', 'server_ssl_ca'): 'ca_certificate', + + # TODO: Add more mappings as discovered +} + +IGNORE_PARAMS = {'IGNORE'} + + +def resolve_answer_file_from_scenario(scenario_file='/etc/foreman-installer/scenarios.d/last_scenario.yaml'): + """Read scenario file and extract answer file path from :answer_file: key.""" + with open(scenario_file, 'r') as f: + try: + scenario_data = yaml.safe_load(f) + if scenario_data is None: + raise ValueError(f"Scenario file {scenario_file} is empty") + + answer_file = scenario_data.get(':answer_file') + if not answer_file: + raise ValueError(f"Scenario file does not contain :answer_file: key") + + return answer_file + except yaml.YAMLError as e: + raise ValueError(f"Invalid YAML in scenario file: {e}") + + +def load_answer_file(file_path): + """Load and parse YAML answer file.""" + with open(file_path, 'r') as f: + try: + data = yaml.safe_load(f) + if data is None: + raise ValueError(f"Answer file {file_path} is empty") + return data + except yaml.YAMLError as e: + raise ValueError(f"Invalid YAML in answer file: {e}") + + +def validate_answer_file(data, file_path): + """ + Validate that the loaded YAML has the expected structure of a foreman-installer answer file. + + Expected structure: Top-level keys should be module names (strings) with nested dictionaries. + Example: {'foreman': {'db_host': 'localhost'}, 'katello': {...}} + """ + if not isinstance(data, dict): + raise ValueError( + f"Answer file {file_path} has invalid structure. " + f"Expected a dictionary but got {type(data).__name__}" + ) + + if len(data) == 0: + raise ValueError(f"Answer file {file_path} is empty (contains no configuration)") + + dict_values = sum(1 for v in data.values() if isinstance(v, dict)) + + if dict_values == 0: + raise ValueError( + f"Answer file {file_path} does not appear to be a valid foreman-installer answer file. " + f"Expected nested module configurations (e.g., 'foreman:', 'katello:') but found flat structure" + ) + + +def flatten_nested_dict(nested_dict, parent_key=''): + """ + Flatten nested dictionary structure from foreman-installer format. + + Example: + {'foreman': {'db_host': 'localhost'}} + becomes: + {('foreman', 'db_host'): 'localhost'} + """ + items = [] + for key, value in nested_dict.items(): + if isinstance(value, dict): + items.extend(flatten_nested_dict(value, key).items()) + else: + if parent_key: + items.append(((parent_key, key), value)) + else: + items.append((key, value)) + return dict(items) + + +def apply_mappings(old_config): + """ + Transform old config to new format using mapping table. + + Returns: + dict: { + 'mapped': {new_param: value}, + 'unmappable': [old_param_name, ...] + } + """ + flat_config = flatten_nested_dict(old_config) + + result = {} + unmappable = [] + + for old_key, old_value in flat_config.items(): + if old_key in PARAMETER_MAP: + mapping = PARAMETER_MAP[old_key] + + if mapping == 'IGNORE': + continue + + if isinstance(mapping, tuple): + new_key, transform_func = mapping + new_value = transform_func(old_value) + else: + new_key = mapping + new_value = old_value + + if new_value is not None: + result[new_key] = new_value + else: + if old_value is not None: + if isinstance(old_key, tuple): + param_name = '::'.join(old_key) + else: + param_name = str(old_key) + unmappable.append(param_name) + + return { + 'mapped': result, + 'unmappable': unmappable + } + + +def write_output(data, output_path=None, working_directory=None): + """Write migrated configuration to file or return as string.""" + yaml_content = yaml.dump(data, default_flow_style=False, sort_keys=True) + + if output_path: + if working_directory and not os.path.isabs(output_path): + absolute_path = os.path.join(working_directory, output_path) + else: + absolute_path = os.path.abspath(output_path) + with open(absolute_path, 'w') as f: + f.write(yaml_content) + return absolute_path + else: + return yaml_content + + +def run_module(): + module_args = dict( + answer_file=dict(type='str', required=False, default=None), + output=dict(type='str', required=False, default=None), + working_directory=dict(type='str', required=False, default=None), + ) + + result = dict( + changed=False, + mapped_count=0, + unmappable_count=0, + unmappable=[], + output_content='', + ) + + module = AnsibleModule( + argument_spec=module_args, + supports_check_mode=True + ) + + try: + answer_file = module.params.get('answer_file') + if not answer_file: + answer_file = resolve_answer_file_from_scenario() + + old_config = load_answer_file(answer_file) + validate_answer_file(old_config, answer_file) + + migration_result = apply_mappings(old_config) + + result['mapped_count'] = len(migration_result['mapped']) + result['unmappable_count'] = len(migration_result['unmappable']) + result['unmappable'] = migration_result['unmappable'] + + # Issue warnings for unmappable parameters + if migration_result['unmappable']: + for param in migration_result['unmappable']: + module.warn(f"Parameter '{param}' could not be mapped and will need manual review") + + if not module.check_mode: + output_path = module.params.get('output') + working_directory = module.params.get('working_directory') + + if output_path: + absolute_path = write_output(migration_result['mapped'], output_path, working_directory) + result['output_file'] = absolute_path + result['changed'] = True + else: + # Output to stdout - store in result so Ansible displays it + yaml_content = write_output(migration_result['mapped'], output_path, working_directory) + result['output_content'] = yaml_content + + module.exit_json(**result) + + except (FileNotFoundError, PermissionError, ValueError) as e: + module.fail_json(msg=str(e), **result) + except Exception as e: + module.fail_json(msg=f"Unexpected error: {e}", **result) + + +def main(): + run_module() + + +if __name__ == '__main__': + main() diff --git a/tests/unit/migrate_test.py b/tests/unit/migrate_test.py new file mode 100644 index 000000000..cf8a3a5f4 --- /dev/null +++ b/tests/unit/migrate_test.py @@ -0,0 +1,198 @@ +import pytest +import sys +import os +import tempfile +import yaml + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), '../../src/plugins/modules')) + +import migrate_answers + + +class TestParameterMapping: + """Test the parameter mapping logic""" + + def test_simple_parameter_mapping(self): + """Test basic parameter translation""" + old_config = { + 'foreman': { + 'db_host': 'localhost', + 'db_port': 5432 + } + } + + result = migrate_answers.apply_mappings(old_config) + + assert result['mapped']['database_host'] == 'localhost' + assert result['mapped']['database_port'] == 5432 + assert result['unmappable'] == [] + + def test_parameter_transformation(self): + """Test parameters that need value transformation""" + old_config = { + 'foreman': { + 'db_manage': True + } + } + + result = migrate_answers.apply_mappings(old_config) + + assert result['mapped']['database_mode'] == 'internal' + + old_config['foreman']['db_manage'] = False + result = migrate_answers.apply_mappings(old_config) + assert result['mapped']['database_mode'] == 'external' + + def test_ignore_parameters(self): + """Test parameters marked as IGNORE""" + old_config = { + 'foreman': { + 'db_manage_rake': True, + 'db_host': 'localhost' + } + } + + result = migrate_answers.apply_mappings(old_config) + + assert 'db_manage_rake' not in result['mapped'] + assert 'db_manage_rake' not in str(result['unmappable']) + assert result['mapped']['database_host'] == 'localhost' + + def test_unmappable_parameters(self): + """Test that unmappable parameters are reported""" + old_config = { + 'foreman': { + 'unknown_param': 'value' + }, + 'katello': { + 'enable_ostree': True + } + } + + result = migrate_answers.apply_mappings(old_config) + + assert 'foreman::unknown_param' in result['unmappable'] + assert 'katello::enable_ostree' in result['unmappable'] + assert len(result['unmappable']) == 2 + + def test_empty_config(self): + """Test with empty configuration""" + old_config = {} + + result = migrate_answers.apply_mappings(old_config) + + assert result['mapped'] == {} + assert result['unmappable'] == [] + + def test_mixed_config(self): + """Test with mix of mappable and unmappable parameters""" + old_config = { + 'foreman': { + 'db_host': 'database.example.com', + 'db_port': 5432, + 'unknown_param': 'test', + 'initial_admin_username': 'admin' + } + } + + result = migrate_answers.apply_mappings(old_config) + + assert result['mapped']['database_host'] == 'database.example.com' + assert result['mapped']['database_port'] == 5432 + assert result['mapped']['foreman_initial_admin_username'] == 'admin' + + assert 'foreman::unknown_param' in result['unmappable'] + assert len(result['unmappable']) == 1 + + def test_skip_none_values_in_unmappable(self): + """Test that None values are not added to unmappable list""" + old_config = { + 'foreman': { + 'db_host': 'localhost', + 'unknown_param': None, + 'another_unknown': 'value' + } + } + + result = migrate_answers.apply_mappings(old_config) + + assert result['mapped']['database_host'] == 'localhost' + assert 'foreman::unknown_param' not in result['unmappable'] + assert 'foreman::another_unknown' in result['unmappable'] + assert len(result['unmappable']) == 1 + + +class TestFileOperations: + """Test file loading and writing""" + + def test_load_valid_yaml(self): + """Test loading a valid YAML file""" + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f: + yaml.dump({'foreman': {'db_host': 'localhost'}}, f) + temp_file = f.name + + try: + result = migrate_answers.load_answer_file(temp_file) + assert result == {'foreman': {'db_host': 'localhost'}} + finally: + os.unlink(temp_file) + + def test_load_empty_yaml(self): + """Test loading an empty YAML file raises ValueError""" + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f: + f.write('---\n') + temp_file = f.name + + try: + with pytest.raises(ValueError, match="is empty"): + migrate_answers.load_answer_file(temp_file) + finally: + os.unlink(temp_file) + + def test_load_nonexistent_file(self): + """Test loading a file that doesn't exist""" + with pytest.raises(FileNotFoundError): + migrate_answers.load_answer_file('/nonexistent/file.yaml') + + def test_load_invalid_yaml(self): + """Test loading invalid YAML""" + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f: + f.write('invalid: yaml: content: [') + temp_file = f.name + + try: + with pytest.raises(ValueError): + migrate_answers.load_answer_file(temp_file) + finally: + os.unlink(temp_file) + + def test_write_output_to_file(self): + """Test writing output to a file""" + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f: + temp_file = f.name + + try: + test_data = {'database_host': 'localhost', 'database_port': 5432} + migrate_answers.write_output(test_data, temp_file) + + with open(temp_file, 'r') as f: + result = yaml.safe_load(f) + assert result == test_data + finally: + os.unlink(temp_file) + + +class TestTransformations: + """Test individual transformation functions""" + + def test_cast_database_mode_true(self): + """Test database mode transformation with True""" + assert migrate_answers.cast_database_mode(True) == 'internal' + + def test_cast_database_mode_false(self): + """Test database mode transformation with False""" + assert migrate_answers.cast_database_mode(False) == 'external' + + def test_cast_database_mode_passthrough(self): + """Test database mode transformation with non-boolean""" + assert migrate_answers.cast_database_mode('already_a_string') == 'already_a_string'