Skip to content

Add Initial Mirage Node Type Support with Dedicated Configuration and Refined Init Checks#851

Merged
dmytrotkk merged 41 commits into
miragefrom
mirage-boot
May 13, 2025
Merged

Add Initial Mirage Node Type Support with Dedicated Configuration and Refined Init Checks#851
dmytrotkk merged 41 commits into
miragefrom
mirage-boot

Conversation

@Masterix0

@Masterix0 Masterix0 commented Apr 17, 2025

Copy link
Copy Markdown
Contributor

Problem

The SKALE ecosystem requires support for a new node type, "Mirage," which operates differently from the standard SKALE node.
node-cli needs to be adapted to manage this new node type, including building a specific binary, providing dedicated CLI commands, handling Mirage-specific static parameters, and ensuring core functionalities correctly differentiate behavior, particularly around initialization and resource management.

Solution

This pull request introduces foundational and refined support for the Mirage node type within node-cli.
It establishes a distinct build target for Mirage, introduces Mirage-specific CLI commands and core logic, and significantly refactors shared components to be node-type aware, especially in configuration handling and initialization.

Key Changes Overview:

  1. Mirage Build Target & CLI Structure:

    • Introduced build script and CI/CD workflow modifications to generate and publish a separate mirage binary.
    • Added new CLI command groups (mirage boot, mirage node) for the Mirage build type.
    • Implemented core logic for essential Mirage operations (boot init, migration, restore), with other commands reusing existing functionalities.
  2. NodeType-Aware Architecture & Configuration Handling:

    • Implemented a NodeType enum (REGULAR, SYNC, MIRAGE) to drive conditional logic throughout the codebase (node management, operations, Docker utilities, Nginx).
    • Introduced support for a dedicated mirage_static_params.yaml for Mirage-specific static configurations, allowing Mirage nodes to use standard ENV_TYPEs (e.g., mainnet, devnet) while sourcing distinct parameters. Core configuration loading (get_static_params) is now NodeType-aware.
  3. Refined Initialization and Resource Management:

    • Standardized the node initialization check (is_node_inited) to rely on the presence of nginx.conf, a common artifact for all initialized node types.
    • Ensured that Mirage-specific workflows correctly bypass SKALE's standard resource allocation calculations and shared BTRFS volume setup logic, aligning with Mirage's different resource model.
  4. Testing & Documentation:

    • Added and updated dedicated test suites for Mirage CLI commands, core logic, static parameter loading, ENV_TYPE validation, and initialization checks.
    • Updated the README.md to include instructions for the mirage binary, its commands, and its use of standard ENV_TYPEs with the new static parameter approach.

Benefits

  • Extensibility: Adds robust support for the Mirage node type, paving the way for future node types.
  • Clearer Configuration: Mirage has its own clean static parameter file (mirage_static_params.yaml), simplifying management and avoiding bloat in the common static_params.yaml.
  • Maintainability: Encapsulates Mirage-specific logic and uses NodeType for clearer control flow. Shared components like initialization checks and host requirement validation are now more universally applicable or correctly specialized.
  • Correctness: Ensures Mirage nodes adhere to their specific operational model by bypassing inapplicable SKALE resource management schemes.
  • Alignment: Integrates with corresponding skale-admin changes and the dedicated docker-compose-mirage.yml.
  • Test Coverage: Includes dedicated and adapted tests for new Mirage functionality and configuration handling.
  • Usability: Provides clear documentation for the mirage binary and its updated configuration paradigm.

Testing Done

  • Added new integration tests for Mirage CLI commands (mocking core layer).
  • Added new unit/integration tests for Mirage core functions (mocking operations layer).
  • Updated environment variable validation tests for Mirage types.
  • Adapted existing core/check/utility tests to handle the NodeType parameter and verify behavior across types.
  • CI workflows updated to build and test the Mirage binary type.

Reviewer Instructions

  1. High-Level Architecture:
    • Focus on the overall strategy for supporting multiple node types via NodeType.
    • Assess the introduction of mirage_static_params.yaml and the NodeType-aware changes to get_static_params.
  2. Mirage Workflow & Configuration:
    • Conceptually review how Mirage nodes (boot and regular) source their configurations and initialize.
    • Verify that Mirage operations correctly bypass resource allocation and shared volume logic as intended.
    • Check the updated ENV_TYPE validation for Mirage.
  3. Initialization Check:
    • Review the updated is_node_inited logic and its impact on relevant decorators and commands.
  4. Testing:
    • Assess the updated test fixtures in tests/conftest.py for both static parameter files.
    • Ensure test adaptations in core_checks_test.py, configs_env_validate_test.py, and Mirage-specific tests correctly reflect the new configuration and initialization logic.
  5. Documentation:
    • Verify README.md updates, especially regarding ENV_TYPE usage for Mirage and any mention of static parameters.

Implements the foundational structure for the Mirage node type within the node-cli, including the build system integration and the core `mirage boot` commands.

Changes include:
- Added a new 'mirage' build type to `scripts/build.sh` and related scripts to generate the `skale-*-mirage` binary.
- Created new CLI files `node_cli/cli/mirage_boot.py` and `node_cli/cli/mirage_node.py`.
- Added the `mirage boot` command group with subcommands:
    - `init`: Implements logic to run the `mirage-boot` service from `docker-compose-mirage.yml`, ensuring `SKALE_NETWORK_TYPE=mirage` is set.
    - `register`: Reuses existing core registration logic.
    - `signature`: Reuses existing core signature logic.
    - `migrate`: Implements logic to switch from mirage boot to mirage main.
- Integrated the base `mirage` command group and existing `logs`, `wallet`, `ssl` groups into `main.py` for the `mirage` build type.
- Updated `README.md` with installation instructions for the `mirage` binary.
Completes the implementation of the `mirage node` command namespace by adding functionality for signature retrieval, backup, and restore operations.

Changes include:
- Implemented `mirage node signature` command, reusing the core `get_node_signature` function.
- Implemented `mirage node backup` command, reusing the core `backup` function.
- Added `restore_mirage` function to `node_cli/operations/base.py` to handle Mirage-specific restoration logic.
- Implemented `mirage node restore` command, utilizing the new `restore_mirage` function.
- Updated `node_cli/operations/base.py` functions (`turn_off`, `update_sync`) to use `NodeTypes` enum for better clarity (refactor).
- Fixed datetime formatting and typo in `node_cli/core/node.py`.
- Removed placeholder file `node_cli/operations/mirage_boot.py` as logic is integrated elsewhere.

Placeholders remain for `mirage node init` and `mirage node register`.
@Masterix0 Masterix0 self-assigned this Apr 17, 2025
Comment thread node_cli/core/node.py
Comment thread node_cli/cli/mirage_node.py Outdated
Comment thread node_cli/core/node.py Outdated
Comment thread node_cli/core/node.py Outdated
Comment thread node_cli/operations/base.py
Comment thread node_cli/core/nginx.py Outdated
Comment thread node_cli/utils/docker_utils.py Outdated
Comment thread node_cli/utils/docker_utils.py Outdated
Comment thread node_cli/utils/meta.py Outdated
Comment thread node_cli/core/node.py Outdated
…ng. Removed some unused options from mirage commands.
Build broke because info.py need NodeType from core.node,
but core.node need env.py, and env.py need NodeType from core.node.
Make circle. Bad.

Move NodeType to utils/node_types.py.
Update imports in core.node, configs.env, generate_info.sh.

No more circle. Build work now.
@Masterix0 Masterix0 marked this pull request as ready for review April 29, 2025 11:33
Comment thread node_cli/cli/mirage_node.py Outdated
Comment thread node_cli/configs/env.py Outdated
Comment thread node_cli/core/mirage_boot.py Outdated
Comment thread node_cli/core/nginx.py Outdated
Comment thread node_cli/core/node.py
Comment thread node_cli/core/node.py Outdated
Comment thread tests/core/nginx_test.py Outdated
Comment thread node_cli/utils/docker_utils.py
Comment thread node_cli/utils/docker_utils.py
Comment thread .github/workflows/publish.yml Outdated
Comment thread node_cli/cli/node.py
Comment thread node_cli/core/mirage_boot.py Outdated
@Masterix0 Masterix0 changed the title Add initial Mirage node type support Add Initial Mirage Node Type SUpport with Dedicated Configuration and Refined Init Checks May 12, 2025
Comment thread node_cli/configs/env.py Outdated
Comment thread text.yml Outdated
Comment thread text.yml Outdated
Comment thread tests/core/core_node_test.py Outdated
Comment thread tests/core/core_mirage_test.py Outdated
Comment thread tests/core/core_mirage_test.py Outdated
Comment thread tests/conftest.py Outdated
Comment thread tests/conftest.py Outdated
@dmytrotkk dmytrotkk merged commit e7dfb97 into mirage May 13, 2025
2 checks passed
@dmytrotkk dmytrotkk deleted the mirage-boot branch May 13, 2025 12:09
@Masterix0 Masterix0 changed the title Add Initial Mirage Node Type SUpport with Dedicated Configuration and Refined Init Checks Add Initial Mirage Node Type Support with Dedicated Configuration and Refined Init Checks May 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants