Skip to content

Refactor to allow mixed Python Rust code - #444

Closed
relf wants to merge 8 commits into
masterfrom
python-wrap
Closed

Refactor to allow mixed Python Rust code #444
relf wants to merge 8 commits into
masterfrom
python-wrap

Conversation

@relf

@relf relf commented Jul 20, 2026

Copy link
Copy Markdown
Owner

Until now the Rust pyo3 project was directly used from Python. With this PR Python wraps the Rust code we get the following structure:

Repository Layout

The project is organized as a Cargo workspace. Core optimization algorithms are implemented as reusable Rust crates. The Python distribution, including the PyO3 bindings, is contained entirely within the python/ directory.

repo/
│
├── Cargo.toml                  # Cargo workspace
├── README.md
├── LICENSE
│
├── crates/
│   ├── doe/
│   ├── gp/
│   ├── moe/
│   └── ego/
│
├── python/
│   ├── Cargo.toml              # PyO3 extension crate
│   ├── pyproject.toml
│   ├── src/
│   │   └── lib.rs
│   │
│   ├── egobox/
│   │   ├── __init__.py
│   │   ├── _core.py
│   │   ├── optimizer.py
│   │   ├── models.py
│   │   ├── acquisition.py
│   │   ├── utils.py
│   │   ├── exceptions.py
│   │   ├── _rust.pyi
│   │   └── py.typed
│   │
│   ├── tests/
│   └── examples/
│
└── website/

Workspace Organization

The Cargo workspace consists of reusable Rust crates:

Crate Responsibility
doe Design of Experiments
gp Gaussian Process models
moe Multi-objective optimization
ego Bayesian optimization engine
python PyO3 bindings exposing the Rust API to Python

The Python binding crate depends on the workspace crates but does not implement optimization algorithms itself.


Python Distribution

The python/ directory is a self-contained Python project.

It contains:

  • the PyO3 extension crate (Cargo.toml, src/)
  • the Python package (egobox/)
  • pyproject.toml
  • Python tests
  • Python examples

Building the Python package is always performed from the python/ directory.


Maturin Configuration

python/pyproject.toml configures maturin.

[tool.maturin]
python-source = "."
module-name = "egobox._rust"

This installs the compiled extension as

egobox/
    _rust.so

alongside the Python modules.


Public API

Only the Python package is public.

import egobox

The compiled module

egobox._rust

is private and should never be imported directly by users.

Python modules wrap the Rust implementation, perform validation, convert NumPy arrays, and expose an idiomatic object-oriented API.


Design Principles

  • Rust crates contain reusable algorithms.
  • The Python binding crate only exposes Rust functionality through PyO3.
  • The Python package defines the public API.
  • Users interact exclusively with the Python package.
  • Rust implementation details remain hidden behind Python wrappers.

@relf
relf marked this pull request as ready for review July 20, 2026 16:31
@relf relf mentioned this pull request Jul 21, 2026
@relf relf closed this in #445 Jul 21, 2026
@relf
relf deleted the python-wrap branch July 21, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant