Skip to content

Feature/python solver#63

Open
chemiskyy wants to merge 81 commits intomasterfrom
feature/python_solver
Open

Feature/python solver#63
chemiskyy wants to merge 81 commits intomasterfrom
feature/python_solver

Conversation

@chemiskyy
Copy link
Member

This pull request introduces major updates for Simcoon v2.0, focusing on replacing legacy C++ executables and file-based workflows with a modern Python-based solver API and JSON configuration. The changes improve usability, documentation, and build system robustness, while removing outdated components and updating dependencies.

Core Build System and Dependency Updates

  • The build system now attempts to find nlohmann_json via system/conda before fetching from GitHub, and explicitly links nlohmann_json::nlohmann_json to the simcoon target for improved JSON support and compatibility. [1] [2] [3]

Removal of Legacy Executables and Scripts

  • Legacy standalone executables (solver, identification, L_eff, Elastic_props, ODF, PDF) have been removed from the build and install scripts; users are directed to use the new Python API for solver workflows. [1] [2]

Documentation and Migration Guidance

  • Added a comprehensive migration guide (docs/migration_guide.md) detailing how to transition from the legacy C++ solver to the Python API, including code examples for simulation, multi-step loading, JSON configuration, and identification workflows.
  • Updated the README.md with a Python quick start example and clarified the transition to the Python API.

Micromechanics and Homogenization Documentation

  • Expanded documentation for micromechanics and homogenization, including Python API usage for phase configuration, JSON I/O functions, and examples for composite materials. [1] [2] [3] [4] [5]

Summary of Most Important Changes

Build System and Dependencies

  • Improved handling of nlohmann_json dependency in CMakeLists.txt and conda recipe, ensuring robust JSON support for new Python and C++ features. [1] [2] [3]

Legacy Executables Removal

  • Removed legacy C++ executables from build and install scripts, fully transitioning solver workflows to the Python API. [1] [2]

Migration and Python API

  • Added docs/migration_guide.md with detailed migration instructions, mapping legacy functions to new Python classes and JSON-based workflows.
  • Updated README.md with Python quick start and guidance for new users.

Micromechanics and Homogenization Documentation

  • Introduced and expanded documentation for Python-based phase configuration, JSON I/O, and homogenization workflows, including usage examples and format specifications. [1] [2] [3] [4] [5]

These changes make Simcoon more user-friendly, modernize the workflow, and provide clear guidance for users migrating from legacy C++ tools to the new Python-based ecosystem.

Update C++ API docs to recommend using the Python simcoon.solver.micromechanics module for new projects and mark legacy C++ I/O helpers as deprecated. Added a prominent note to docs/cpp_api/simulation/phase.rst outlining benefits (JSON configs, dataclasses, .dat conversion utilities) with a short usage example, and updated docs/cpp_api/simulation_overview.md to flag read.hpp/write.hpp as deprecated and link to the Micromechanics I/O documentation.
Add a new docs page (docs/simulation/micromechanics.rst) documenting the simcoon.solver.micromechanics Python module: data classes (Phase, Ellipsoid, Cylinder, Layer, Section), JSON I/O functions, JSON format examples, legacy .dat conversion helpers, and standalone usage notes. Update docs/simulation/index.rst to include the new micromechanics page, and extend docs/homogenization/index.rst with a Phase Configuration example showing how to create and save ellipsoid phases with the micromechanics API. These docs introduce JSON-based micromechanics I/O as the recommended replacement for legacy .dat formats and provide migration guidance.
Add a sample ellipsoids.json and update the heterogeneous example to use JSON-based phase configurations instead of legacy .dat files. README.rst now documents how to load, modify, and save ellipsoid phase data and notes the deprecated .dat format and migration helper. effective_props.py is updated to import load/save helpers, read ellipsoids.json, modify semi-axes programmatically for aspect-ratio sweeps, save changes, and restore original values (removing direct file edits of Nellipsoids0.dat).
Update C++ API documentation to recommend the Python JSON-based micromechanics interface. phase.rst replaces the old admonition with a plain "Python Interface" section, streamlines the guidance and example import formatting. simulation_overview.md replaces the deprecated read.hpp entry with read_json.hpp, removes the deprecation marker from write.hpp, and replaces the legacy note with a concise pointer to the Micromechanics I/O docs and JSON format. These changes clarify that JSON configuration and the Python dataclasses should be used for new projects.
Add a comprehensive migration guide (docs/migration_guide.md) for Simcoon v2.0. The guide explains the new Python Solver API (Solver, Block, StepMeca/StepThermomeca), JSON configuration loading, material identification tools, improved history/state access, control/corate type mappings, and unchanged micromechanics functions. It includes migration examples for basic and multi-step simulations, parameter identification workflows, common issues and troubleshooting tips, and links to documentation, examples, and issues.
Remove legacy .dat/.txt conversion and deprecation notes and update documentation to use JSON-based I/O. micromechanics.rst: drop legacy conversion sections and add JSON example for cylinders. solver.rst: replace path.txt/material.dat examples with JSON schemas and examples, document JSON fields (control_type, corate_type, blocks, steps, spin, F, DEtot, Dsigma, etc.), update usage to show simcoon.solver.io load/save functions, and remove references to legacy conversion helpers. Also adjust descriptions and examples for mechanical, thermomechanical and finite-deformation cases to reflect the new JSON formats.
Rewrite the objective_rates example to use the new Python Solver API (Solver/Block/StepMeca), remove legacy file-based solver calls, and compute/plot results directly from the solver history. Add stress-comparison plots and a summary printout; compute rotation and organize results per objective rate (Jaumann, Green-Naghdi, Logarithmic). Also update examples/analysis/README.rst: improve formatting, add example descriptions, quick-start usage with the new API, run instructions, and links to related docs.
Refactor example suite to use the new Python Solver API (Solver / Block / StepMeca) instead of the legacy simcoon.solver file-based interface. Key changes:

- Updated examples/continuum_mechanics/README.rst: expanded documentation and usage examples for tensor operations, stress measures, rotations and yield criteria.
- Renamed examples/heterogeneous/data/ellipsoids.json -> ellipsoids0.json and updated examples/heterogeneous/effective_props.py to load the new filename.
- Refactored hyperelastic and UMAT example scripts (examples/hyperelasticity/HYPER_umat.py and multiple files under examples/umats/ such as ELISO.py, ELIST.py, ELORT.py, EPCHA.py, etc.) to build StepMeca/Block objects, call Solver.solve(), and extract results from the returned history rather than reading/writing solver output files. Added convenience helper functions, plotting improvements, and basic verification prints.

This is primarily an API migration and documentation update to use the in-process solver API and improve example clarity; no algorithmic model changes were introduced.
Move Phase I/O to JSON and modernize headers for v2.0. Replaced legacy Phase read/write headers with a new include/simcoon/Simulation/Phase/read_json.hpp that provides JSON-based read/write APIs for ellipsoids, layers, cylinders and generic phases; removed the old read.hpp and write.hpp. Enhanced geometry headers (cylinder, ellipsoid, layer) with improved Doxygen, examples, and expanded cylinder class declaration (constructors, destructor, assignment, doc-comments, ostream friend). Identification script header (Identification/script.hpp) was updated to mark legacy identification functions as deprecated, add Python/scipy usage notes and deprecation attributes. Solver headers were cleaned up: read.hpp documents/marks legacy functions (read_matprops noted and other legacy helpers retained/deprecated) and solver.hpp removes the old legacy solver() signature while adding notes pointing to the Python API. Overall this commit modernizes I/O to JSON, improves documentation, and deprecates/redirects legacy C++ workflows in favor of the Python-based API.
Add example Python scripts and accompanying material/path JSON configs for two models: ELISO (isotropic linear elasticity) and EPICP (isotropic J2 plasticity with isotropic hardening). Files added: python-setup/examples/ELISO/{example_eliso.py,material.json,path.json} and python-setup/examples/EPICP/{example_epicp.py,material.json,path.json}. Examples demonstrate the new Python solver API (programmatic and JSON-driven usage), cyclic loading cases, plotting of stress–strain and state-variable histories, and saving result figures.
Add a set of micromechanics examples demonstrating JSON-based phase definitions and conversion utilities. New JSON files: ellipsoids.json, fiber_composite.json, glass_epoxy_composite.json, laminate_090.json, and layers.json provide example ellipsoid and laminate phase configurations. Add example_micromechanics.py which shows programmatic creation of Layer and Ellipsoid objects, saving/loading JSON via the simcoon.solver.micromechanics API, computes simple Voigt/Reuss bounds, and includes helpers to convert legacy .dat phase files to JSON. These examples document usage of the micromechanics module and allow running examples without building simcoon._core.
Introduce a new simcoon.identification subpackage and enhance the top-level simcoon package init.

- Added python-setup/simcoon/identification/__init__.py: provides module-level docs, examples, and re-exports core APIs (IdentificationProblem, OptimizationResult, ParameterSpec), optimizers (levenberg_marquardt, differential_evolution, hybrid_optimization, nelder_mead), cost functions (mse, mae, r2, weighted_mse, huber_loss, CostFunction) and sensitivity utilities (compute_sensitivity, compute_jacobian, correlation_matrix). Notes that it replaces legacy C++ identification functions.
- Modified python-setup/simcoon/__init__.py: added package docstring/quickstart and imported common submodules (solver, identification, properties, odf, pdf, parameter, constant, data) for convenient top-level access while retaining the backward compatibility alias (simmit).

This change exposes the new Python-based identification API and improves discoverability of submodules from simcoon.
Introduce a new identification subpackage with four modules: problem, cost_functions, optimizers, and sensitivity. problem.py defines ParameterSpec, IdentificationProblem and OptimizationResult including handling of fixed parameters, bounds/initials, simulation wrapping, residual vector construction, interpolation for mismatched lengths, and eval counting. cost_functions.py provides common metrics (MSE, MAE, RMSE, NRMSE, Huber, R²) and an abstract CostFunction with class implementations. optimizers.py implements optimization routines (Levenberg–Marquardt, differential evolution, Nelder–Mead and a hybrid global+local flow) that wrap scipy optimizers and attempt covariance estimation. sensitivity.py offers finite-difference sensitivity, Jacobian, correlation matrix, parameter uncertainty and identifiability checks to help assess parameter interactions and identifiability.
Introduce two new modules under python-setup/simcoon: odf.py and pdf.py. odf.py provides ODF data classes (ODFPeak, ODF), C++ bindings wrappers (get_densities, discretize via simcoon._core), file I/O (create/load/save peaks), ODF generators (random, uniform, fiber) and plotting utilities. pdf.py implements common probability distributions (lognormal, weibull, normal, gamma, mixtures), PDF discretization/discretize_to_phases, distribution fitting and statistics (fit_*, fit_distribution, pdf_statistics) and plotting helpers; it uses scipy when available for more accurate routines. Both modules include docstrings and examples and are intended to replace legacy executables for ODF/PDF handling.
Introduce a pure-Python 0D material-point solver and supporting I/O/micromechanics modules under python-setup/simcoon/solver. Adds:

- solver.py: Python implementation of the Newton–Raphson 0D solver, state-variable dataclasses (StateVariables, StateVariablesM/T), control/corate mappings, and in-place array handling; relies on simcoon._core for UMAT bindings.
- micromechanics.py: Standalone dataclasses for micromechanics (Phase, Layer, Ellipsoid, Cylinder, Section, orientations) with JSON load/save helpers so micromechanics work without _core.
- io.py: JSON-based material/path I/O, lazy-imports solver classes so material-only I/O works without building the C++ extension, and provides load_simulation_json to combine material + path.
- __init__.py: Exposes public API for solver, I/O and micromechanics helpers.

The changes enable JSON-configured simulations, micromechanics preprocessing without the C++ core, and minimize array copies for performance.
Add a comprehensive pytest suite under python-setup/test covering simcoon.identification, simcoon.solver.micromechanics and simcoon.solver (including UMAT integrations). Tests include ParameterSpec, cost functions, IdentificationProblem, optimizers (LM, Nelder-Mead, DE), sensitivity/Jacobian utilities and OptimizationResult; micromechanics dataclasses (Material/GeometryOrientation, Layer, Ellipsoid, Cylinder) and JSON I/O round-trips; solver primitives (StateVariables, Step/StepMeca/StepThermomeca, Block), Jacobian helpers (Lt_2_K, Lth_2_K) and high-level Solver behavior. UMAT/material-model tests (ELISO, ELIST, ELORT, EPICP) and some integration/homogenization tests are provided but marked to skip until simcoon._core and Python solver integration are available. Tests use numpy, tempfile and pytest fixtures where appropriate.
Remove outdated references to legacy executables and legacy module names and streamline top-level comments. Updates include: simplified identification docstring to mention scipy.optimize only; removed “replaces the legacy …” lines from odf, pdf, and properties docstrings; clarified alias/comment wording in __init__.py and adjusted submodule import comment. Files changed: __init__.py, identification/__init__.py, odf.py, pdf.py, properties.py.
Delete six deprecated C++ executables from the software/ directory: Elastic_props.cpp, L_eff.cpp, ODF.cpp, PDF.cpp, identification.cpp, and solver.cpp. These were standalone example/utility programs for simcoon (now removed as part of a cleanup/migration of functionality to the library or because they are no longer maintained). No other source files were changed.
Replace the legacy .dat-based phase readers with JSON-based readers: include read_json.hpp and call read_ellipsoid_json/read_layer_json instead of read_ellipsoid/read_layer. Update input filenames from "NellipsoidsN.dat"/"NlayersN.dat" to "ellipsoidsN.json"/"layersN.json". Remove the old reader implementation (src/Simulation/Phase/read.cpp). Minor local variable scope fix in umat_L_elastic.cpp (declare inputfile locally).
Delete a large set of legacy documentation (.rst) files under docs_old for Cpp and Python, including Continuum_Mechanics (Functions, Homogenization, Material, Micromechanics, Umat, Unit_cell, kinematics, criteria, damage, derivatives, etc.), Simulation (Geometry, Identification, Maths, Phase, Solver), and various index files. Cleans up outdated/duplicated documentation from the repository.
Delete legacy Identification headers under include/simcoon/Simulation/Identification. Removed files: constants.hpp, doe.hpp, generation.hpp, identification.hpp, individual.hpp, methods.hpp, opti_data.hpp, optimize.hpp, parameters.hpp, read.hpp, and script.hpp. This clears the old C++ parameter-identification interfaces and related utilities (script.hpp included deprecation notes referencing Python-based workflows).
Remove legacy data/parameter files and rewrite tutorials to use simcoon.solver JSON-based API. Deleted E_exp.txt, Nellipsoids0.dat and parameters.inp in 01A and 01B. Updated test.py in 01A and 01B to create Phase and Ellipsoid objects, save phases/ellipsoids as JSON in temporary directories, call sim.L_eff (MIMTN/MISCN), extract isotropic properties, and plot/save results. Also removed pandas/key-file parsing, added docstrings and clearer material/homogenization setup and logging.
Add a new test suite (python-setup/test/test_io.py) that exercises simcoon.solver.io JSON I/O for materials, paths, simulations, phases and sections, plus error handling. Extend test_micromechanics.py to include Phase/Section dataclass tests and JSON save/load coverage and add missing imports for phases/sections. Extend test_solver.py with HistoryPoint unit tests, control/corate mapping checks, advanced solver scenarios (skipped if core missing), and solver parameter handling tests. Also updated imports where needed to support the new tests.
Add _USE_MATH_DEFINES and cmath include for M_PI constant
which is not available by default on Windows MSVC builds.
- Increase number of increments for plasticity examples
- Use 'jaumann' corate_type which is more stable for mixed BC
- Change EPICP increment test to use realistic values (50-500)
- Ensure examples converge during Sphinx Gallery doc build
Mixed boundary conditions with Newton-Raphson iteration don't
converge for nonlinear materials (plasticity, hyperelasticity).
Use pure strain control instead with approximate lateral strains.

This is a known limitation - mixed BC for nonlinear materials
needs further development in the Python solver.
The switch statement for hyperelastic UMATs was missing case 25 (GETHH),
causing "Umat could not be found" error when running GETHH hyperelastic
model.
@chemiskyy chemiskyy force-pushed the feature/python_solver branch from 58ea907 to 7bac103 Compare January 31, 2026 22:05
Replace and greatly expand the migration guide to cover the Simcoon v2.0 API. migration_guide.md was rewritten with an executive summary, Table of Contents, detailed Solver/Block/Step class examples, JSON configuration I/O, identification module usage, micromechanics dataclasses and JSON examples, API comparison tables, common migration patterns, and troubleshooting tips. Added examples for identification optimizers, ParameterSpec, and migration helpers; updated code snippets to the new class-based workflow and JSON I/O utilities. Removed the old docs/migration_guide_v2.md file (now consolidated into migration_guide.md).
Add two new headers for the optimized C++ solver infrastructure:

- include/simcoon/Simulation/Solver_optimized/solver_engine.hpp: Declares SolverEngine and related types (HistoryPointCpp, BlockConfig, StepConfig, SolverParams) with pre-allocated Newton–Raphson buffers, backup/rollback state, kinematics helpers and method prototypes mirroring the Python solver API for compatibility. Uses Armadillo and existing state_variables_M types and includes GPL license header.

- include/simcoon/Simulation/Solver_optimized/umat_dispatch.hpp: Introduces UmatDispatch singleton to cache UMAT name lookup and provide a call_umat_M interface for efficient UMAT invocation (avoids rebuilding maps per call). Also includes documentation of parameters and uses Armadillo types.

These headers prepare the codebase for a faster, allocation-minimizing solver implementation by moving expensive setup into reusable structures and a static dispatch mechanism.
Expand UmatDispatch to act as a single source of truth for UMAT selection: bump version to 2.0, clarify purpose in comments, and add explicit support for three UMAT categories (mechanical small strain, mechanical finite strain, and thermomechanical). Introduces dedicated has_/get_/call_ methods and separate dispatch maps for each category (dispatch_map_M_, dispatch_map_M_finite_, dispatch_map_T_), plus a legacy has_umat alias for backward compatibility. These changes centralize name-to-function mapping and improve clarity and performance for varied UMAT calls.
Add a comprehensive solver architecture comparison and benchmarking script. Introduces docs/simulation/solver_comparison.rst and registers it in docs/simulation/index.rst. Adds a new benchmark script python-setup/test/benchmark_comprehensive.py to compare Legacy C++, Python, and C++ optimized solvers. Removes the obsolete python-setup/simcoon/solver.py implementation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant

Comments