Perform topological sort to order components by cmacmackin · Pull Request #428 · boutproject/hermes-3

cmacmackin · 2025-11-27T18:24:54Z

Using the access control information introduced in #421, this PR makes it possible for Hermes-3 to work out the order of components at run-time. This will make things far simpler and more robust for users. It will also fail faster if there is an unsatisfiable or circular dependency.

Closes #384.

cmacmackin · 2025-11-27T18:30:22Z

This is nearly done. Remaining tasks:

Merge Control access to state variables in transform method #421 and rebase.
There is one integration test giving me strange h5py errors on my computer. It doesn't look to be related to any changes I've made. We'll see what happens in CI. I seem to recall we saw something similar elsewhere at one point.
Update documentation.

codecov · 2025-11-27T18:32:44Z

Codecov Report

❌ Patch coverage is 53.89049% with 160 lines in your changes missing coverage. Please review.
✅ Project coverage is 49.60%. Comparing base (0a67d50) to head (ae0b58e).
⚠️ Report is 7 commits behind head on master.

Files with missing lines	Patch %	Lines
src/sheath_boundary_simple.cxx	0.00%	111 Missing ⚠️
src/anomalous_diffusion.cxx	35.00%	13 Missing ⚠️
include/component.hxx	63.15%	7 Missing ⚠️
src/component_scheduler.cxx	94.73%	0 Missing and 6 partials ⚠️
src/relax_potential.cxx	0.00%	6 Missing ⚠️
src/evolve_pressure.cxx	0.00%	5 Missing ⚠️
src/quasineutral.cxx	66.66%	1 Missing and 1 partial ⚠️
src/sheath_closure.cxx	81.81%	2 Missing ⚠️
src/vorticity.cxx	0.00%	2 Missing ⚠️
src/braginskii_friction.cxx	66.66%	1 Missing ⚠️
... and 5 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #428      +/-   ##
==========================================
+ Coverage   49.32%   49.60%   +0.27%     
==========================================
  Files          96       96              
  Lines       10029    10199     +170     
  Branches     1453     1481      +28     
==========================================
+ Hits         4947     5059     +112     
- Misses       4587     4629      +42     
- Partials      495      511      +16

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

mikekryjak · 2026-01-07T13:48:51Z

I missed this in #421, but I see that you added a long-needed feature - a better way to determine species type which is based on the charge:

hermes-3/src/component_scheduler.cxx

Lines 339 to 352 in 8fc17e0

    
           // FIXME: Would there be any spcies without AA? Is there any other 
        
           // reliable way to identify what is a species? 
        
           else if (component_options[name_trimmed].isSet("AA")) { 
        
             if (component_options[name_trimmed].isSet("charge")) { 
        
               const BoutReal charge = component_options[name_trimmed]["charge"]; 
        
               if (charge > 1e-5) { 
        
                 positive_ions.push_back(name_trimmed); 
        
               } else if (charge < -1e-5) { 
        
                 negative_ions.push_back(name_trimmed); 
        
               } else { 
        
                 neutrals.push_back(name_trimmed); 
        
               } 
        
             } else { 
        
               neutrals.push_back(name_trimmed);

There is already a function that does the same in hermes_utils and is used in a few places in the code:

hermes-3/include/hermes_utils.hxx

Lines 61 to 70 in 053c9bb

    
           /// Identify species name string as electron, ion or neutral 
        
           inline SpeciesType identifySpeciesType(const std::string& species) { 
        
             if (species == "e") { 
        
               return SpeciesType::electron; 
        
             } else if ((species == "i") or 
        
                        species.find(std::string("+")) != std::string::npos) { 
        
               return SpeciesType::ion; 
        
             } 
        
             // Not electron or ion -> neutral 
        
             return SpeciesType::neutral;

Ideally there should be only one tool to do this, and your new one seems more robust. Is there any reason not to replace the one in hermes_utils with the new one?

cmacmackin · 2026-01-07T13:54:37Z

Ideally there should be only one tool to do this, and your new one seems more robust. Is there any reason not to replace the one in hermes_utils with the new one?

Probably not. I just wasn't sure if people would be happy about me changing that bit of code. Note that this will require changing the function signature of identifySpeciesType so that it takes the Options instead of just a string.

mikekryjak · 2026-01-07T13:55:01Z

I went through the sorting algorithm and I see that you have left a good amount of comments on the individual bits. However, I found it difficult to get my head around what's going on just because of the amount of steps involved. It would be very useful to have a paragraph describing how the algorithm works step-by-step, either in the docs or in the comments (or both)

cmacmackin · 2026-01-07T13:57:29Z

There are some places (e.g., when writing tests) where it was convenient to just be able to have a list of species names and use the old heuristics to categorise them. Probably not a good enough reason to keep that though.

ZedThree

LGTM, thanks @cmacmackin!

There's some trivial bits that I'm happy to fix myself

ZedThree · 2026-01-07T11:41:13Z

+const std::set<std::string> ComponentScheduler::predeclared_variables = {
+    "time",          "linear",      "units:inv_meters_cubed", "units:eV", "units:Tesla",
+    "units:seconds", "units:meters"};


Given this is only used in this file, we can move the declaration out of the header and just have it here.

ZedThree · 2026-01-07T11:45:16Z

+
+/// Get all the parent sections of a variable "path". Sections are
+/// separated by colons in the path.
+std::set<std::string> getParents(const std::string& name) {


Is this just strsplit(name, ':')?

No, because rather than splitting on ':' it is returning the hierarchy of parent sections. So, e.g., getParents("species:d:collision_frequencies:d_d_coll") would return {"species", "species:d", "species:d:collision_frequencies"}.

Ah, I see! I'll add that example to the docstring, that's very clear

ZedThree · 2026-01-07T11:56:04Z

+    // FIXME: this isn't an empty set
+    //result.insert({name, {name}});
+    result.insert({name, {}});


I'm a bit confused by the comment -- is it outdated?

Yes, should have deleted that and the commented line. I must have left that during some debugging.

ZedThree · 2026-01-07T16:12:59Z

+/// In pratice, `item` here represents the index of a particular
+/// component. The indices of the dependencies of `item` are stored in
+/// the corresponding element of `dependencies`.
+void topological_sort(const std::vector<std::set<size_t>>& dependencies, size_t item,


All these local functions can go in an anonymous namespace

What is the advantage of putting them in an anonymous namespace?

It's a guard against One Definition Rule (ODR) violations -- basically ensures that functions have names unique to the translation unit, and so if there happens to be another function with the same name somewhere else, they won't clash at link time.

cmacmackin · 2026-01-07T18:12:59Z

+
+All component constructors must pass a `Permissions` object to the
+constructor on the `Component::Component` base class. This specifies
+which variables will be read/written by the `Component::transform`
+method and will be used to construct a `GuardedOptions` object to be
+passed into `Component::transform_impl`. The `Permissions` object
+(`Component::state_variable_access`) can be further updated in the
+body of the constructor of your component, based on what
+configurations were specified in ``alloptions``. You should give read
+and write permissions to the minimum number of variables necessary, to
+avoid circular dependencies arising among components.


This paragraph appears to be a duplicate which arose due to a mistake during a rebase (almost certainly my fault!). It should be deleted.

Explicitly evaluate BinaryExpr expressions into fields when needed.

Note that components still aren't setting up any permissions, so there would be runtime errors when executing the transform methods. Furthermore, there are some missing methods I realise I need for GuardedOptions. The unit tests are also failing to compile, although I don't understand what the problem is for them, yet.

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

cmacmackin changed the title ~~WIP: Perform topologicalk sort to order components~~ WIP: Perform topological sort to order components Nov 27, 2025

cmacmackin marked this pull request as draft November 27, 2025 18:45

cmacmackin force-pushed the cmacmackin/topological_sort_components branch from a75b2ce to 8d049e2 Compare December 2, 2025 15:57

cmacmackin marked this pull request as ready for review December 5, 2025 15:07

cmacmackin changed the title ~~WIP: Perform topological sort to order components~~ Perform topological sort to order components Dec 5, 2025

This was referenced Dec 8, 2025

Control access to state variables in transform method #421

Merged

Remove use of const_cast with Options #443

Open

cmacmackin mentioned this pull request Dec 18, 2025

Allow components to call for creation of sub components #445

Open

cmacmackin added a commit that referenced this pull request Jan 2, 2026

Responded to some comments by @ZedThree

e650c1c

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

cmacmackin force-pushed the cmacmackin/topological_sort_components branch from c31490e to e650c1c Compare January 2, 2026 14:28

cmacmackin added a commit that referenced this pull request Jan 5, 2026

Responded to some comments by @ZedThree

fff0581

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

cmacmackin force-pushed the cmacmackin/topological_sort_components branch from e650c1c to 9dbeed5 Compare January 5, 2026 17:42

cmacmackin added a commit that referenced this pull request Jan 6, 2026

Responded to some comments by @ZedThree

b1dfb25

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

cmacmackin force-pushed the cmacmackin/topological_sort_components branch from 9dbeed5 to 04a8977 Compare January 6, 2026 16:00

ZedThree pushed a commit that referenced this pull request Jan 7, 2026

Responded to some comments by @ZedThree

975a912

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

ZedThree force-pushed the cmacmackin/topological_sort_components branch from 04a8977 to 8fc17e0 Compare January 7, 2026 10:46

ZedThree approved these changes Jan 7, 2026

View reviewed changes

cmacmackin commented Jan 7, 2026

View reviewed changes

This was referenced Jan 14, 2026

Suppress permission warnings #468

Merged

Components not covered by integrated tests #475

Open

bendudson and others added 4 commits June 22, 2026 13:19

Evaluate field expressions

ed3726a

Explicitly evaluate BinaryExpr expressions into fields when needed.

Added unit test for topological sorting of components

fa564f1

Add ability to order components automatically

470cf87

cmacmackin added 8 commits June 23, 2026 17:07

Ensure ordering of components accounts for pre-set state variables

2e65a4c

Added support for sorting components based on section permissions

2abbea9

Mention new sorting and remove references to component ordering

6d2bf35

Check results are the same regardless of component order

a689d4e

Check only one component can make final write

7a5ebe4

Fixed some incorrect access controls causing errors

3c64a45

Add better comments explaining component sorting process

2d9eb11

Responded to some comments by @ZedThree

73593b7

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

cmacmackin force-pushed the cmacmackin/topological_sort_components branch from 8fc17e0 to 73593b7 Compare June 25, 2026 14:44

Applied linter suggestions

ae0b58e

cmacmackin mentioned this pull request Jun 25, 2026

Recent changes change component order requirements for braginskii_conduction and sheath_boundary_simple #588

Open

Uh oh!

Conversation

cmacmackin commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmacmackin commented Nov 27, 2025 • edited by ZedThree Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mikekryjak commented Jan 7, 2026

Uh oh!

cmacmackin commented Jan 7, 2026

Uh oh!

mikekryjak commented Jan 7, 2026

Uh oh!

cmacmackin commented Jan 7, 2026

Uh oh!

ZedThree left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cmacmackin commented Nov 27, 2025 •

edited

Loading

cmacmackin commented Nov 27, 2025 •

edited by ZedThree

Loading

codecov Bot commented Nov 27, 2025 •

edited

Loading