Skip to content

Make toksearch the only data source - #90

Open
AreWeDreaming wants to merge 37 commits into
mainfrom
ssd/fetch_with_toksearch
Open

AreWeDreaming wants to merge 37 commits into
mainfrom
ssd/fetch_with_toksearch

Conversation

@AreWeDreaming

Copy link
Copy Markdown
Collaborator

This PR make imas_composer only allow fetching via toksearch. It also adds the toksearch_d3d and omas submodule temporarily to handle the dependency on the two work in progress repos.
This PR comes with CI that needs a BEARER_TOKEN secret which is then used to load the DIII-D data in both imas_composer and omas via fdp.

- Also lower standard precision because the default was ridiculus before.
@AreWeDreaming

Copy link
Copy Markdown
Collaborator Author

I see that we are already inheriting a BEARER_TOKEN from the org so these tests actually do something.

@AreWeDreaming

Copy link
Copy Markdown
Collaborator Author

The charge_exchange test fails because I cannot run the required TCL mdsplus function. It fails with:

Failed to resolve VERTICAL_32_NZ__data's true location from  Error: unable to parse ./mdsdcl_commands.xml

The other tests should do better with the commit I'll push in a moment.

@AreWeDreaming

Copy link
Copy Markdown
Collaborator Author

This is now in principle in reasonable shape to be merged but I would rather have all the dependencies:

Merged down and published first because I do not want all these submodules here for long. Now we could in principle keep the omas submodule around because it serves there general pattern of adding mappings to both imas_composer and omas at the same time.

AreWeDreaming and others added 3 commits August 26, 2026 10:03
- Remove mappings in core_profiles
DIII-D MDSplus stores AEQDSK.SEPLIM, GAPIN, GAPOUT, GAPTOP, and GAPBOT
in metres with units explicitly labelled in the tree. Remove the erroneous
/100 from _compose_closest_wall_distance and _compose_gap_values. Drop the
stale tolerance overrides in test_config for these fields.

Follow-up to #113 (same issue confirmed for RSURF/ZSURF/AMINOR).
Depends on gafusion/omas#443 before omas_validation tests will pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
torrinba and others added 14 commits October 8, 2026 20:01
I've confirmed that these are all stored in MDSplus in meters.

In addition, the correction of the filters applied in OMAS exposed some
weaknesses of the filters and test comparison in imas_composer. In
particular, it is possible for the x-points and strike points to have
missing interior values so simply using awkward arrays is not
sufficient. The recommended solution appears to be using Nones for
missing interior entries in the awkward arrays. This has been
implemented and fixes the comparisons with OMAS.

Also cleaning up some junk claude comments.
TRANSPORT.GLOBAL.TIMES.TAUE occasionally contains unphysical outliers
(e.g., 154 s at t≈2.32 s near the L→H transition) that pass the
existing tau_e > 0 guard but inflate rms(ref) by ~40x, making NRMS
values artifactually close to 1. Adding the < 5.0 s upper bound
removes these artifacts while retaining all physically meaningful
confinement times for DIII-D H-mode shots (typical range 0.05–0.4 s).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Follows gafusion/omas#440, which made the ECE IDS homogeneous with
ece.time taken from the TECE01 time base.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er merged in

Brings in the AEQDSK length-scale unit fixes and the He beam fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follow-up to 38b8139 mirroring gafusion/omas a006adc9 and 8a088272:
SEPLIM, GAPIN/GAPOUT/GAPTOP/GAPBOT and the strike points RVS*/ZVS* are
stored in metres and AREA in m^2, so drop the /100 and /10000 scalings.
The strike point sentinel is -0.89 on the stored value. Drop the
tolerance overrides that absorbed the 32-bit rounding of the conversion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…arch

Supersedes ssd/fix_remaining_efit_length_units (#124): equilibrium.py
takes #114's version throughout; conftest.py keeps the stack's default
compare_values tolerances.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and the fetch error fix

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants