`KeyError(:MonitoredLine)` or `KeyError(:Line)` during `build!` due to `Symbol("PowerSystems.MonitoredLine")` and `Symbol("PowerSystems.Line")`

[yasirroni]

Describe the bug

Warning

This bug is inconsistent. Every fresh Julia session on my machine always trigger this bug. Re-running without restarting Julia sometimes didn't trigger the bug.

ProblemTemplate.branches can contain branch model keys using namespaced symbols such as:

Symbol("PowerSystems.MonitoredLine")
Symbol("PowerSystems.Line")

instead of the plain symbols expected later by PowerSimulations:

:MonitoredLine
:Line

This causes DecisionModel build failure for systems containing MonitoredLine components, because PowerSimulations later looks up:

branch_models[:MonitoredLine]

but the dictionary contains:

branch_models[Symbol("PowerSystems.MonitoredLine")]

The resulting error is:

KeyError: key :MonitoredLine not found

This appears to happen when using the 3-argument form of set_device_model! for branch models:

set_device_model!(template, MonitoredLine, StaticBranchBounds)
set_device_model!(template, Line, StaticBranch)

Warning

One additional complication is that this behavior appears inconsistent. In some Julia sessions or call paths, the same build_problem_base_uc() function produced the expected keys, such as :MonitoredLine and :Line. In other runs, especially after restart/reload, it produced Symbol("PowerSystems.MonitoredLine") and Symbol("PowerSystems.Line"). This made the issue harder to diagnose because the template display looked correct in both cases, while the underlying dictionary keys differed.

The display table still prints MonitoredLine and Line, so the template looks correct visually, but the internal dictionary keys are not compatible with the later lookup path.

To Reproduce

This is the relevant UC template builder:

using HydroPowerSimulations
using StorageSystemsSimulations
using PowerModels
using PowerSystems
using PowerSimulations

function build_problem_base_uc(; network_model = NFAPowerModel)
    template_uc = ProblemTemplate()

    set_device_model!(template_uc, MonitoredLine, StaticBranchBounds)
    set_device_model!(template_uc, Line, StaticBranch)

    set_device_model!(template_uc, ThermalStandard, ThermalStandardUnitCommitment)
    set_device_model!(template_uc, RenewableDispatch, RenewableFullDispatch)
    set_device_model!(template_uc, RenewableNonDispatch, FixedOutput)
    set_device_model!(template_uc, PowerLoad, StaticPowerLoad)

    storage_model = DeviceModel(
        EnergyReservoirStorage,
        StorageDispatchWithReserves;
        attributes = Dict(
            "reservation" => true,
            "energy_target" => false,
            "cycling_limits" => false,
            "regularization" => false,
        ),
        use_slacks = false,
    )

    set_device_model!(template_uc, storage_model)

    set_network_model!(
        template_uc,
        NetworkModel(network_model; use_slacks = true),
    )

    return template_uc
end

Then inspect the branch keys:

template_uc = build_problem_base_uc()

println("template.branches keys:")
for k in keys(template_uc.branches)
    println("  ", repr(k))
end

println("has :MonitoredLine = ", haskey(template_uc.branches, :MonitoredLine))
println(
    "has Symbol(\"PowerSystems.MonitoredLine\") = ",
    haskey(template_uc.branches, Symbol("PowerSystems.MonitoredLine")),
)

Observed output:

template.branches keys:
  Symbol("PowerSystems.Line")
  Symbol("PowerSystems.MonitoredLine")

has :MonitoredLine = false
has Symbol("PowerSystems.MonitoredLine") = true

For a system containing MonitoredLine components, the model then fails during build:

decision_model = DecisionModel(
    template_uc,
    sys;
    horizon = Hour(48),
    initial_time = DateTime("2025-01-08T00:00:00"),
    optimizer = optimizer,
    name = "UC",
)

build!(decision_model; output_dir = mktempdir())

Error:

DecisionModel Build Failed

KeyError: key :MonitoredLine not found

Stacktrace:
 [1] getindex(h::Dict{Symbol, DeviceModel{<:PowerSystems.Branch}}, key::Symbol)
 [2] _any_branch_model_configures_param(
        network_model::NetworkModel{NFAPowerModel},
        branch_models::Dict{Symbol, DeviceModel{<:PowerSystems.Branch}},
        param_type::Type{BranchRatingTimeSeriesParameter},
     )
 [3] _add_timeseries_irreducible_buses!(...)
 [4] _get_irreducible_buses_due_to_monitored_components(...)
 [5] instantiate_network_model!(...)
 [6] build_impl!(model::DecisionModel{GenericOpProblem})

A workaround is to normalize the branch keys before returning the template:

function _device_type_from_model(::DeviceModel{D}) where {D}
    return D
end

function normalize_template_branch_keys!(template::ProblemTemplate)
    bad_keys = [
        k for k in keys(template.branches)
        if occursin(".", String(k))
    ]

    if isempty(bad_keys)
        return template
    end

    @warn "Detected namespaced branch model keys in template.branches; normalizing before returning template" bad_keys

    fixed = Dict{Symbol, DeviceModel{<:Branch}}()

    for (old_key, model) in template.branches
        device_type = _device_type_from_model(model)
        new_key = nameof(device_type)
        fixed[new_key] = model
    end

    empty!(template.branches)

    for (k, v) in fixed
        template.branches[k] = v
    end

    @warn "Normalized template.branches keys" normalized_keys = collect(keys(template.branches))

    return template
end

Calling this before returning the template fixes the issue:

normalize_template_branch_keys!(template_uc)
return template_uc

After normalization:

template.branches keys:
  :MonitoredLine
  :Line

has :MonitoredLine = true
has Symbol("PowerSystems.MonitoredLine") = false

The model then proceeds past the previous KeyError.

Expected behavior

set_device_model! should store branch models under stable, non-namespaced keys matching the internal lookup convention used later by PowerSimulations:

:MonitoredLine
:Line

or the later lookup path should use the same key convention as the setter.

The following should not result in keys like Symbol("PowerSystems.MonitoredLine"):

set_device_model!(template, MonitoredLine, StaticBranchBounds)
set_device_model!(template, Line, StaticBranch)

Screenshots

Not applicable. The relevant debug output is:

template.branches keys:
  Symbol("PowerSystems.Line")
  Symbol("PowerSystems.MonitoredLine")

has :MonitoredLine = false
has Symbol("PowerSystems.MonitoredLine") = true

and the build failure is:

KeyError: key :MonitoredLine not found

Additional context

Environment where this was observed:

PowerSimulations v0.35.0
PowerSystems v5.10.0
InfrastructureSystems v3.6.0
HydroPowerSimulations v0.16.0
StorageSystemsSimulations v0.17.0
PowerModels v0.21.5

The system being solved contains MonitoredLine components and no Line components:

Line count          = 0
MonitoredLine count = 17

The issue is easy to miss because the printed ProblemTemplate table displays:

Branch Models
MonitoredLine | StaticBranchBounds
Line          | StaticBranch

even when the underlying dictionary keys are namespaced and incompatible with the later internal lookup.

A robust internal fix may be to derive branch dictionary keys from the device type using:

nameof(device_type)

rather than any string representation that may include the module prefix.

Another possible fix would be to make the DecisionModel build/solve path tolerant of both key conventions. For example, when looking up a branch model, PowerSimulations could accept either :MonitoredLine or Symbol("PowerSystems.MonitoredLine") for the same device type. This would make the downstream build path robust even if a template contains namespaced symbols. However, normalizing the keys at template construction time may still be cleaner, because it keeps one canonical internal representation.

[jd-lara]

Which version are you using? I fixed this in 0.35 by changing Symbol to nameof

[yasirroni]

Hi, @jd-lara

[[deps.PowerSimulations]]
deps = ["CSV", "DataFrames", "DataFramesMeta", "DataStructures", "Dates", "Distributed", "Distributions", "DocStringExtensions", "HDF5", "InfrastructureSystems", "InteractiveUtils", "JSON3", "JuMP", "LinearAlgebra", "Logging", "MathOptInterface", "PowerFlows", "PowerModels", "PowerNetworkMatrices", "PowerSystems", "PrettyTables", "ProgressMeter", "Random", "Serialization", "SparseArrays", "TimeSeries", "TimerOutputs"]
git-tree-sha1 = "f2c190c03b866880a0e6fc3e5136477add7721a8"
uuid = "e690365d-45e2-57bb-ac84-44ba829e73c4"
version = "0.35.0"

According to my manifest, and this bug appears after I upgrade Sienna anyway.

[jd-lara]

This will be fixed when we merge #1619