New observation

[montali]

Some key concepts:

• Knowing where the station is, maybe a one-hot encoded array with compass-like directions -> can use this to actually avoid deadlocks when they are going in opposite directions
• Heatmap to avoid others and go to stations: other trains are super hot, the destination is super cold. The temperature propagates in space and time: when our agent travels a path, it stays mildly hot for a while
• Communication between trains
• Deadlocks avoiding using a predictor
• Newton's law: trains have a weight proportional to their distance to the station, and repell other trains when they're near; when a train is malfunctioning we'll have to avoid it
• Normalized distance (like Manhattan) from the station
• Punish deadlocks using rewards
• When an agent gets to its destination, remove all its heat
• Consider the flow in how the temperature propagates: no backward heatening

[montali]

Epsilon might then be multiplied by the temperature of the train's room, so that we explore more when we're far from the station