[Question] PD Torque Behavior in Isaac Lab and Policy Actions Are Too High and Don't Match Actual Joint Pos

[ptk-nuos]

Question

Hi everyone, I’m developing a reinforcement-learning controller for a bipedal leg in Isaac Lab (training with PPO using RSL-RL), and I’m running into an issue with how the PD actuator behaves in simulation and dont know why Policy Actions Are Too High and Don’t Match Actual Joint Positions.

I’m using DCMotor config for my actuators. Here’s a simplified snippet of my setup:

actuators={
    "hip_knee": DCMotorCfg(
        joint_names_expr=[".*_10_9_.*"],
        stiffness={".*": 10.0},
        damping={".*": 1.0},
        effort_limit=36.0,
        saturation_effort=53.0 * 2
        velocity_limit=235.0*2*3.14159/60,
        ...
    ),
    "hip_thigh": DCMotorCfg(
        joint_names_expr=[".*_70_9_.*", ".*_70_10_.*"],
        stiffness={".*": 4.0},
        damping={".*": 0.4},
        effort_limit=8.3 * 2,
        saturation_effort=24.8 * 2,
        velocity_limit =310.0 * 2 * 3.14159 / 60,
        ...
    ),
    "ankles": DCMotorCfg(
        joint_names_expr=[".*_60_6_.*"],
        stiffness={".*": 8.0},
        damping={".*": 0.8},
        effort_limit= 6.0 * 1.4 * 2,
        saturation_effort=18.0 * 1.4 * 2,
        velocity_limit = 490.0 * 2 * 3.14159 / 60 ,
        ...
    ),
}

After training, when I played the policy and plotted

the policy action (target position in radians)
the actual joint position in the environment
the torque output by each joint in the environment,

the behavior is as follow.

I observe that:

torque ≈ Kp * target_position(policy)

instead of the expected PD relationship:

τ = Kp * (q_des – q) + Kd * (qdot_des – qdot) + feedforward torque

The torque graph nearly matches the policy’s target-position graph, even though the actual joint position is different, causing the policy actions to be much higher than expected.

My question:

Is this behavior expected when using DCMotorCfg with the PD torque model in Isaac Lab? Or does this indicate a misconfiguration in the actuator setup or RL training?

Build Info

• Isaac Lab Version: [2.0.2]
• Isaac Sim Version: [4.5.0-rc.36+release.19112.f59b3005.gl]

[RandomOakForest]

Thank you for posting this. This behavior is expected with DCMotorCfg and usually does not indicate a bug, but it does expose two important configuration/details: (1) how the PD model is actually implemented in Isaac Lab and (2) what exactly your policy is outputting and how you log it.¹²³ I'll move this post to our Discussions section for follow up.

What the DC motor PD actually does

The DCMotor actuator in Isaac Lab inherits the ideal PD actuator model and then applies a DC-motor torque–speed saturation curve on top. Internally, the torque is computed as:²¹

$$ \tau = k_p (q_{des} - q) + k_d (\dot{q}_{des} - \dot{q}) + \tau_{ff} $$

and then this torque is clamped by a motor model based on saturation_effort, gear ratio, and the velocity-dependent torque–speed curve.³²

Key implications:

• The PD law is always in terms of errors (desired minus actual), not directly equal to the desired position.[^1_4]²
• After computing $\tau$, the DCMotor implementation clamps it with:
  • effort_limit (your effort_limit field), and
  • a velocity-dependent limit defined via saturation_effort (max motor torque).[^1_5]²
• If the position error and velocity error are small, the PD term is small and the final torque is small, even if the command is large; conversely, if the errors stay large, the torque will be approximately proportional to the command.

Why your torque looks proportional to the action

From your plots, you effectively see:

$$ \tau \approx k_p \cdot \text{action} $$

even though the actual joint position does not match the commanded position. This can happen in Isaac Lab with DCMotorCfg for one of these common reasons:

1. “Action” is already treated as $q_{des}$ in joint space, and you are logging the same tensor as “policy action” and “desired position”.
   • In many Isaac Lab + RSL-RL examples (e.g., Spot locomotion), the policy outputs joint position deltas or targets, which are then directly assigned to control_action.joint_positions for the actuator.⁴⁵
   • The PD block then uses exactly those values as $q_{des}$, so if $q$ is far from $q_{des}$, the error is essentially proportional to the command, and torque looks like a scaled version of the command.
2. Gains are small relative to inertia and contact forces, so the joint never gets close to $q_{des}$ during the fast-changing RL commands.
   • With stiffness values like 4–10 and low damping (0.4–1.0) on a loaded leg, the joint motion can be underactuated relative to the command bandwidth from PPO.¹³
   • The policy may keep pushing larger targets; the motor torque saturates around $k_p(q_{des}-q)$, so visually torque mirrors the target.
3. Velocity and feedforward commands are effectively zero.
   • In many RL tasks, you only set joint_positions and do not provide joint_velocities or joint_efforts (feedforward), so the implementation uses:

$$ \tau \approx k_p(q_{des}-q) $$

making it look “pure proportional”.²[^1_4]

4. You may be plotting pre-saturation torques or using high saturation_effort.
   • With large saturation_effort and effort_limit, the torque-speed saturation rarely clips, so the final torque is very close to the raw PD torque, which is again $k_p$ times the error and, in your regime, roughly $k_p$ times the command.[^1_5]²

Checks to confirm configuration vs. misconfiguration

These are concrete things to check to see if your setup is behaving as designed:

• Verify what tensor is passed into the actuator:
  • Inspect your task’s compute_actions() or RSL-RL wrapper and see exactly how the policy output is mapped to ArticulationActions:
    • Is it interpreted as absolute joint position $q_{des}$?
    • Is it scaled (e.g., actions * action_scale + default_joint_pos)?⁵⁶
  • Log control_action.joint_positions, joint_pos, joint_vel, and control_action.joint_velocities / joint_efforts to see the actual PD inputs.
• Confirm the actuator model is DCMotor and not an implicit PhysX drive:
  • Ensure your joint drive gains in the articulation are set to zero or left default if you rely solely on explicit actuators; otherwise you may accidentally combine PhysX joint drives with the Isaac Lab PD/motor, confusing your interpretation of torques.⁷³
• Check torque scale manually on a frozen pose:
  • Freeze the environment or use a single environment, set q constant, vary the policy action in a sine or step pattern, and log:
    • command (policy action),
    • q_des,
    • q,
    • tau.
  • You should see:
    • For small q - q_des errors, $\tau \approx k_p(q_{des}-q)$.
    • Once the error or speed is large enough, the torque saturates according to your saturation_effort and velocity limit.³²

How to reduce “too high” actions and mismatched tracking

If your goal is better tracking and smaller policy amplitudes rather than strict DC-motor realism:

• Increase stiffness and damping within realistic ranges so that the joints track q_des more closely at the control frequency.¹³
• Reduce action range / apply action scaling so the policy outputs are closer to feasible joint motion and do not lean on saturation. Many Isaac Lab locomotion examples use a moderate action scale (e.g., fraction of joint range around a nominal pose).⁴⁵
• Consider IdealPDActuatorCfg instead of DCMotorCfg if you do not need a torque–speed curve and want a more “pure” PD behavior without additional saturation complexity.²¹
• Optionally expose desired velocity or torque feedforward in the action space if you want a true PD+FF structure that the policy can exploit, though this complicates training.

Summary

So, yes: seeing torque closely follow $k_p \times$ (policy command) with DCMotorCfg is consistent with Isaac Lab’s PD-plus-motor-saturation model when the policy output is interpreted as $q_{des}$ and velocity/FF commands are zero, especially with relatively low stiffness and large saturation limits. It usually reflects the current gains and action mapping rather than a fundamental misconfiguration or bug in the actuator implementation.³¹²

Footnotes

1. https://isaac-sim.github.io/IsaacLab/main/source/api/lab/isaaclab.actuators.html ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

2. https://isaac-sim.github.io/IsaacLab/main/source/api/lab/isaaclab.actuators.html#isaaclab.actuators.IdealPDActuator ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰

3. https://isaac-sim.github.io/IsaacLab/main/source/overview/core-concepts/actuators.html ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

4. https://developer.nvidia.com/blog/closing-the-sim-to-real-gap-training-spot-quadruped-locomotion-with-nvidia-isaac-lab/ ↩ ↩²

5. https://docs.nvidia.com/learning/physical-ai/getting-started-with-isaac-lab/latest/train-your-second-robot-with-isaac-lab/02-robot-configuration-in-isaac-lab.html ↩ ↩² ↩³

6. https://isaac-sim.github.io/IsaacLab/main/source/api/lab_rl/isaaclab_rl.html ↩

7. https://github.com/isaac-sim/IsaacLab/issues/2369 ↩