Gear assembly sim to real #4044

ashwinvkNV · 2025-11-19T19:23:11Z

rl-video-step-137600.mp4

Description

This PR introduces a new Gear Assembly manipulation task for sim-to-real training with the UR10e robot arm. This environment enables training policies for precise gear insertion tasks using reinforcement learning, with comprehensive sim-to-real transfer capabilities.

Summary of Changes

New Features

Gear Assembly Environment: Complete environment implementation for gear insertion tasks
- Environment configuration (gear_assembly_env_cfg.py)
- UR10e-specific joint position control configuration (joint_pos_env_cfg.py)
- RSL-RL PPO training configuration (rsl_rl_ppo_cfg.py)
MDP Components: Task-specific observation, reward, termination, and event functions
- mdp/events.py: Randomization and reset events for robust training
- mdp/observations.py: State observation functions
- mdp/rewards.py: Reward shaping for gear insertion
- mdp/terminations.py: Episode termination conditions
Noise Models: Enhanced noise simulation for domain randomization
- Added configurable noise models (noise_model.py, noise_cfg.py)
- Integration with observation and action spaces for realistic sim-to-real transfer

Documentation

Sim-to-Real Training Walkthrough: Complete guide for training and deploying the gear assembly task
- Step-by-step training instructions
- Real robot deployment guidelines
- Visual assets (GIFs and screenshots)

Core Enhancements

Training Script: Enhanced train.py with additional logging and configuration options
UR10e Robot Configuration: Updated universal_robots.py with gear assembly specific parameters
Reward System: Extended core reward functions in isaaclab/envs/mdp/rewards.py
RL Configuration: Updated RSL-RL integration (rl_cfg.py, setup.py)

Type of change

New feature (non-breaking change which adds functionality)
Documentation update

Checklist

I have read and understood the contribution guidelines
I have run the pre-commit checks with ./isaaclab.sh --format
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
I have updated the changelog and the corresponding version in the extension's config/extension.toml file
I have added my name to the CONTRIBUTORS.md or my name already exists there

Usage Example

# Train the gear assembly task
python scripts/reinforcement_learning/rsl_rl/train.py \
  --task Isaac-Deploy-GearAssembly-UR10e-2F140-ROS-Inference-v0 \
  --num_envs 256 \
  --headless

# Run inference with trained policy
python scripts/reinforcement_learning/rsl_rl/play.py \
  --task Isaac-Deploy-GearAssembly-UR10e-2F140-ROS-Inference-v0 \
  --num_envs 1 \
 --checkpoint <checkpoint_path>

greptile-apps · 2025-11-19T19:26:08Z

Greptile Summary

Introduces complete gear assembly sim-to-real environment for UR10e with PPO/LSTM training supporting 2F-140 and 2F-85 Robotiq grippers
Implements class-based MDP components with pre-cached tensors for efficient batch operations including dynamic gear type randomization, keypoint-based rewards, and IK-based grasp initialization
Adds ResetSampledNoiseModel for domain randomization that samples noise once per episode reset rather than every step

Confidence Score: 4/5

Safe to merge with minor style improvements recommended
Well-structured implementation with comprehensive reward shaping, termination conditions, and domain randomization. Code follows IsaacLab patterns with class-based terms and proper tensor caching. Minor redundant operations in IK loop and temporary USD path workaround noted but non-critical.
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/deploy/mdp/events.py has redundant joint state reads in IK loop; source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/deploy/gear_assembly/config/ur_10e/joint_pos_env_cfg.py uses temporary USD path pending bug fix

Important Files Changed

Filename	Overview
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/deploy/mdp/events.py	Implements gear type randomization and IK-based robot grasp pose initialization with pre-cached tensors for efficient batch operations; IK loop reads joint state redundantly on each iteration (line 232)
source/isaaclab/isaaclab/utils/noise/noise_model.py	Adds ResetSampledNoiseModel class that samples noise only during reset and applies it consistently throughout the episode
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/deploy/gear_assembly/config/ur_10e/joint_pos_env_cfg.py	UR10e-specific configuration with 2F-140 and 2F-85 gripper support, gripper-specific joint setters, and IK-based grasp pose initialization; uses temporary USD path override (line 415)

Sequence Diagram

sequenceDiagram
    participant User
    participant TrainingScript
    participant Environment
    participant GearTypeManager
    participant RobotIK
    participant PPOAgent
    participant RewardManager

    User->>TrainingScript: run train.py with task config
    TrainingScript->>Environment: create env with UR10e gear assembly config
    Environment->>GearTypeManager: initialize RandomizeGearType event
    GearTypeManager->>Environment: register as _gear_type_manager
    Environment->>Environment: setup scene with robot and 3 gear types
    
    loop Training Episodes
        Environment->>GearTypeManager: reset - randomize gear type
        GearTypeManager->>Environment: set active gear per env
        Environment->>RobotIK: SetRobotToGraspPose event
        RobotIK->>RobotIK: run IK to compute grasp pose
        RobotIK->>Environment: update robot joint positions
        Environment->>Environment: RandomizeGearsAndBasePose event
        
        loop Episode Steps
            Environment->>PPOAgent: get observation (joint pos/vel, gear shaft pose)
            PPOAgent->>Environment: return action (delta joint positions)
            Environment->>Environment: apply action and step simulation
            Environment->>RewardManager: compute keypoint distance rewards
            RewardManager->>Environment: return reward signal
            Environment->>Environment: check terminations (gear dropped, orientation)
        end
        
        Environment->>PPOAgent: collect episode data
        PPOAgent->>PPOAgent: update policy with PPO
    end
    
    TrainingScript->>User: save trained model checkpoint

greptile-apps

_{22 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}
_{React with 👍 or 👎 to share your feedback on this new summary format}

source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/deploy/mdp/events.py

greptile-apps · 2025-11-19T19:26:08Z

...lab_tasks/manager_based/manipulation/deploy/gear_assembly/config/ur_10e/joint_pos_env_cfg.py

+                # TODO: @ashwinvk: Revert to default USD after https://jirasw.nvidia.com/browse/ISIM-4733 is resolved
+                usd_path="omniverse://isaac-dev.ov.nvidia.com/Projects/isaac_ros_gear_insertion/ur10e_default_2f85.usd",


style: temporary USD path override pending ISIM-4733 resolution - ensure this is removed after the issue is fixed

source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/deploy/mdp/rewards.py

...tasks/manager_based/manipulation/deploy/gear_assembly/config/ur_10e/agents/rsl_rl_ppo_cfg.py

source/isaaclab/isaaclab/envs/mdp/rewards.py

ooctipus · 2025-11-20T22:16:31Z

source/isaaclab_rl/isaaclab_rl/rsl_rl/rl_cfg.py

    noise_std_type: Literal["scalar", "log"] = "scalar"
    """The type of noise standard deviation for the policy. Default is scalar."""

+    state_dependent_std: bool = False


this might be a different PR.

This param in used in the rsl rl config for the sim to real env. Are you saying that it should be seperated into a new PR?

yes a different PR to introduce this argument in rl_cfg.py

...tasks/manager_based/manipulation/deploy/gear_assembly/config/ur_10e/agents/rsl_rl_ppo_cfg.py

ooctipus · 2025-11-20T22:20:27Z

...asks/isaaclab_tasks/manager_based/manipulation/deploy/gear_assembly/gear_assembly_env_cfg.py

+        prim_path="{ENV_REGEX_NS}/FactoryGearBase",
+        # TODO: change to common isaac sim directory
+        spawn=sim_utils.UsdFileCfg(
+            usd_path="omniverse://isaac-dev.ov.nvidia.com/Isaac/Props/Factory/gear_assets/factory_gear_base/factory_gear_base.usd",


please use import NUCLEUS Directory import instead write raw url

Waiting on the sim team to upload these assets to the AWS server and I will replace it.

...asks/isaaclab_tasks/manager_based/manipulation/deploy/gear_assembly/gear_assembly_env_cfg.py

ooctipus · 2025-11-20T22:28:49Z

docs/source/_static/setup/walkthrough_sim_real_gear_assembly_train.png

please capture a high quality image with robot base. : )))

done. Does this one work?

kellyguo11

could we try to avoid having large .gif files in the repo directly? we can upload them to the server if needed and referenced from docs.

docs/source/setup/walkthrough/index.rst

kellyguo11 · 2025-11-24T17:25:07Z

...asks/isaaclab_tasks/manager_based/manipulation/deploy/gear_assembly/gear_assembly_env_cfg.py

+        prim_path="{ENV_REGEX_NS}/FactoryGearSmall",
+        # TODO: change to common isaac sim directory
+        spawn=sim_utils.UsdFileCfg(
+            usd_path="omniverse://isaac-dev.ov.nvidia.com/Isaac/Props/Factory/gear_assets/factory_gear_small/factory_gear_small.usd",


we shouldn't use this path directly in code being merged to main, users will not have access to these. are the assets available on the S3 bucket?

yup, waiting on the sim team to uplaod assets and I will update before this PR is merged

ashwinvkNV · 2025-11-25T23:25:00Z

...lab_tasks/manager_based/manipulation/deploy/gear_assembly/config/ur_10e/joint_pos_env_cfg.py

+        mode="reset",
+        params={
+            "gear_types": ["gear_small", "gear_medium", "gear_large"]
+            # "gear_types": ["gear_small"]


will remove

ashwinvkNV · 2025-11-25T23:25:22Z

...lab_tasks/manager_based/manipulation/deploy/gear_assembly/config/ur_10e/joint_pos_env_cfg.py

+        mode="startup",
+        params={
+            "asset_cfg": SceneEntityCfg("robot", body_names=".*finger"),
+            "static_friction_range": (1000.0, 1000.0),


will test and update to 1.5 instead of 1000.0

ooctipus · 2025-11-25T23:33:48Z

Thanks for the edit and contribution : )),

I'd like to ask a high level questions why not put this PR in deploy folder we created for the reach eariler? Are we planing to add peg insert and nut thread as well? if that's the intention, it might be more benefitial to work a general structure with all of them, right now mdp seems just tailored to gearmesh.

ooctipus · 2025-11-25T23:36:58Z

docs/source/_static/policy_deployment/02_gear_assembly/sim_real_gear_assembly_train.png

having the asset in the air is a bit odd, can we add a desk to it?

The base is a fixed asset whose pose is randomzied in the all 6 degrees of freedom. I think adding a table would cause collision issues or visual penetration. And it might make the user think we're actually simulating the base->table physics interactions.

Do you think I should I should add a note in the docs about this?

Initial commit for gear assembly sim to real

2fa2cc0

ashwinvkNV requested review from ClemensSchwarke, Mayankm96, jtigue-bdai, kellyguo11, ooctipus and pascal-roth as code owners November 19, 2025 19:23

github-actions bot added documentation Improvements or additions to documentation asset New asset feature or request labels Nov 19, 2025

greptile-apps bot reviewed Nov 19, 2025

View reviewed changes

remove redundant joint_pos and joint_vel get

a01d7f6

iakinola23 reviewed Nov 20, 2025

View reviewed changes