[Newton] Graphed Workflows

[AntoineRichard]

Description

Adds graphed workflows as a separate pipeline for now.

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context..

Type of change

• New feature (non-breaking change which adds functionality)

How to use:

./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Cartpole-Warp-v0 --num_envs 4096 --headless --max_iterations 300
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Ant-Warp-v0 --num_envs 4096 --headless --max_iterations 300
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Humanoid-Warp-v0 --num_envs 4096 --headless --max_iterations 300

Results

Provides some speed-up on the Isaac front:
For Cartpole:

• Torch API:
  • full collection step: Mean: 3398.505688 us, Std: 462.092124 us, N: 4800
  • actuators: Mean: 458.664717 us, Std: 56.303926 us, N: 9600
  • newton step: Mean: 418.564702 us, Std: 27.599639 us, N: 9599
  • post-processing (rl-related calculations): Mean: 1727.790608 us, Std: 427.153509 us, N: 4800
  • Lab overhead: 2560us
• Warp API:
  • full collection step: Mean: 1097.855936 us, Std: 73.003888 us, N: 4800
  • actuators: Mean: 17.609461 us, Std: 4.145735 us, N: 9599
  • newton step: Mean: 370.281310 us, Std: 29.087660 us, N: 9599
  • post-processing (rl-related calculations): Mean: 24.169094 us, Std: 8.389473 us, N: 4800
  • Lab overhead: 357us

For Ant:

• Torch API:
  • full collection step: Mean: 7646.153628 us, Std: 1809.778802 us, N: 32000
  • actuators: 352.259001 us, Mean: 363.953334 us, Std: 29.461383 us, N: 64000
  • newton step: 2011.052002 us, Mean: 2121.656286 us, Std: 321.844311 us, N: 64000
  • post-processing (rl-related calculations): 2433.182002 us, Mean: 2435.709163 us, Std: 1679.963760 us, N: 32000
  • Lab overhead: 3404us
• Warp API:
  • full collection step: Mean: 4603.307170 us, Std: 257.345966 us, N: 32000
  • actuators: Mean: 18.158209 us, Std: 2.621708 us, N: 64000
  • newton step: Mean: 2066.688861 us, Std: 128.275069 us, N: 63999
  • post-processing (rl-related calculations): Mean: 59.394046 us, Std: 5.979035 us, N: 32000
  • Lab overhead: 471us

For Humanoid:

• Torch API:
  • full collection step: Mean: 17507.664525 us, Std: 2403.361261 us, N: 16000
  • actuators: Mean: 404.597538 us, Std: 137.743652 us, N: 31999
  • newton step: Mean: 6812.051563 us, Std: 331.997178 us, N: 31999
  • post-processing (rl-related calculations): Mean: 2780.663383 us, Std: 2256.250511 us, N: 16000
  • Lab overhead: 3883us
• Warp API:
  • full collection step: Mean: 14906.382912 us, Std: 668.738434 us, N: 16000
  • actuators: Mean: 23.225196 us, Std: 3.397187 us, N: 32000
  • newton step: Mean: 7172.681914 us, Std: 355.531817 us, N: 32000
  • post-processing (rl-related calculations): Mean: 80.205396 us, Std: 6.993390 us, N: 16000
  • Lab overhead: 562us

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there