Skip to content

[Feature] auto_wrap_envs in PEnv #3284

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vmoens merged 2 commits into from
Jan 1, 2026
Merged

[Feature] auto_wrap_envs in PEnv #3284

vmoens merged 2 commits into from
Jan 1, 2026

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Dec 31, 2025 •
edited
Loading

@vmoens
Update
c7acdda 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Dec 31, 2025
@vmoens
[Feature] auto_wrap_envs in PEnv
a84dbf9 
ghstack-source-id: 3a05b7b
Pull-Request: #3284
@pytorch-bot
Copy link

pytorch-bot bot commented Dec 31, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3284

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 4 New Failures, 1 Unrelated Failure

As of commit 469c599 with merge base 7866d11 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Dec 31, 2025
Merged
Merged
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 31, 2025
@vmoens
Update
469c599 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Dec 31, 2025
@vmoens
[Feature] auto_wrap_envs in PEnv
fb2a027 
ghstack-source-id: 8dceb5d
Pull-Request: #3284
@github-actions
Copy link

github-actions bot commented Dec 31, 2025

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 164. Improved: $\large\color{#35bf28}23$. Worsened: $\large\color{#d91a1a}10$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 83.7593μs 82.7418μs 12.0858 KOps/s 12.5180 KOps/s $\color{#d91a1a}-3.45\%$
test_tensor_to_bytestream_speed[torch.save] 0.1382ms 0.1379ms 7.2491 KOps/s 7.2143 KOps/s $\color{#35bf28}+0.48\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1189s 0.1186s 8.4314 Ops/s 8.6364 Ops/s $\color{#d91a1a}-2.37\%$
test_tensor_to_bytestream_speed[numpy] 2.7301μs 2.7176μs 367.9653 KOps/s 376.7175 KOps/s $\color{#d91a1a}-2.32\%$
test_tensor_to_bytestream_speed[safetensors] 37.2625μs 37.0385μs 26.9989 KOps/s 26.5488 KOps/s $\color{#35bf28}+1.70\%$
test_simple 0.5530s 0.5436s 1.8397 Ops/s 1.7541 Ops/s $\color{#35bf28}+4.88\%$
test_transformed 1.1149s 1.1110s 0.9001 Ops/s 0.8805 Ops/s $\color{#35bf28}+2.22\%$
test_serial 1.6420s 1.6407s 0.6095 Ops/s 0.5966 Ops/s $\color{#35bf28}+2.17\%$
test_parallel 1.2192s 1.1144s 0.8973 Ops/s 0.9021 Ops/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-True-True-True-True] 0.2530ms 44.7140μs 22.3643 KOps/s 22.5237 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[True-True-True-True-False] 94.5210μs 25.2360μs 39.6259 KOps/s 39.4909 KOps/s $\color{#35bf28}+0.34\%$
test_step_mdp_speed[True-True-True-False-True] 0.4566ms 25.0698μs 39.8886 KOps/s 39.5551 KOps/s $\color{#35bf28}+0.84\%$
test_step_mdp_speed[True-True-True-False-False] 0.4583ms 14.4671μs 69.1224 KOps/s 70.4383 KOps/s $\color{#d91a1a}-1.87\%$
test_step_mdp_speed[True-True-False-True-True] 84.0610μs 48.3399μs 20.6869 KOps/s 21.1721 KOps/s $\color{#d91a1a}-2.29\%$
test_step_mdp_speed[True-True-False-True-False] 0.4611ms 28.0330μs 35.6722 KOps/s 35.6340 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-True-False-False-True] 0.4659ms 28.3354μs 35.2916 KOps/s 35.9266 KOps/s $\color{#d91a1a}-1.77\%$
test_step_mdp_speed[True-True-False-False-False] 47.0510μs 16.5652μs 60.3675 KOps/s 60.3124 KOps/s $\color{#35bf28}+0.09\%$
test_step_mdp_speed[True-False-True-True-True] 79.8320μs 51.3872μs 19.4601 KOps/s 19.8202 KOps/s $\color{#d91a1a}-1.82\%$
test_step_mdp_speed[True-False-True-True-False] 0.4591ms 30.9934μs 32.2649 KOps/s 32.5072 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-False-True-False-True] 0.4859ms 27.6220μs 36.2030 KOps/s 36.4508 KOps/s $\color{#d91a1a}-0.68\%$
test_step_mdp_speed[True-False-True-False-False] 67.3810μs 16.6717μs 59.9819 KOps/s 59.9515 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[True-False-False-True-True] 0.4858ms 53.6267μs 18.6474 KOps/s 18.8879 KOps/s $\color{#d91a1a}-1.27\%$
test_step_mdp_speed[True-False-False-True-False] 0.4898ms 33.6128μs 29.7506 KOps/s 30.5466 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[True-False-False-False-True] 0.4753ms 31.0133μs 32.2443 KOps/s 32.7965 KOps/s $\color{#d91a1a}-1.68\%$
test_step_mdp_speed[True-False-False-False-False] 49.4500μs 19.5585μs 51.1287 KOps/s 50.4052 KOps/s $\color{#35bf28}+1.44\%$
test_step_mdp_speed[False-True-True-True-True] 0.4956ms 51.2110μs 19.5271 KOps/s 19.8718 KOps/s $\color{#d91a1a}-1.73\%$
test_step_mdp_speed[False-True-True-True-False] 66.6510μs 31.0579μs 32.1980 KOps/s 32.5225 KOps/s $\color{#d91a1a}-1.00\%$
test_step_mdp_speed[False-True-True-False-True] 2.2355ms 31.6891μs 31.5566 KOps/s 31.6109 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[False-True-True-False-False] 50.2200μs 18.2126μs 54.9070 KOps/s 53.6438 KOps/s $\color{#35bf28}+2.35\%$
test_step_mdp_speed[False-True-False-True-True] 0.4843ms 53.8843μs 18.5583 KOps/s 19.1999 KOps/s $\color{#d91a1a}-3.34\%$
test_step_mdp_speed[False-True-False-True-False] 0.4627ms 34.0741μs 29.3478 KOps/s 30.1861 KOps/s $\color{#d91a1a}-2.78\%$
test_step_mdp_speed[False-True-False-False-True] 0.4616ms 34.8751μs 28.6738 KOps/s 28.9050 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[False-True-False-False-False] 54.6810μs 21.0350μs 47.5398 KOps/s 46.9114 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[False-False-True-True-True] 0.4889ms 57.2738μs 17.4600 KOps/s 17.9932 KOps/s $\color{#d91a1a}-2.96\%$
test_step_mdp_speed[False-False-True-True-False] 0.5529ms 36.0387μs 27.7479 KOps/s 27.6756 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[False-False-True-False-True] 81.4020μs 34.6899μs 28.8268 KOps/s 28.5517 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[False-False-True-False-False] 67.3010μs 20.9317μs 47.7745 KOps/s 47.3926 KOps/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[False-False-False-True-True] 0.1055ms 59.0150μs 16.9448 KOps/s 17.1668 KOps/s $\color{#d91a1a}-1.29\%$
test_step_mdp_speed[False-False-False-True-False] 76.0420μs 39.2073μs 25.5054 KOps/s 25.9314 KOps/s $\color{#d91a1a}-1.64\%$
test_step_mdp_speed[False-False-False-False-True] 76.6720μs 37.5497μs 26.6314 KOps/s 27.3119 KOps/s $\color{#d91a1a}-2.49\%$
test_step_mdp_speed[False-False-False-False-False] 54.9110μs 24.0306μs 41.6136 KOps/s 42.6069 KOps/s $\color{#d91a1a}-2.33\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.8657s 0.7639s 1.3090 Ops/s 1.3024 Ops/s $\color{#35bf28}+0.50\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7266s 0.6310s 1.5848 Ops/s 1.5809 Ops/s $\color{#35bf28}+0.24\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7613s 1.6716s 0.5982 Ops/s 0.5993 Ops/s $\color{#d91a1a}-0.19\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.5521s 1.4515s 0.6889 Ops/s 0.6891 Ops/s $\color{#d91a1a}-0.02\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 1.9818s 1.9155s 0.5221 Ops/s 0.5240 Ops/s $\color{#d91a1a}-0.37\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.8123s 1.7307s 0.5778 Ops/s 0.5902 Ops/s $\color{#d91a1a}-2.11\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.6010s 4.5536s 0.2196 Ops/s 0.2185 Ops/s $\color{#35bf28}+0.50\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.5465s 4.4416s 0.2251 Ops/s 0.2268 Ops/s $\color{#d91a1a}-0.75\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 2.0376s 1.9409s 0.5152 Ops/s 0.5076 Ops/s $\color{#35bf28}+1.50\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.8935s 1.7553s 0.5697 Ops/s 0.6017 Ops/s $\textbf{\color{#d91a1a}-5.32\%}$
test_values[generalized_advantage_estimate-True-True] 10.7989ms 10.3237ms 96.8647 Ops/s 98.4961 Ops/s $\color{#d91a1a}-1.66\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.8447ms 17.7595ms 56.3078 Ops/s 88.0074 Ops/s $\textbf{\color{#d91a1a}-36.02\%}$
test_values[td0_return_estimate-False-False] 3.8335ms 0.2058ms 4.8587 KOps/s 7.7017 KOps/s $\textbf{\color{#d91a1a}-36.91\%}$
test_values[td1_return_estimate-False-False] 27.7335ms 27.3897ms 36.5101 Ops/s 36.0102 Ops/s $\color{#35bf28}+1.39\%$
test_values[vec_td1_return_estimate-False-False] 18.7542ms 17.8046ms 56.1652 Ops/s 88.1430 Ops/s $\textbf{\color{#d91a1a}-36.28\%}$
test_values[td_lambda_return_estimate-True-False] 42.9686ms 40.3336ms 24.7932 Ops/s 23.8845 Ops/s $\color{#35bf28}+3.80\%$
test_values[vec_td_lambda_return_estimate-True-False] 21.9664ms 17.9157ms 55.8169 Ops/s 88.3387 Ops/s $\textbf{\color{#d91a1a}-36.81\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.3005ms 9.1792ms 108.9418 Ops/s 109.9576 Ops/s $\color{#d91a1a}-0.92\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7735ms 1.4825ms 674.5208 Ops/s 667.0957 Ops/s $\color{#35bf28}+1.11\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5168ms 0.4155ms 2.4068 KOps/s 2.3864 KOps/s $\color{#35bf28}+0.86\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 35.9326ms 34.6159ms 28.8884 Ops/s 37.0815 Ops/s $\textbf{\color{#d91a1a}-22.09\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.0228ms 1.7674ms 565.7941 Ops/s 548.7147 Ops/s $\color{#35bf28}+3.11\%$
test_dqn_speed[False-None] 1.5869ms 1.3894ms 719.7173 Ops/s 713.3420 Ops/s $\color{#35bf28}+0.89\%$
test_dqn_speed[False-backward] 1.9891ms 1.8954ms 527.5987 Ops/s 525.9112 Ops/s $\color{#35bf28}+0.32\%$
test_dqn_speed[True-None] 0.7199ms 0.5243ms 1.9072 KOps/s 1.8607 KOps/s $\color{#35bf28}+2.49\%$
test_dqn_speed[True-backward] 1.0060ms 0.9612ms 1.0403 KOps/s 869.0021 Ops/s $\textbf{\color{#35bf28}+19.72\%}$
test_dqn_speed[reduce-overhead-None] 0.5744ms 0.5175ms 1.9323 KOps/s 1.8425 KOps/s $\color{#35bf28}+4.87\%$
test_dqn_speed[reduce-overhead-backward] 0.9801ms 0.9449ms 1.0583 KOps/s 932.3574 Ops/s $\textbf{\color{#35bf28}+13.51\%}$
test_ddpg_speed[False-None] 3.1114ms 2.8161ms 355.1008 Ops/s 355.2052 Ops/s $\color{#d91a1a}-0.03\%$
test_ddpg_speed[False-backward] 4.1092ms 4.0328ms 247.9665 Ops/s 248.2071 Ops/s $\color{#d91a1a}-0.10\%$
test_ddpg_speed[True-None] 1.4395ms 1.3639ms 733.1659 Ops/s 715.0729 Ops/s $\color{#35bf28}+2.53\%$
test_ddpg_speed[True-backward] 2.3588ms 2.3178ms 431.4344 Ops/s 355.4640 Ops/s $\textbf{\color{#35bf28}+21.37\%}$
test_ddpg_speed[reduce-overhead-None] 1.4655ms 1.3555ms 737.7121 Ops/s 708.1679 Ops/s $\color{#35bf28}+4.17\%$
test_ddpg_speed[reduce-overhead-backward] 2.3981ms 2.3048ms 433.8782 Ops/s 389.8993 Ops/s $\textbf{\color{#35bf28}+11.28\%}$
test_sac_speed[False-None] 8.2792ms 7.8006ms 128.1959 Ops/s 126.2100 Ops/s $\color{#35bf28}+1.57\%$
test_sac_speed[False-backward] 11.5142ms 11.1062ms 90.0396 Ops/s 89.9412 Ops/s $\color{#35bf28}+0.11\%$
test_sac_speed[True-None] 2.3226ms 2.0932ms 477.7285 Ops/s 468.6671 Ops/s $\color{#35bf28}+1.93\%$
test_sac_speed[True-backward] 3.9839ms 3.8957ms 256.6937 Ops/s 245.7879 Ops/s $\color{#35bf28}+4.44\%$
test_sac_speed[reduce-overhead-None] 2.3040ms 2.0813ms 480.4575 Ops/s 472.8504 Ops/s $\color{#35bf28}+1.61\%$
test_sac_speed[reduce-overhead-backward] 4.1000ms 3.9299ms 254.4624 Ops/s 217.7223 Ops/s $\textbf{\color{#35bf28}+16.87\%}$
test_redq_speed[False-None] 14.5673ms 10.3329ms 96.7783 Ops/s 97.6498 Ops/s $\color{#d91a1a}-0.89\%$
test_redq_speed[False-backward] 23.6402ms 18.2771ms 54.7132 Ops/s 56.7401 Ops/s $\color{#d91a1a}-3.57\%$
test_redq_speed[True-None] 4.5807ms 4.3853ms 228.0369 Ops/s 226.7422 Ops/s $\color{#35bf28}+0.57\%$
test_redq_speed[True-backward] 10.1872ms 9.7402ms 102.6670 Ops/s 99.6831 Ops/s $\color{#35bf28}+2.99\%$
test_redq_speed[reduce-overhead-None] 4.4877ms 4.3208ms 231.4401 Ops/s 234.8488 Ops/s $\color{#d91a1a}-1.45\%$
test_redq_speed[reduce-overhead-backward] 10.3970ms 9.8259ms 101.7718 Ops/s 105.6543 Ops/s $\color{#d91a1a}-3.67\%$
test_redq_deprec_speed[False-None] 11.2236ms 10.9075ms 91.6803 Ops/s 90.4807 Ops/s $\color{#35bf28}+1.33\%$
test_redq_deprec_speed[False-backward] 16.0800ms 15.7260ms 63.5889 Ops/s 63.5611 Ops/s $\color{#35bf28}+0.04\%$
test_redq_deprec_speed[True-None] 3.7657ms 3.6043ms 277.4432 Ops/s 275.9758 Ops/s $\color{#35bf28}+0.53\%$
test_redq_deprec_speed[True-backward] 7.6660ms 7.4345ms 134.5087 Ops/s 123.7985 Ops/s $\textbf{\color{#35bf28}+8.65\%}$
test_redq_deprec_speed[reduce-overhead-None] 3.7697ms 3.5840ms 279.0197 Ops/s 267.6876 Ops/s $\color{#35bf28}+4.23\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.6799ms 7.5104ms 133.1491 Ops/s 132.9120 Ops/s $\color{#35bf28}+0.18\%$
test_td3_speed[False-None] 8.1442ms 7.9250ms 126.1832 Ops/s 125.7925 Ops/s $\color{#35bf28}+0.31\%$
test_td3_speed[False-backward] 11.3592ms 10.7761ms 92.7984 Ops/s 92.3868 Ops/s $\color{#35bf28}+0.45\%$
test_td3_speed[True-None] 1.8432ms 1.7878ms 559.3524 Ops/s 561.3730 Ops/s $\color{#d91a1a}-0.36\%$
test_td3_speed[True-backward] 3.6506ms 3.5336ms 282.9945 Ops/s 255.6995 Ops/s $\textbf{\color{#35bf28}+10.67\%}$
test_td3_speed[reduce-overhead-None] 1.7805ms 1.7511ms 571.0696 Ops/s 571.0273 Ops/s $+0.01\%$
test_td3_speed[reduce-overhead-backward] 3.6656ms 3.5544ms 281.3429 Ops/s 236.7603 Ops/s $\textbf{\color{#35bf28}+18.83\%}$
test_cql_speed[False-None] 28.7664ms 25.9925ms 38.4726 Ops/s 38.6946 Ops/s $\color{#d91a1a}-0.57\%$
test_cql_speed[False-backward] 40.3766ms 35.9745ms 27.7974 Ops/s 28.7701 Ops/s $\color{#d91a1a}-3.38\%$
test_cql_speed[True-None] 12.4585ms 12.1166ms 82.5312 Ops/s 82.2866 Ops/s $\color{#35bf28}+0.30\%$
test_cql_speed[True-backward] 18.3116ms 18.1073ms 55.2264 Ops/s 57.3451 Ops/s $\color{#d91a1a}-3.69\%$
test_cql_speed[reduce-overhead-None] 12.6578ms 12.3363ms 81.0619 Ops/s 81.0591 Ops/s $+0.00\%$
test_cql_speed[reduce-overhead-backward] 18.4470ms 18.0985ms 55.2532 Ops/s 56.3927 Ops/s $\color{#d91a1a}-2.02\%$
test_a2c_speed[False-None] 5.5269ms 5.3408ms 187.2382 Ops/s 177.3766 Ops/s $\textbf{\color{#35bf28}+5.56\%}$
test_a2c_speed[False-backward] 12.0492ms 11.7734ms 84.9370 Ops/s 84.1760 Ops/s $\color{#35bf28}+0.90\%$
test_a2c_speed[True-None] 3.7497ms 3.6281ms 275.6280 Ops/s 255.2711 Ops/s $\textbf{\color{#35bf28}+7.97\%}$
test_a2c_speed[True-backward] 8.7419ms 8.4779ms 117.9532 Ops/s 110.2635 Ops/s $\textbf{\color{#35bf28}+6.97\%}$
test_a2c_speed[reduce-overhead-None] 3.8364ms 3.6625ms 273.0357 Ops/s 273.4735 Ops/s $\color{#d91a1a}-0.16\%$
test_a2c_speed[reduce-overhead-backward] 8.8701ms 8.6138ms 116.0930 Ops/s 115.7944 Ops/s $\color{#35bf28}+0.26\%$
test_ppo_speed[False-None] 6.0304ms 5.8007ms 172.3933 Ops/s 169.1343 Ops/s $\color{#35bf28}+1.93\%$
test_ppo_speed[False-backward] 12.7816ms 12.5131ms 79.9162 Ops/s 80.2096 Ops/s $\color{#d91a1a}-0.37\%$
test_ppo_speed[True-None] 3.7122ms 3.5554ms 281.2611 Ops/s 273.0883 Ops/s $\color{#35bf28}+2.99\%$
test_ppo_speed[True-backward] 8.4005ms 8.2130ms 121.7585 Ops/s 119.1700 Ops/s $\color{#35bf28}+2.17\%$
test_ppo_speed[reduce-overhead-None] 3.7223ms 3.5598ms 280.9149 Ops/s 281.9254 Ops/s $\color{#d91a1a}-0.36\%$
test_ppo_speed[reduce-overhead-backward] 8.7970ms 8.6171ms 116.0485 Ops/s 115.7476 Ops/s $\color{#35bf28}+0.26\%$
test_reinforce_speed[False-None] 4.9031ms 4.5188ms 221.2998 Ops/s 219.0044 Ops/s $\color{#35bf28}+1.05\%$
test_reinforce_speed[False-backward] 7.5773ms 7.3776ms 135.5445 Ops/s 136.7594 Ops/s $\color{#d91a1a}-0.89\%$
test_reinforce_speed[True-None] 2.9708ms 2.8129ms 355.5015 Ops/s 344.0076 Ops/s $\color{#35bf28}+3.34\%$
test_reinforce_speed[True-backward] 7.8492ms 7.6143ms 131.3326 Ops/s 116.1856 Ops/s $\textbf{\color{#35bf28}+13.04\%}$
test_reinforce_speed[reduce-overhead-None] 3.0208ms 2.8029ms 356.7678 Ops/s 351.2553 Ops/s $\color{#35bf28}+1.57\%$
test_reinforce_speed[reduce-overhead-backward] 8.0588ms 7.8258ms 127.7823 Ops/s 126.3027 Ops/s $\color{#35bf28}+1.17\%$
test_iql_speed[False-None] 20.3314ms 19.6603ms 50.8638 Ops/s 51.7906 Ops/s $\color{#d91a1a}-1.79\%$
test_iql_speed[False-backward] 30.7615ms 30.1364ms 33.1825 Ops/s 33.6155 Ops/s $\color{#d91a1a}-1.29\%$
test_iql_speed[True-None] 8.6851ms 8.4070ms 118.9492 Ops/s 116.7411 Ops/s $\color{#35bf28}+1.89\%$
test_iql_speed[True-backward] 16.7705ms 16.5255ms 60.5127 Ops/s 59.6256 Ops/s $\color{#35bf28}+1.49\%$
test_iql_speed[reduce-overhead-None] 8.6943ms 8.4388ms 118.5004 Ops/s 124.5442 Ops/s $\color{#d91a1a}-4.85\%$
test_iql_speed[reduce-overhead-backward] 17.0093ms 16.8284ms 59.4235 Ops/s 58.1062 Ops/s $\color{#35bf28}+2.27\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5140ms 5.8978ms 169.5559 Ops/s 169.2060 Ops/s $\color{#35bf28}+0.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6008ms 0.3246ms 3.0807 KOps/s 2.7919 KOps/s $\textbf{\color{#35bf28}+10.34\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5758ms 0.3143ms 3.1817 KOps/s 2.8106 KOps/s $\textbf{\color{#35bf28}+13.20\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9010ms 5.6427ms 177.2196 Ops/s 177.0289 Ops/s $\color{#35bf28}+0.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.2216ms 0.3273ms 3.0551 KOps/s 2.7123 KOps/s $\textbf{\color{#35bf28}+12.64\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5157ms 0.3114ms 3.2116 KOps/s 2.9085 KOps/s $\textbf{\color{#35bf28}+10.42\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6701ms 1.2931ms 773.3280 Ops/s 708.7613 Ops/s $\textbf{\color{#35bf28}+9.11\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5645ms 1.2206ms 819.2998 Ops/s 761.0322 Ops/s $\textbf{\color{#35bf28}+7.66\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.2545ms 5.9467ms 168.1618 Ops/s 171.4507 Ops/s $\color{#d91a1a}-1.92\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.2577ms 0.5087ms 1.9659 KOps/s 2.0204 KOps/s $\color{#d91a1a}-2.70\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6990ms 0.4826ms 2.0720 KOps/s 2.0815 KOps/s $\color{#d91a1a}-0.45\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.7486ms 5.6319ms 177.5596 Ops/s 175.8221 Ops/s $\color{#35bf28}+0.99\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7319ms 0.2790ms 3.5843 KOps/s 2.7598 KOps/s $\textbf{\color{#35bf28}+29.87\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4763ms 0.2623ms 3.8128 KOps/s 2.8739 KOps/s $\textbf{\color{#35bf28}+32.67\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9410ms 5.5965ms 178.6836 Ops/s 178.5749 Ops/s $\color{#35bf28}+0.06\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5104ms 0.3516ms 2.8445 KOps/s 2.7829 KOps/s $\color{#35bf28}+2.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5237ms 0.3419ms 2.9246 KOps/s 2.9047 KOps/s $\color{#35bf28}+0.68\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.3072ms 5.7460ms 174.0333 Ops/s 173.3517 Ops/s $\color{#35bf28}+0.39\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.8767ms 0.4873ms 2.0521 KOps/s 2.0582 KOps/s $\color{#d91a1a}-0.30\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8537ms 0.5008ms 1.9966 KOps/s 2.1119 KOps/s $\textbf{\color{#d91a1a}-5.46\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.6297s 17.4931ms 57.1653 Ops/s 196.4612 Ops/s $\textbf{\color{#d91a1a}-70.90\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.0420ms 2.0483ms 488.1997 Ops/s 430.6188 Ops/s $\textbf{\color{#35bf28}+13.37\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.8832ms 1.2090ms 827.1340 Ops/s 869.4902 Ops/s $\color{#d91a1a}-4.87\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.7487ms 5.0184ms 199.2648 Ops/s 53.5538 Ops/s $\textbf{\color{#35bf28}+272.08\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.9537ms 2.0503ms 487.7429 Ops/s 496.8820 Ops/s $\color{#d91a1a}-1.84\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.8935ms 1.2017ms 832.1449 Ops/s 946.4569 Ops/s $\textbf{\color{#d91a1a}-12.08\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.6360s 17.9227ms 55.7952 Ops/s 188.7447 Ops/s $\textbf{\color{#d91a1a}-70.44\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 12.7694ms 2.1476ms 465.6319 Ops/s 457.7372 Ops/s $\color{#35bf28}+1.72\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.4436ms 1.0120ms 988.1789 Ops/s 713.3933 Ops/s $\textbf{\color{#35bf28}+38.52\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 39.1602ms 34.0682ms 29.3529 Ops/s 28.6231 Ops/s $\color{#35bf28}+2.55\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.6525ms 17.7679ms 56.2813 Ops/s 55.4960 Ops/s $\color{#35bf28}+1.42\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 36.5721ms 34.4878ms 28.9958 Ops/s 28.5209 Ops/s $\color{#35bf28}+1.67\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.8789ms 17.8328ms 56.0764 Ops/s 55.8136 Ops/s $\color{#35bf28}+0.47\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 38.4065ms 36.2731ms 27.5687 Ops/s 26.9672 Ops/s $\color{#35bf28}+2.23\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.3617ms 19.6288ms 50.9454 Ops/s 51.2765 Ops/s $\color{#d91a1a}-0.65\%$

This was referenced Dec 31, 2025
Merged
Open
Closed
vmoens added a commit that referenced this pull request Jan 1, 2026
@vmoens
[Feature] auto_wrap_envs in PEnv
d781f9e 
ghstack-source-id: 8dceb5d
Pull-Request: #3284
@vmoens vmoens merged commit 469c599 into gh/vmoens/170/base Jan 1, 2026
101 of 106 checks passed
@vmoens vmoens deleted the gh/vmoens/170/head branch January 1, 2026 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens