Skip to content

[Feature] Auto-wrap lambda functions with EnvCreator for spawn compatibility #3277

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vmoens merged 1 commit into from
Dec 31, 2025
Merged

[Feature] Auto-wrap lambda functions with EnvCreator for spawn compatibility #3277

vmoens merged 1 commit into from
Dec 31, 2025

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Dec 31, 2025 •
edited
Loading

Stack from ghstack (oldest at bottom):

@vmoens
Update
f057f43 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Dec 31, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3277

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 31, 2025
This was referenced Dec 31, 2025
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
vmoens added a commit that referenced this pull request Dec 31, 2025
@vmoens
[Feature] Auto-wrap lambda functions with EnvCreator for spawn compat…
d250c18 
…ibility

ghstack-source-id: ef34962
Pull-Request: #3277
@vmoens vmoens merged commit f057f43 into gh/vmoens/177/base Dec 31, 2025
81 of 96 checks passed
@vmoens vmoens deleted the gh/vmoens/177/head branch December 31, 2025 09:17
@github-actions
Copy link

github-actions bot commented Dec 31, 2025

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 164. Improved: $\large\color{#35bf28}15$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 82.7504μs 81.9230μs 12.2066 KOps/s 12.1440 KOps/s $\color{#35bf28}+0.52\%$
test_tensor_to_bytestream_speed[torch.save] 0.1413ms 0.1407ms 7.1057 KOps/s 7.0479 KOps/s $\color{#35bf28}+0.82\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1276s 0.1273s 7.8525 Ops/s 7.7534 Ops/s $\color{#35bf28}+1.28\%$
test_tensor_to_bytestream_speed[numpy] 2.8116μs 2.8040μs 356.6389 KOps/s 365.5735 KOps/s $\color{#d91a1a}-2.44\%$
test_tensor_to_bytestream_speed[safetensors] 38.6928μs 38.5272μs 25.9557 KOps/s 26.2990 KOps/s $\color{#d91a1a}-1.31\%$
test_simple 0.5559s 0.5550s 1.8017 Ops/s 1.7264 Ops/s $\color{#35bf28}+4.36\%$
test_transformed 1.1386s 1.1377s 0.8790 Ops/s 0.8607 Ops/s $\color{#35bf28}+2.12\%$
test_serial 1.6798s 1.6777s 0.5961 Ops/s 0.5875 Ops/s $\color{#35bf28}+1.47\%$
test_parallel 1.1136s 1.0922s 0.9156 Ops/s 0.8666 Ops/s $\textbf{\color{#35bf28}+5.65\%}$
test_step_mdp_speed[True-True-True-True-True] 0.3206ms 44.9724μs 22.2359 KOps/s 22.1377 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[True-True-True-True-False] 51.8910μs 25.1129μs 39.8201 KOps/s 39.5833 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-True-True-False-True] 54.9610μs 24.8698μs 40.2095 KOps/s 38.9746 KOps/s $\color{#35bf28}+3.17\%$
test_step_mdp_speed[True-True-True-False-False] 42.0810μs 13.6516μs 73.2514 KOps/s 70.3372 KOps/s $\color{#35bf28}+4.14\%$
test_step_mdp_speed[True-True-False-True-True] 79.5310μs 47.7598μs 20.9381 KOps/s 20.7936 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-True-False-True-False] 95.6110μs 27.4932μs 36.3726 KOps/s 35.3880 KOps/s $\color{#35bf28}+2.78\%$
test_step_mdp_speed[True-True-False-False-True] 0.1012ms 28.0320μs 35.6735 KOps/s 35.5590 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-True-False-False-False] 44.9610μs 16.5085μs 60.5749 KOps/s 59.7582 KOps/s $\color{#35bf28}+1.37\%$
test_step_mdp_speed[True-False-True-True-True] 90.9210μs 51.1556μs 19.5482 KOps/s 19.4865 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-False-True-True-False] 63.4910μs 30.2678μs 33.0384 KOps/s 31.8386 KOps/s $\color{#35bf28}+3.77\%$
test_step_mdp_speed[True-False-True-False-True] 57.0710μs 27.8032μs 35.9671 KOps/s 35.2666 KOps/s $\color{#35bf28}+1.99\%$
test_step_mdp_speed[True-False-True-False-False] 46.1110μs 16.4966μs 60.6187 KOps/s 59.9736 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[True-False-False-True-True] 87.5410μs 52.9816μs 18.8745 KOps/s 18.8303 KOps/s $\color{#35bf28}+0.23\%$
test_step_mdp_speed[True-False-False-True-False] 57.2510μs 33.3468μs 29.9879 KOps/s 29.9332 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[True-False-False-False-True] 61.8310μs 30.6406μs 32.6365 KOps/s 32.7782 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-False-False-False-False] 52.9710μs 19.1518μs 52.2145 KOps/s 51.7030 KOps/s $\color{#35bf28}+0.99\%$
test_step_mdp_speed[False-True-True-True-True] 78.8110μs 50.5322μs 19.7893 KOps/s 19.7597 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[False-True-True-True-False] 61.3010μs 30.3070μs 32.9957 KOps/s 32.4255 KOps/s $\color{#35bf28}+1.76\%$
test_step_mdp_speed[False-True-True-False-True] 2.3561ms 31.5683μs 31.6773 KOps/s 31.3590 KOps/s $\color{#35bf28}+1.02\%$
test_step_mdp_speed[False-True-True-False-False] 52.6810μs 18.1012μs 55.2449 KOps/s 54.2785 KOps/s $\color{#35bf28}+1.78\%$
test_step_mdp_speed[False-True-False-True-True] 86.7710μs 52.8929μs 18.9061 KOps/s 18.9442 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-True-False-True-False] 61.3710μs 33.1459μs 30.1697 KOps/s 30.1270 KOps/s $\color{#35bf28}+0.14\%$
test_step_mdp_speed[False-True-False-False-True] 68.8010μs 33.8517μs 29.5406 KOps/s 29.4046 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-True-False-False-False] 51.4910μs 20.8836μs 47.8844 KOps/s 47.8679 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[False-False-True-True-True] 89.0820μs 55.5053μs 18.0163 KOps/s 17.7454 KOps/s $\color{#35bf28}+1.53\%$
test_step_mdp_speed[False-False-True-True-False] 66.3010μs 35.9577μs 27.8104 KOps/s 27.5519 KOps/s $\color{#35bf28}+0.94\%$
test_step_mdp_speed[False-False-True-False-True] 64.6020μs 33.7163μs 29.6592 KOps/s 29.3589 KOps/s $\color{#35bf28}+1.02\%$
test_step_mdp_speed[False-False-True-False-False] 51.4810μs 20.8927μs 47.8637 KOps/s 47.7698 KOps/s $\color{#35bf28}+0.20\%$
test_step_mdp_speed[False-False-False-True-True] 0.1261ms 56.5633μs 17.6793 KOps/s 17.2583 KOps/s $\color{#35bf28}+2.44\%$
test_step_mdp_speed[False-False-False-True-False] 68.5610μs 38.1627μs 26.2036 KOps/s 25.9020 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[False-False-False-False-True] 74.3310μs 36.4878μs 27.4064 KOps/s 27.4689 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[False-False-False-False-False] 97.6220μs 23.0534μs 43.3776 KOps/s 42.4935 KOps/s $\color{#35bf28}+2.08\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.8891s 0.7879s 1.2692 Ops/s 1.2606 Ops/s $\color{#35bf28}+0.68\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7528s 0.6524s 1.5329 Ops/s 1.5212 Ops/s $\color{#35bf28}+0.77\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7951s 1.7166s 0.5825 Ops/s 0.5811 Ops/s $\color{#35bf28}+0.25\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.5655s 1.4859s 0.6730 Ops/s 0.6640 Ops/s $\color{#35bf28}+1.35\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 2.0412s 1.9611s 0.5099 Ops/s 0.5102 Ops/s $\color{#d91a1a}-0.05\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.8147s 1.7351s 0.5763 Ops/s 0.5782 Ops/s $\color{#d91a1a}-0.32\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.9619s 4.7020s 0.2127 Ops/s 0.2096 Ops/s $\color{#35bf28}+1.49\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.6767s 4.4690s 0.2238 Ops/s 0.2217 Ops/s $\color{#35bf28}+0.91\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 2.1020s 2.0126s 0.4969 Ops/s 0.4993 Ops/s $\color{#d91a1a}-0.49\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.7953s 1.6974s 0.5891 Ops/s 0.5897 Ops/s $\color{#d91a1a}-0.10\%$
test_values[generalized_advantage_estimate-True-True] 11.2013ms 10.3200ms 96.8990 Ops/s 97.4220 Ops/s $\color{#d91a1a}-0.54\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.2439ms 17.9894ms 55.5882 Ops/s 85.3100 Ops/s $\textbf{\color{#d91a1a}-34.84\%}$
test_values[td0_return_estimate-False-False] 0.2291ms 0.1250ms 7.9982 KOps/s 7.8492 KOps/s $\color{#35bf28}+1.90\%$
test_values[td1_return_estimate-False-False] 28.8809ms 28.1680ms 35.5012 Ops/s 35.4853 Ops/s $\color{#35bf28}+0.05\%$
test_values[vec_td1_return_estimate-False-False] 19.1022ms 18.2250ms 54.8696 Ops/s 87.7867 Ops/s $\textbf{\color{#d91a1a}-37.50\%}$
test_values[td_lambda_return_estimate-True-False] 41.1820ms 40.8092ms 24.5042 Ops/s 23.8368 Ops/s $\color{#35bf28}+2.80\%$
test_values[vec_td_lambda_return_estimate-True-False] 19.7685ms 18.1615ms 55.0616 Ops/s 88.2524 Ops/s $\textbf{\color{#d91a1a}-37.61\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.3066ms 9.1844ms 108.8806 Ops/s 108.5234 Ops/s $\color{#35bf28}+0.33\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7231ms 1.5531ms 643.8713 Ops/s 641.9848 Ops/s $\color{#35bf28}+0.29\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5922ms 0.4102ms 2.4377 KOps/s 2.4017 KOps/s $\color{#35bf28}+1.50\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 35.8411ms 35.1386ms 28.4587 Ops/s 37.1274 Ops/s $\textbf{\color{#d91a1a}-23.35\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.2368ms 1.7861ms 559.8744 Ops/s 559.9939 Ops/s $\color{#d91a1a}-0.02\%$
test_dqn_speed[False-None] 1.5366ms 1.3972ms 715.7148 Ops/s 715.2478 Ops/s $\color{#35bf28}+0.07\%$
test_dqn_speed[False-backward] 2.2980ms 1.9051ms 524.8974 Ops/s 522.0488 Ops/s $\color{#35bf28}+0.55\%$
test_dqn_speed[True-None] 0.7431ms 0.5379ms 1.8591 KOps/s 1.8362 KOps/s $\color{#35bf28}+1.25\%$
test_dqn_speed[True-backward] 1.1113ms 0.9957ms 1.0043 KOps/s 897.4254 Ops/s $\textbf{\color{#35bf28}+11.91\%}$
test_dqn_speed[reduce-overhead-None] 0.9555ms 0.5317ms 1.8807 KOps/s 1.8207 KOps/s $\color{#35bf28}+3.29\%$
test_dqn_speed[reduce-overhead-backward] 1.0194ms 0.9798ms 1.0206 KOps/s 905.4146 Ops/s $\textbf{\color{#35bf28}+12.72\%}$
test_ddpg_speed[False-None] 3.2358ms 2.8611ms 349.5158 Ops/s 345.8803 Ops/s $\color{#35bf28}+1.05\%$
test_ddpg_speed[False-backward] 4.1738ms 4.0511ms 246.8467 Ops/s 241.9774 Ops/s $\color{#35bf28}+2.01\%$
test_ddpg_speed[True-None] 1.8100ms 1.3934ms 717.6856 Ops/s 716.1237 Ops/s $\color{#35bf28}+0.22\%$
test_ddpg_speed[True-backward] 2.6159ms 2.4023ms 416.2740 Ops/s 381.1702 Ops/s $\textbf{\color{#35bf28}+9.21\%}$
test_ddpg_speed[reduce-overhead-None] 2.0916ms 1.4072ms 710.6147 Ops/s 708.8107 Ops/s $\color{#35bf28}+0.25\%$
test_ddpg_speed[reduce-overhead-backward] 2.4168ms 2.3636ms 423.0914 Ops/s 342.3023 Ops/s $\textbf{\color{#35bf28}+23.60\%}$
test_sac_speed[False-None] 8.4186ms 7.9076ms 126.4603 Ops/s 125.9821 Ops/s $\color{#35bf28}+0.38\%$
test_sac_speed[False-backward] 11.5352ms 11.0665ms 90.3626 Ops/s 89.7754 Ops/s $\color{#35bf28}+0.65\%$
test_sac_speed[True-None] 2.4608ms 2.1706ms 460.7067 Ops/s 447.2224 Ops/s $\color{#35bf28}+3.02\%$
test_sac_speed[True-backward] 4.1819ms 4.0717ms 245.5958 Ops/s 220.3369 Ops/s $\textbf{\color{#35bf28}+11.46\%}$
test_sac_speed[reduce-overhead-None] 2.5976ms 2.1568ms 463.6581 Ops/s 445.3191 Ops/s $\color{#35bf28}+4.12\%$
test_sac_speed[reduce-overhead-backward] 4.4333ms 4.1019ms 243.7908 Ops/s 228.5549 Ops/s $\textbf{\color{#35bf28}+6.67\%}$
test_redq_speed[False-None] 15.2220ms 10.6848ms 93.5908 Ops/s 95.0681 Ops/s $\color{#d91a1a}-1.55\%$
test_redq_speed[False-backward] 22.6736ms 18.2578ms 54.7712 Ops/s 55.7349 Ops/s $\color{#d91a1a}-1.73\%$
test_redq_speed[True-None] 4.7484ms 4.4965ms 222.3940 Ops/s 215.7654 Ops/s $\color{#35bf28}+3.07\%$
test_redq_speed[True-backward] 10.2181ms 9.9441ms 100.5625 Ops/s 99.3764 Ops/s $\color{#35bf28}+1.19\%$
test_redq_speed[reduce-overhead-None] 4.7979ms 4.5000ms 222.2235 Ops/s 215.7767 Ops/s $\color{#35bf28}+2.99\%$
test_redq_speed[reduce-overhead-backward] 10.7785ms 10.2206ms 97.8414 Ops/s 99.1580 Ops/s $\color{#d91a1a}-1.33\%$
test_redq_deprec_speed[False-None] 11.4081ms 11.0713ms 90.3239 Ops/s 89.7203 Ops/s $\color{#35bf28}+0.67\%$
test_redq_deprec_speed[False-backward] 16.3575ms 15.9352ms 62.7541 Ops/s 62.9828 Ops/s $\color{#d91a1a}-0.36\%$
test_redq_deprec_speed[True-None] 4.0914ms 3.7703ms 265.2321 Ops/s 258.7319 Ops/s $\color{#35bf28}+2.51\%$
test_redq_deprec_speed[True-backward] 8.1167ms 7.9078ms 126.4582 Ops/s 124.4405 Ops/s $\color{#35bf28}+1.62\%$
test_redq_deprec_speed[reduce-overhead-None] 4.2815ms 3.7476ms 266.8396 Ops/s 257.0279 Ops/s $\color{#35bf28}+3.82\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.0736ms 7.7936ms 128.3103 Ops/s 124.7508 Ops/s $\color{#35bf28}+2.85\%$
test_td3_speed[False-None] 8.2829ms 7.9855ms 125.2265 Ops/s 124.8378 Ops/s $\color{#35bf28}+0.31\%$
test_td3_speed[False-backward] 11.2378ms 10.8202ms 92.4197 Ops/s 92.4517 Ops/s $\color{#d91a1a}-0.03\%$
test_td3_speed[True-None] 1.8939ms 1.8647ms 536.2681 Ops/s 533.0833 Ops/s $\color{#35bf28}+0.60\%$
test_td3_speed[True-backward] 3.8559ms 3.6883ms 271.1282 Ops/s 269.7193 Ops/s $\color{#35bf28}+0.52\%$
test_td3_speed[reduce-overhead-None] 1.8874ms 1.8463ms 541.6371 Ops/s 542.2451 Ops/s $\color{#d91a1a}-0.11\%$
test_td3_speed[reduce-overhead-backward] 3.8615ms 3.7436ms 267.1250 Ops/s 245.8323 Ops/s $\textbf{\color{#35bf28}+8.66\%}$
test_cql_speed[False-None] 29.3826ms 26.2937ms 38.0320 Ops/s 37.9132 Ops/s $\color{#35bf28}+0.31\%$
test_cql_speed[False-backward] 38.6925ms 35.5776ms 28.1076 Ops/s 28.1075 Ops/s $+0.00\%$
test_cql_speed[True-None] 13.0000ms 12.5172ms 79.8900 Ops/s 76.6981 Ops/s $\color{#35bf28}+4.16\%$
test_cql_speed[True-backward] 18.7642ms 18.4485ms 54.2048 Ops/s 55.4486 Ops/s $\color{#d91a1a}-2.24\%$
test_cql_speed[reduce-overhead-None] 12.9759ms 12.7579ms 78.3827 Ops/s 79.9351 Ops/s $\color{#d91a1a}-1.94\%$
test_cql_speed[reduce-overhead-backward] 19.4489ms 18.9069ms 52.8907 Ops/s 54.6111 Ops/s $\color{#d91a1a}-3.15\%$
test_a2c_speed[False-None] 5.8099ms 5.4707ms 182.7916 Ops/s 180.0801 Ops/s $\color{#35bf28}+1.51\%$
test_a2c_speed[False-backward] 12.4373ms 11.9140ms 83.9350 Ops/s 83.2093 Ops/s $\color{#35bf28}+0.87\%$
test_a2c_speed[True-None] 4.0666ms 3.7195ms 268.8545 Ops/s 256.9865 Ops/s $\color{#35bf28}+4.62\%$
test_a2c_speed[True-backward] 9.1743ms 8.7257ms 114.6045 Ops/s 114.6836 Ops/s $\color{#d91a1a}-0.07\%$
test_a2c_speed[reduce-overhead-None] 4.2623ms 3.7538ms 266.3958 Ops/s 264.4786 Ops/s $\color{#35bf28}+0.72\%$
test_a2c_speed[reduce-overhead-backward] 9.5821ms 8.9088ms 112.2490 Ops/s 103.3680 Ops/s $\textbf{\color{#35bf28}+8.59\%}$
test_ppo_speed[False-None] 6.8747ms 5.8358ms 171.3574 Ops/s 168.1460 Ops/s $\color{#35bf28}+1.91\%$
test_ppo_speed[False-backward] 12.6517ms 12.3430ms 81.0173 Ops/s 78.6847 Ops/s $\color{#35bf28}+2.96\%$
test_ppo_speed[True-None] 4.0447ms 3.6886ms 271.1059 Ops/s 271.1599 Ops/s $\color{#d91a1a}-0.02\%$
test_ppo_speed[True-backward] 9.0612ms 8.5285ms 117.2539 Ops/s 116.3581 Ops/s $\color{#35bf28}+0.77\%$
test_ppo_speed[reduce-overhead-None] 4.0590ms 3.6882ms 271.1378 Ops/s 272.7096 Ops/s $\color{#d91a1a}-0.58\%$
test_ppo_speed[reduce-overhead-backward] 9.1267ms 8.7585ms 114.1754 Ops/s 111.9746 Ops/s $\color{#35bf28}+1.97\%$
test_reinforce_speed[False-None] 4.9701ms 4.5213ms 221.1741 Ops/s 213.9937 Ops/s $\color{#35bf28}+3.36\%$
test_reinforce_speed[False-backward] 7.7218ms 7.3678ms 135.7257 Ops/s 131.9770 Ops/s $\color{#35bf28}+2.84\%$
test_reinforce_speed[True-None] 3.2582ms 2.9195ms 342.5252 Ops/s 338.8295 Ops/s $\color{#35bf28}+1.09\%$
test_reinforce_speed[True-backward] 7.9447ms 7.7524ms 128.9924 Ops/s 126.2347 Ops/s $\color{#35bf28}+2.18\%$
test_reinforce_speed[reduce-overhead-None] 3.3591ms 2.9004ms 344.7835 Ops/s 344.6348 Ops/s $\color{#35bf28}+0.04\%$
test_reinforce_speed[reduce-overhead-backward] 8.2295ms 8.0026ms 124.9595 Ops/s 122.9542 Ops/s $\color{#35bf28}+1.63\%$
test_iql_speed[False-None] 24.8613ms 20.2704ms 49.3330 Ops/s 47.7210 Ops/s $\color{#35bf28}+3.38\%$
test_iql_speed[False-backward] 35.6212ms 30.7042ms 32.5688 Ops/s 32.6085 Ops/s $\color{#d91a1a}-0.12\%$
test_iql_speed[True-None] 8.8740ms 8.6090ms 116.1569 Ops/s 112.4039 Ops/s $\color{#35bf28}+3.34\%$
test_iql_speed[True-backward] 17.9277ms 16.9172ms 59.1114 Ops/s 55.5831 Ops/s $\textbf{\color{#35bf28}+6.35\%}$
test_iql_speed[reduce-overhead-None] 10.9376ms 8.7687ms 114.0417 Ops/s 112.3536 Ops/s $\color{#35bf28}+1.50\%$
test_iql_speed[reduce-overhead-backward] 18.2628ms 17.4241ms 57.3918 Ops/s 55.7589 Ops/s $\color{#35bf28}+2.93\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 8.2014ms 6.0923ms 164.1428 Ops/s 162.2256 Ops/s $\color{#35bf28}+1.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5449ms 0.2905ms 3.4425 KOps/s 3.4372 KOps/s $\color{#35bf28}+0.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5510ms 0.2716ms 3.6816 KOps/s 3.6398 KOps/s $\color{#35bf28}+1.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.1157ms 5.8336ms 171.4207 Ops/s 170.4581 Ops/s $\color{#35bf28}+0.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6351s 0.8923ms 1.1207 KOps/s 2.9661 KOps/s $\textbf{\color{#d91a1a}-62.22\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6616ms 0.3676ms 2.7203 KOps/s 2.8215 KOps/s $\color{#d91a1a}-3.59\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5551ms 1.3138ms 761.1666 Ops/s 690.9355 Ops/s $\textbf{\color{#35bf28}+10.16\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4670ms 1.2403ms 806.2677 Ops/s 737.0634 Ops/s $\textbf{\color{#35bf28}+9.39\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.1165ms 5.9845ms 167.0970 Ops/s 164.8229 Ops/s $\color{#35bf28}+1.38\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.0080ms 0.5057ms 1.9776 KOps/s 1.9166 KOps/s $\color{#35bf28}+3.18\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8498ms 0.4892ms 2.0443 KOps/s 1.9875 KOps/s $\color{#35bf28}+2.86\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.0253ms 5.8570ms 170.7348 Ops/s 169.7821 Ops/s $\color{#35bf28}+0.56\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8743ms 0.3651ms 2.7392 KOps/s 2.6990 KOps/s $\color{#35bf28}+1.49\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4957ms 0.3092ms 3.2336 KOps/s 2.8460 KOps/s $\textbf{\color{#35bf28}+13.62\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0988ms 5.7757ms 173.1384 Ops/s 171.8281 Ops/s $\color{#35bf28}+0.76\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7059ms 0.3578ms 2.7952 KOps/s 2.7531 KOps/s $\color{#35bf28}+1.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5583ms 0.3458ms 2.8921 KOps/s 2.8811 KOps/s $\color{#35bf28}+0.38\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.3380ms 6.0841ms 164.3637 Ops/s 165.5009 Ops/s $\color{#d91a1a}-0.69\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.3298ms 0.5347ms 1.8703 KOps/s 1.9978 KOps/s $\textbf{\color{#d91a1a}-6.38\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6611ms 0.4679ms 2.1370 KOps/s 2.0657 KOps/s $\color{#35bf28}+3.45\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.6097s 17.0826ms 58.5391 Ops/s 50.9974 Ops/s $\textbf{\color{#35bf28}+14.79\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.7955ms 2.1213ms 471.4076 Ops/s 455.5881 Ops/s $\color{#35bf28}+3.47\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.2113ms 1.2128ms 824.5475 Ops/s 883.6763 Ops/s $\textbf{\color{#d91a1a}-6.69\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.5729ms 4.9607ms 201.5858 Ops/s 195.8567 Ops/s $\color{#35bf28}+2.93\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.4846ms 2.0986ms 476.5144 Ops/s 488.4437 Ops/s $\color{#d91a1a}-2.44\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.6688ms 1.2503ms 799.8217 Ops/s 781.3272 Ops/s $\color{#35bf28}+2.37\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 6.6008ms 5.1530ms 194.0620 Ops/s 189.1505 Ops/s $\color{#35bf28}+2.60\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 13.1612ms 2.1780ms 459.1463 Ops/s 438.1114 Ops/s $\color{#35bf28}+4.80\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.2479ms 1.2604ms 793.4108 Ops/s 723.0572 Ops/s $\textbf{\color{#35bf28}+9.73\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 37.7899ms 34.8473ms 28.6966 Ops/s 28.3700 Ops/s $\color{#35bf28}+1.15\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.3079ms 17.7806ms 56.2412 Ops/s 56.0159 Ops/s $\color{#35bf28}+0.40\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 38.3503ms 36.0564ms 27.7343 Ops/s 26.9002 Ops/s $\color{#35bf28}+3.10\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.6194ms 18.0152ms 55.5086 Ops/s 54.7648 Ops/s $\color{#35bf28}+1.36\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 39.7422ms 38.1146ms 26.2367 Ops/s 25.5304 Ops/s $\color{#35bf28}+2.77\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.9771ms 19.6006ms 51.0190 Ops/s 50.7064 Ops/s $\color{#35bf28}+0.62\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens