Skip to content

[BugFix] Fix CUDA sync in forked subprocess #3269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vmoens merged 2 commits into from
Dec 31, 2025
Merged

[BugFix] Fix CUDA sync in forked subprocess #3269

vmoens merged 2 commits into from
Dec 31, 2025

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Dec 31, 2025 •
edited
Loading

Stack from ghstack (oldest at bottom):

@vmoens
Update
4596935 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Dec 31, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3269

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 31, 2025
This was referenced Dec 31, 2025
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
@vmoens
Update
2da3715 
[ghstack-poisoned]
This was referenced Dec 31, 2025
Merged
Merged
Merged
vmoens added a commit that referenced this pull request Dec 31, 2025
@vmoens
[BugFix] Fix CUDA sync in forked subprocess
4a2a274 
ghstack-source-id: aae7978
Pull-Request: #3269
@vmoens vmoens merged commit 2da3715 into gh/vmoens/169/base Dec 31, 2025
78 of 96 checks passed
@vmoens vmoens deleted the gh/vmoens/169/head branch December 31, 2025 09:17
@github-actions
Copy link

github-actions bot commented Dec 31, 2025

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 164. Improved: $\large\color{#35bf28}23$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 82.9600μs 81.5807μs 12.2578 KOps/s 12.2288 KOps/s $\color{#35bf28}+0.24\%$
test_tensor_to_bytestream_speed[torch.save] 0.1428ms 0.1421ms 7.0349 KOps/s 6.9617 KOps/s $\color{#35bf28}+1.05\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1275s 0.1274s 7.8484 Ops/s 8.4161 Ops/s $\textbf{\color{#d91a1a}-6.75\%}$
test_tensor_to_bytestream_speed[numpy] 2.6770μs 2.6735μs 374.0427 KOps/s 374.9000 KOps/s $\color{#d91a1a}-0.23\%$
test_tensor_to_bytestream_speed[safetensors] 39.5672μs 39.4447μs 25.3520 KOps/s 24.2043 KOps/s $\color{#35bf28}+4.74\%$
test_simple 0.5594s 0.5587s 1.7900 Ops/s 1.7184 Ops/s $\color{#35bf28}+4.17\%$
test_transformed 1.1410s 1.1388s 0.8781 Ops/s 0.8569 Ops/s $\color{#35bf28}+2.47\%$
test_serial 1.6881s 1.6861s 0.5931 Ops/s 0.5832 Ops/s $\color{#35bf28}+1.70\%$
test_parallel 1.3003s 1.1421s 0.8756 Ops/s 0.8730 Ops/s $\color{#35bf28}+0.30\%$
test_step_mdp_speed[True-True-True-True-True] 0.1552ms 45.0571μs 22.1941 KOps/s 21.9569 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[True-True-True-True-False] 0.1313ms 25.8987μs 38.6119 KOps/s 39.1170 KOps/s $\color{#d91a1a}-1.29\%$
test_step_mdp_speed[True-True-True-False-True] 57.2810μs 25.8343μs 38.7082 KOps/s 39.6934 KOps/s $\color{#d91a1a}-2.48\%$
test_step_mdp_speed[True-True-True-False-False] 37.5610μs 14.5641μs 68.6618 KOps/s 71.2732 KOps/s $\color{#d91a1a}-3.66\%$
test_step_mdp_speed[True-True-False-True-True] 82.5010μs 48.9631μs 20.4235 KOps/s 20.5314 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-True-False-True-False] 58.0510μs 28.1946μs 35.4678 KOps/s 35.7656 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[True-True-False-False-True] 61.0510μs 29.1871μs 34.2617 KOps/s 35.1286 KOps/s $\color{#d91a1a}-2.47\%$
test_step_mdp_speed[True-True-False-False-False] 48.7010μs 17.4286μs 57.3769 KOps/s 58.9057 KOps/s $\color{#d91a1a}-2.60\%$
test_step_mdp_speed[True-False-True-True-True] 97.9510μs 51.8873μs 19.2725 KOps/s 19.6567 KOps/s $\color{#d91a1a}-1.95\%$
test_step_mdp_speed[True-False-True-True-False] 63.0210μs 31.2909μs 31.9582 KOps/s 32.1818 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[True-False-True-False-True] 62.2310μs 29.0046μs 34.4773 KOps/s 35.5036 KOps/s $\color{#d91a1a}-2.89\%$
test_step_mdp_speed[True-False-True-False-False] 46.8810μs 17.4381μs 57.3457 KOps/s 59.3915 KOps/s $\color{#d91a1a}-3.44\%$
test_step_mdp_speed[True-False-False-True-True] 97.0410μs 54.6860μs 18.2862 KOps/s 18.9811 KOps/s $\color{#d91a1a}-3.66\%$
test_step_mdp_speed[True-False-False-True-False] 75.5120μs 33.8229μs 29.5658 KOps/s 29.7005 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-False-False-False-True] 71.5610μs 31.0670μs 32.1885 KOps/s 33.0978 KOps/s $\color{#d91a1a}-2.75\%$
test_step_mdp_speed[True-False-False-False-False] 48.2210μs 20.1672μs 49.5854 KOps/s 52.0002 KOps/s $\color{#d91a1a}-4.64\%$
test_step_mdp_speed[False-True-True-True-True] 0.1060ms 52.5599μs 19.0259 KOps/s 19.8532 KOps/s $\color{#d91a1a}-4.17\%$
test_step_mdp_speed[False-True-True-True-False] 72.8810μs 31.0777μs 32.1774 KOps/s 32.5129 KOps/s $\color{#d91a1a}-1.03\%$
test_step_mdp_speed[False-True-True-False-True] 2.2937ms 33.2617μs 30.0646 KOps/s 31.3801 KOps/s $\color{#d91a1a}-4.19\%$
test_step_mdp_speed[False-True-True-False-False] 68.9210μs 19.2289μs 52.0051 KOps/s 54.6675 KOps/s $\color{#d91a1a}-4.87\%$
test_step_mdp_speed[False-True-False-True-True] 94.5720μs 54.9755μs 18.1899 KOps/s 18.8529 KOps/s $\color{#d91a1a}-3.52\%$
test_step_mdp_speed[False-True-False-True-False] 66.1710μs 34.0563μs 29.3632 KOps/s 29.7753 KOps/s $\color{#d91a1a}-1.38\%$
test_step_mdp_speed[False-True-False-False-True] 82.2810μs 35.3940μs 28.2534 KOps/s 29.0646 KOps/s $\color{#d91a1a}-2.79\%$
test_step_mdp_speed[False-True-False-False-False] 49.9510μs 21.9885μs 45.4783 KOps/s 47.0422 KOps/s $\color{#d91a1a}-3.32\%$
test_step_mdp_speed[False-False-True-True-True] 0.1325ms 57.2533μs 17.4662 KOps/s 17.7385 KOps/s $\color{#d91a1a}-1.54\%$
test_step_mdp_speed[False-False-True-True-False] 69.4410μs 37.2672μs 26.8332 KOps/s 27.5512 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[False-False-True-False-True] 99.0920μs 35.2114μs 28.3999 KOps/s 29.0426 KOps/s $\color{#d91a1a}-2.21\%$
test_step_mdp_speed[False-False-True-False-False] 50.0910μs 22.2333μs 44.9776 KOps/s 46.7972 KOps/s $\color{#d91a1a}-3.89\%$
test_step_mdp_speed[False-False-False-True-True] 0.1095ms 60.0503μs 16.6527 KOps/s 17.0182 KOps/s $\color{#d91a1a}-2.15\%$
test_step_mdp_speed[False-False-False-True-False] 0.1022ms 39.7170μs 25.1781 KOps/s 25.9510 KOps/s $\color{#d91a1a}-2.98\%$
test_step_mdp_speed[False-False-False-False-True] 70.7710μs 38.0648μs 26.2710 KOps/s 27.2417 KOps/s $\color{#d91a1a}-3.56\%$
test_step_mdp_speed[False-False-False-False-False] 60.3410μs 24.6041μs 40.6436 KOps/s 42.7082 KOps/s $\color{#d91a1a}-4.83\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.8831s 0.7847s 1.2744 Ops/s 1.2707 Ops/s $\color{#35bf28}+0.29\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7450s 0.6486s 1.5417 Ops/s 1.5349 Ops/s $\color{#35bf28}+0.44\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7921s 1.7121s 0.5841 Ops/s 0.5823 Ops/s $\color{#35bf28}+0.31\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.5601s 1.4821s 0.6747 Ops/s 0.6708 Ops/s $\color{#35bf28}+0.58\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 2.0406s 1.9564s 0.5112 Ops/s 0.5096 Ops/s $\color{#35bf28}+0.30\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.8145s 1.7323s 0.5773 Ops/s 0.5733 Ops/s $\color{#35bf28}+0.69\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.7977s 4.6777s 0.2138 Ops/s 0.2099 Ops/s $\color{#35bf28}+1.86\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.6747s 4.5366s 0.2204 Ops/s 0.2219 Ops/s $\color{#d91a1a}-0.66\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 2.0704s 1.9817s 0.5046 Ops/s 0.4864 Ops/s $\color{#35bf28}+3.75\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.7959s 1.7011s 0.5879 Ops/s 0.5773 Ops/s $\color{#35bf28}+1.83\%$
test_values[generalized_advantage_estimate-True-True] 11.1607ms 10.6735ms 93.6902 Ops/s 96.4921 Ops/s $\color{#d91a1a}-2.90\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.9009ms 17.7181ms 56.4394 Ops/s 55.8913 Ops/s $\color{#35bf28}+0.98\%$
test_values[td0_return_estimate-False-False] 3.6662ms 0.1997ms 5.0080 KOps/s 7.7945 KOps/s $\textbf{\color{#d91a1a}-35.75\%}$
test_values[td1_return_estimate-False-False] 28.5456ms 28.2359ms 35.4159 Ops/s 35.4681 Ops/s $\color{#d91a1a}-0.15\%$
test_values[vec_td1_return_estimate-False-False] 21.5598ms 17.9759ms 55.6299 Ops/s 55.8595 Ops/s $\color{#d91a1a}-0.41\%$
test_values[td_lambda_return_estimate-True-False] 44.6508ms 41.4898ms 24.1023 Ops/s 23.5944 Ops/s $\color{#35bf28}+2.15\%$
test_values[vec_td_lambda_return_estimate-True-False] 21.9586ms 18.0294ms 55.4648 Ops/s 55.9345 Ops/s $\color{#d91a1a}-0.84\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.7543ms 9.5616ms 104.5854 Ops/s 107.9471 Ops/s $\color{#d91a1a}-3.11\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.9712ms 1.5311ms 653.1179 Ops/s 663.2621 Ops/s $\color{#d91a1a}-1.53\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6196ms 0.4199ms 2.3816 KOps/s 2.3421 KOps/s $\color{#35bf28}+1.68\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 35.0342ms 34.4708ms 29.0101 Ops/s 28.2850 Ops/s $\color{#35bf28}+2.56\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.1319ms 1.8136ms 551.3991 Ops/s 543.9403 Ops/s $\color{#35bf28}+1.37\%$
test_dqn_speed[False-None] 1.7663ms 1.4435ms 692.7565 Ops/s 706.1464 Ops/s $\color{#d91a1a}-1.90\%$
test_dqn_speed[False-backward] 2.0994ms 1.9404ms 515.3445 Ops/s 522.1193 Ops/s $\color{#d91a1a}-1.30\%$
test_dqn_speed[True-None] 0.6934ms 0.5517ms 1.8124 KOps/s 1.8569 KOps/s $\color{#d91a1a}-2.40\%$
test_dqn_speed[True-backward] 1.1094ms 0.9730ms 1.0277 KOps/s 847.2672 Ops/s $\textbf{\color{#35bf28}+21.30\%}$
test_dqn_speed[reduce-overhead-None] 0.9252ms 0.5331ms 1.8759 KOps/s 1.7913 KOps/s $\color{#35bf28}+4.72\%$
test_dqn_speed[reduce-overhead-backward] 1.0657ms 0.9656ms 1.0356 KOps/s 899.6286 Ops/s $\textbf{\color{#35bf28}+15.11\%}$
test_ddpg_speed[False-None] 3.2556ms 2.9031ms 344.4645 Ops/s 339.8559 Ops/s $\color{#35bf28}+1.36\%$
test_ddpg_speed[False-backward] 4.3787ms 4.1843ms 238.9868 Ops/s 239.3225 Ops/s $\color{#d91a1a}-0.14\%$
test_ddpg_speed[True-None] 1.7718ms 1.3953ms 716.6840 Ops/s 695.8565 Ops/s $\color{#35bf28}+2.99\%$
test_ddpg_speed[True-backward] 2.7473ms 2.3720ms 421.5940 Ops/s 371.8922 Ops/s $\textbf{\color{#35bf28}+13.36\%}$
test_ddpg_speed[reduce-overhead-None] 1.7757ms 1.3822ms 723.4918 Ops/s 717.8903 Ops/s $\color{#35bf28}+0.78\%$
test_ddpg_speed[reduce-overhead-backward] 2.7009ms 2.3526ms 425.0700 Ops/s 360.4926 Ops/s $\textbf{\color{#35bf28}+17.91\%}$
test_sac_speed[False-None] 8.4237ms 7.9676ms 125.5085 Ops/s 123.1373 Ops/s $\color{#35bf28}+1.93\%$
test_sac_speed[False-backward] 11.6763ms 11.2574ms 88.8302 Ops/s 87.4145 Ops/s $\color{#35bf28}+1.62\%$
test_sac_speed[True-None] 3.5921ms 2.1735ms 460.0971 Ops/s 456.3981 Ops/s $\color{#35bf28}+0.81\%$
test_sac_speed[True-backward] 4.0993ms 3.9159ms 255.3683 Ops/s 231.9351 Ops/s $\textbf{\color{#35bf28}+10.10\%}$
test_sac_speed[reduce-overhead-None] 2.3317ms 2.1085ms 474.2697 Ops/s 465.5475 Ops/s $\color{#35bf28}+1.87\%$
test_sac_speed[reduce-overhead-backward] 4.0190ms 3.9170ms 255.2998 Ops/s 215.1847 Ops/s $\textbf{\color{#35bf28}+18.64\%}$
test_redq_speed[False-None] 14.9737ms 10.5349ms 94.9226 Ops/s 94.8166 Ops/s $\color{#35bf28}+0.11\%$
test_redq_speed[False-backward] 22.8395ms 17.9775ms 55.6250 Ops/s 55.1583 Ops/s $\color{#35bf28}+0.85\%$
test_redq_speed[True-None] 4.5257ms 4.2465ms 235.4885 Ops/s 227.2326 Ops/s $\color{#35bf28}+3.63\%$
test_redq_speed[True-backward] 9.7155ms 9.3949ms 106.4410 Ops/s 101.1023 Ops/s $\textbf{\color{#35bf28}+5.28\%}$
test_redq_speed[reduce-overhead-None] 4.4105ms 4.2138ms 237.3149 Ops/s 227.1550 Ops/s $\color{#35bf28}+4.47\%$
test_redq_speed[reduce-overhead-backward] 9.7669ms 9.5394ms 104.8283 Ops/s 103.5275 Ops/s $\color{#35bf28}+1.26\%$
test_redq_deprec_speed[False-None] 11.4895ms 11.0791ms 90.2602 Ops/s 88.9044 Ops/s $\color{#35bf28}+1.52\%$
test_redq_deprec_speed[False-backward] 16.5447ms 15.9547ms 62.6775 Ops/s 62.6898 Ops/s $\color{#d91a1a}-0.02\%$
test_redq_deprec_speed[True-None] 4.0511ms 3.6698ms 272.4965 Ops/s 266.1821 Ops/s $\color{#35bf28}+2.37\%$
test_redq_deprec_speed[True-backward] 8.0368ms 7.6533ms 130.6633 Ops/s 130.0093 Ops/s $\color{#35bf28}+0.50\%$
test_redq_deprec_speed[reduce-overhead-None] 3.9635ms 3.5887ms 278.6486 Ops/s 266.1329 Ops/s $\color{#35bf28}+4.70\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.9017ms 7.6413ms 130.8685 Ops/s 114.1697 Ops/s $\textbf{\color{#35bf28}+14.63\%}$
test_td3_speed[False-None] 8.1396ms 8.0394ms 124.3881 Ops/s 123.5958 Ops/s $\color{#35bf28}+0.64\%$
test_td3_speed[False-backward] 11.4109ms 10.9294ms 91.4967 Ops/s 91.8694 Ops/s $\color{#d91a1a}-0.41\%$
test_td3_speed[True-None] 1.8475ms 1.8004ms 555.4407 Ops/s 551.1443 Ops/s $\color{#35bf28}+0.78\%$
test_td3_speed[True-backward] 3.6936ms 3.5586ms 281.0112 Ops/s 256.1351 Ops/s $\textbf{\color{#35bf28}+9.71\%}$
test_td3_speed[reduce-overhead-None] 1.8284ms 1.7862ms 559.8604 Ops/s 561.6152 Ops/s $\color{#d91a1a}-0.31\%$
test_td3_speed[reduce-overhead-backward] 3.7730ms 3.6032ms 277.5321 Ops/s 274.0959 Ops/s $\color{#35bf28}+1.25\%$
test_cql_speed[False-None] 28.5915ms 25.8926ms 38.6211 Ops/s 38.9695 Ops/s $\color{#d91a1a}-0.89\%$
test_cql_speed[False-backward] 35.8252ms 34.9792ms 28.5884 Ops/s 28.7122 Ops/s $\color{#d91a1a}-0.43\%$
test_cql_speed[True-None] 12.5478ms 12.2041ms 81.9398 Ops/s 76.9777 Ops/s $\textbf{\color{#35bf28}+6.45\%}$
test_cql_speed[True-backward] 18.1602ms 17.8046ms 56.1653 Ops/s 56.0320 Ops/s $\color{#35bf28}+0.24\%$
test_cql_speed[reduce-overhead-None] 12.6167ms 12.2421ms 81.6855 Ops/s 80.1235 Ops/s $\color{#35bf28}+1.95\%$
test_cql_speed[reduce-overhead-backward] 18.4168ms 17.9835ms 55.6065 Ops/s 56.8270 Ops/s $\color{#d91a1a}-2.15\%$
test_a2c_speed[False-None] 5.7128ms 5.3375ms 187.3546 Ops/s 183.5946 Ops/s $\color{#35bf28}+2.05\%$
test_a2c_speed[False-backward] 12.0893ms 11.8454ms 84.4212 Ops/s 84.2996 Ops/s $\color{#35bf28}+0.14\%$
test_a2c_speed[True-None] 3.8118ms 3.6463ms 274.2543 Ops/s 266.9231 Ops/s $\color{#35bf28}+2.75\%$
test_a2c_speed[True-backward] 8.7417ms 8.4859ms 117.8421 Ops/s 116.2380 Ops/s $\color{#35bf28}+1.38\%$
test_a2c_speed[reduce-overhead-None] 4.0132ms 3.6912ms 270.9135 Ops/s 269.9056 Ops/s $\color{#35bf28}+0.37\%$
test_a2c_speed[reduce-overhead-backward] 9.6298ms 8.6881ms 115.1004 Ops/s 111.4911 Ops/s $\color{#35bf28}+3.24\%$
test_ppo_speed[False-None] 6.4464ms 5.8881ms 169.8346 Ops/s 169.7147 Ops/s $\color{#35bf28}+0.07\%$
test_ppo_speed[False-backward] 12.7435ms 12.3046ms 81.2707 Ops/s 79.9628 Ops/s $\color{#35bf28}+1.64\%$
test_ppo_speed[True-None] 4.0168ms 3.6019ms 277.6339 Ops/s 291.7044 Ops/s $\color{#d91a1a}-4.82\%$
test_ppo_speed[True-backward] 8.6202ms 8.3227ms 120.1539 Ops/s 123.2952 Ops/s $\color{#d91a1a}-2.55\%$
test_ppo_speed[reduce-overhead-None] 3.9999ms 3.5386ms 282.6002 Ops/s 299.5384 Ops/s $\textbf{\color{#d91a1a}-5.65\%}$
test_ppo_speed[reduce-overhead-backward] 9.0124ms 8.5973ms 116.3152 Ops/s 119.6108 Ops/s $\color{#d91a1a}-2.76\%$
test_reinforce_speed[False-None] 4.7466ms 4.4781ms 223.3073 Ops/s 233.8295 Ops/s $\color{#d91a1a}-4.50\%$
test_reinforce_speed[False-backward] 7.6679ms 7.3360ms 136.3148 Ops/s 140.0910 Ops/s $\color{#d91a1a}-2.70\%$
test_reinforce_speed[True-None] 3.2749ms 2.8510ms 350.7562 Ops/s 382.9942 Ops/s $\textbf{\color{#d91a1a}-8.42\%}$
test_reinforce_speed[True-backward] 7.8983ms 7.5772ms 131.9744 Ops/s 130.1005 Ops/s $\color{#35bf28}+1.44\%$
test_reinforce_speed[reduce-overhead-None] 3.0795ms 2.8393ms 352.1984 Ops/s 350.9324 Ops/s $\color{#35bf28}+0.36\%$
test_reinforce_speed[reduce-overhead-backward] 8.1316ms 7.8970ms 126.6303 Ops/s 120.3207 Ops/s $\textbf{\color{#35bf28}+5.24\%}$
test_iql_speed[False-None] 20.3674ms 19.8489ms 50.3807 Ops/s 51.4799 Ops/s $\color{#d91a1a}-2.14\%$
test_iql_speed[False-backward] 31.1720ms 30.2813ms 33.0237 Ops/s 33.6423 Ops/s $\color{#d91a1a}-1.84\%$
test_iql_speed[True-None] 8.8187ms 8.4130ms 118.8643 Ops/s 117.3077 Ops/s $\color{#35bf28}+1.33\%$
test_iql_speed[True-backward] 16.7584ms 16.3647ms 61.1071 Ops/s 60.9895 Ops/s $\color{#35bf28}+0.19\%$
test_iql_speed[reduce-overhead-None] 8.9547ms 8.4729ms 118.0234 Ops/s 117.2019 Ops/s $\color{#35bf28}+0.70\%$
test_iql_speed[reduce-overhead-backward] 17.6292ms 16.7576ms 59.6744 Ops/s 59.0478 Ops/s $\color{#35bf28}+1.06\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 8.1758ms 6.1898ms 161.5568 Ops/s 163.1394 Ops/s $\color{#d91a1a}-0.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5728ms 0.3358ms 2.9784 KOps/s 2.9037 KOps/s $\color{#35bf28}+2.57\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6387ms 0.2814ms 3.5533 KOps/s 3.0261 KOps/s $\textbf{\color{#35bf28}+17.42\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.1983ms 5.8654ms 170.4915 Ops/s 172.9817 Ops/s $\color{#d91a1a}-1.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.2597ms 0.3063ms 3.2643 KOps/s 2.7937 KOps/s $\textbf{\color{#35bf28}+16.85\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5511ms 0.2849ms 3.5098 KOps/s 3.1164 KOps/s $\textbf{\color{#35bf28}+12.62\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7447ms 1.2760ms 783.6889 Ops/s 789.0783 Ops/s $\color{#d91a1a}-0.68\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4471ms 1.1889ms 841.1456 Ops/s 835.0741 Ops/s $\color{#35bf28}+0.73\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 10.1016ms 6.1967ms 161.3768 Ops/s 165.6673 Ops/s $\color{#d91a1a}-2.59\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7992ms 0.4378ms 2.2843 KOps/s 2.1190 KOps/s $\textbf{\color{#35bf28}+7.80\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6781ms 0.4402ms 2.2715 KOps/s 2.2181 KOps/s $\color{#35bf28}+2.41\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.0501ms 5.9110ms 169.1749 Ops/s 170.7182 Ops/s $\color{#d91a1a}-0.90\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.6707ms 0.2856ms 3.5016 KOps/s 3.1279 KOps/s $\textbf{\color{#35bf28}+11.95\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5673ms 0.2639ms 3.7891 KOps/s 3.2162 KOps/s $\textbf{\color{#35bf28}+17.81\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0849ms 5.8607ms 170.6271 Ops/s 172.5847 Ops/s $\color{#d91a1a}-1.13\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0986ms 0.2802ms 3.5694 KOps/s 639.2559 Ops/s $\textbf{\color{#35bf28}+458.36\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6919ms 0.2683ms 3.7271 KOps/s 3.0691 KOps/s $\textbf{\color{#35bf28}+21.44\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.1718ms 6.0480ms 165.3440 Ops/s 163.3198 Ops/s $\color{#35bf28}+1.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9403ms 0.4368ms 2.2896 KOps/s 2.0712 KOps/s $\textbf{\color{#35bf28}+10.55\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6953ms 0.4240ms 2.3588 KOps/s 2.2057 KOps/s $\textbf{\color{#35bf28}+6.94\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.5873ms 5.1392ms 194.5831 Ops/s 194.7720 Ops/s $\color{#d91a1a}-0.10\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 10.6084ms 2.3657ms 422.6999 Ops/s 484.3612 Ops/s $\textbf{\color{#d91a1a}-12.73\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.7005ms 1.1428ms 875.0071 Ops/s 848.2247 Ops/s $\color{#35bf28}+3.16\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.6764ms 5.1840ms 192.9014 Ops/s 196.4139 Ops/s $\color{#d91a1a}-1.79\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 6.0109ms 2.3319ms 428.8378 Ops/s 476.1597 Ops/s $\textbf{\color{#d91a1a}-9.94\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 9.3440ms 1.3236ms 755.5270 Ops/s 914.4292 Ops/s $\textbf{\color{#d91a1a}-17.38\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.6176s 17.7101ms 56.4648 Ops/s 53.4709 Ops/s $\textbf{\color{#35bf28}+5.60\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 9.4124ms 2.1648ms 461.9419 Ops/s 551.4373 Ops/s $\textbf{\color{#d91a1a}-16.23\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.2886ms 1.0281ms 972.6813 Ops/s 974.6263 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 36.0362ms 34.1641ms 29.2705 Ops/s 28.6632 Ops/s $\color{#35bf28}+2.12\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 18.9325ms 17.5709ms 56.9123 Ops/s 55.1847 Ops/s $\color{#35bf28}+3.13\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 37.0964ms 35.0279ms 28.5487 Ops/s 27.0119 Ops/s $\textbf{\color{#35bf28}+5.69\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.4062ms 17.9276ms 55.7799 Ops/s 53.7593 Ops/s $\color{#35bf28}+3.76\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 39.9388ms 37.4315ms 26.7155 Ops/s 25.7287 Ops/s $\color{#35bf28}+3.84\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 22.6060ms 19.6284ms 50.9466 Ops/s 49.3422 Ops/s $\color{#35bf28}+3.25\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens