Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix init_random_frames=0 #2645

Merged
merged 1 commit into from
Dec 12, 2024
Merged

[BugFix] Fix init_random_frames=0 #2645

merged 1 commit into from
Dec 12, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Dec 11, 2024 •
edited
Loading

Stack from ghstack (oldest at bottom):

@vmoens
Update
e3107b5 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Dec 11, 2024
@vmoens
[BugFix] Fix init_random_frames=0
61684e1 
ghstack-source-id: 38a544ea15631f9affb4c385c09e7c4df94af55d
Pull Request resolved: #2645
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Dec 11, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2645

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 9 Unrelated Failures

As of commit e3107b5 with merge base b840a77 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 11, 2024
@vmoens vmoens linked an issue Dec 11, 2024 that may be closed by this pull request
[BUG] SyncDataCollector Crashes when init_random_frames=0 with a policy that is NOT random #2534
Open
3 tasks
@vmoens vmoens added the bug Something isn't working label Dec 11, 2024
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}55$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.4542s 0.4497s 2.2236 Ops/s 2.0227 Ops/s $\textbf{\color{#35bf28}+9.93\%}$
test_transformed 0.7911s 0.6693s 1.4941 Ops/s 1.4811 Ops/s $\color{#35bf28}+0.88\%$
test_serial 1.4137s 1.4062s 0.7111 Ops/s 0.6775 Ops/s $\color{#35bf28}+4.96\%$
test_parallel 1.3723s 1.3571s 0.7368 Ops/s 0.7085 Ops/s $\color{#35bf28}+4.01\%$
test_step_mdp_speed[True-True-True-True-True] 0.3261ms 30.3111μs 32.9912 KOps/s 33.3386 KOps/s $\color{#d91a1a}-1.04\%$
test_step_mdp_speed[True-True-True-True-False] 59.0510μs 17.7333μs 56.3911 KOps/s 56.4491 KOps/s $\color{#d91a1a}-0.10\%$
test_step_mdp_speed[True-True-True-False-True] 92.2030μs 17.1266μs 58.3886 KOps/s 58.6228 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[True-True-True-False-False] 46.6480μs 10.1951μs 98.0861 KOps/s 98.8772 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[True-True-False-True-True] 0.1190ms 32.4786μs 30.7895 KOps/s 31.0061 KOps/s $\color{#d91a1a}-0.70\%$
test_step_mdp_speed[True-True-False-True-False] 54.3620μs 19.2691μs 51.8965 KOps/s 50.5269 KOps/s $\color{#35bf28}+2.71\%$
test_step_mdp_speed[True-True-False-False-True] 81.4320μs 18.6541μs 53.6074 KOps/s 52.9569 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[True-True-False-False-False] 46.6880μs 11.8224μs 84.5849 KOps/s 83.2939 KOps/s $\color{#35bf28}+1.55\%$
test_step_mdp_speed[True-False-True-True-True] 0.1021ms 34.0447μs 29.3731 KOps/s 29.1313 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[True-False-True-True-False] 67.5170μs 21.4568μs 46.6054 KOps/s 46.8853 KOps/s $\color{#d91a1a}-0.60\%$
test_step_mdp_speed[True-False-True-False-True] 85.0290μs 19.0649μs 52.4523 KOps/s 53.7294 KOps/s $\color{#d91a1a}-2.38\%$
test_step_mdp_speed[True-False-True-False-False] 53.7210μs 11.8718μs 84.2330 KOps/s 85.5151 KOps/s $\color{#d91a1a}-1.50\%$
test_step_mdp_speed[True-False-False-True-True] 0.1292ms 35.8458μs 27.8973 KOps/s 28.6640 KOps/s $\color{#d91a1a}-2.67\%$
test_step_mdp_speed[True-False-False-True-False] 67.3160μs 22.9318μs 43.6076 KOps/s 43.7145 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[True-False-False-False-True] 94.2670μs 20.3528μs 49.1334 KOps/s 50.0742 KOps/s $\color{#d91a1a}-1.88\%$
test_step_mdp_speed[True-False-False-False-False] 49.2520μs 13.7802μs 72.5677 KOps/s 73.9448 KOps/s $\color{#d91a1a}-1.86\%$
test_step_mdp_speed[False-True-True-True-True] 0.1204ms 33.7578μs 29.6228 KOps/s 29.5147 KOps/s $\color{#35bf28}+0.37\%$
test_step_mdp_speed[False-True-True-True-False] 57.5380μs 21.2701μs 47.0142 KOps/s 46.1679 KOps/s $\color{#35bf28}+1.83\%$
test_step_mdp_speed[False-True-True-False-True] 93.5550μs 21.1967μs 47.1771 KOps/s 46.8604 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[False-True-True-False-False] 52.1480μs 13.0219μs 76.7937 KOps/s 76.5844 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-True-False-True-True] 0.1069ms 35.2597μs 28.3610 KOps/s 28.6301 KOps/s $\color{#d91a1a}-0.94\%$
test_step_mdp_speed[False-True-False-True-False] 63.4180μs 22.9268μs 43.6170 KOps/s 44.0397 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[False-True-False-False-True] 3.0597ms 23.0594μs 43.3662 KOps/s 42.8081 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[False-True-False-False-False] 51.2260μs 14.9102μs 67.0682 KOps/s 67.6617 KOps/s $\color{#d91a1a}-0.88\%$
test_step_mdp_speed[False-False-True-True-True] 0.1094ms 37.4411μs 26.7086 KOps/s 26.7174 KOps/s $\color{#d91a1a}-0.03\%$
test_step_mdp_speed[False-False-True-True-False] 69.9610μs 24.6358μs 40.5913 KOps/s 39.5969 KOps/s $\color{#35bf28}+2.51\%$
test_step_mdp_speed[False-False-True-False-True] 96.8820μs 23.2037μs 43.0965 KOps/s 42.9785 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-False-True-False-False] 69.0590μs 15.0523μs 66.4350 KOps/s 66.6766 KOps/s $\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-False-False-True-True] 0.1308ms 38.4102μs 26.0348 KOps/s 25.1765 KOps/s $\color{#35bf28}+3.41\%$
test_step_mdp_speed[False-False-False-True-False] 0.1116ms 26.0803μs 38.3430 KOps/s 38.2063 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-False-False-True] 0.1244ms 24.5345μs 40.7589 KOps/s 41.3512 KOps/s $\color{#d91a1a}-1.43\%$
test_step_mdp_speed[False-False-False-False-False] 80.7180μs 16.2440μs 61.5613 KOps/s 61.4495 KOps/s $\color{#35bf28}+0.18\%$
test_values[generalized_advantage_estimate-True-True] 10.4256ms 10.0497ms 99.5056 Ops/s 98.6151 Ops/s $\color{#35bf28}+0.90\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.0123ms 34.8225ms 28.7170 Ops/s 26.4326 Ops/s $\textbf{\color{#35bf28}+8.64\%}$
test_values[td0_return_estimate-False-False] 0.2784ms 0.2202ms 4.5415 KOps/s 4.5065 KOps/s $\color{#35bf28}+0.78\%$
test_values[td1_return_estimate-False-False] 25.2931ms 24.8416ms 40.2550 Ops/s 39.9457 Ops/s $\color{#35bf28}+0.77\%$
test_values[vec_td1_return_estimate-False-False] 37.2326ms 35.1235ms 28.4710 Ops/s 26.0396 Ops/s $\textbf{\color{#35bf28}+9.34\%}$
test_values[td_lambda_return_estimate-True-False] 36.5228ms 35.5752ms 28.1095 Ops/s 27.8593 Ops/s $\color{#35bf28}+0.90\%$
test_values[vec_td_lambda_return_estimate-True-False] 37.9345ms 34.9367ms 28.6232 Ops/s 25.3496 Ops/s $\textbf{\color{#35bf28}+12.91\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.8338ms 8.4263ms 118.6763 Ops/s 115.5843 Ops/s $\color{#35bf28}+2.68\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.6162ms 1.9199ms 520.8513 Ops/s 472.5695 Ops/s $\textbf{\color{#35bf28}+10.22\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5074ms 0.3691ms 2.7096 KOps/s 2.6875 KOps/s $\color{#35bf28}+0.82\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 44.6811ms 42.2745ms 23.6549 Ops/s 20.6134 Ops/s $\textbf{\color{#35bf28}+14.76\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.7827ms 3.0439ms 328.5244 Ops/s 305.0400 Ops/s $\textbf{\color{#35bf28}+7.70\%}$
test_dqn_speed[False-None] 6.1377ms 1.4087ms 709.8701 Ops/s 678.4518 Ops/s $\color{#35bf28}+4.63\%$
test_dqn_speed[False-backward] 1.9838ms 1.8839ms 530.8252 Ops/s 509.0287 Ops/s $\color{#35bf28}+4.28\%$
test_dqn_speed[True-None] 0.6914ms 0.4618ms 2.1654 KOps/s 2.0748 KOps/s $\color{#35bf28}+4.37\%$
test_dqn_speed[True-backward] 0.9488ms 0.9017ms 1.1091 KOps/s 1.0494 KOps/s $\textbf{\color{#35bf28}+5.68\%}$
test_dqn_speed[reduce-overhead-None] 0.7158ms 0.4677ms 2.1383 KOps/s 2.0711 KOps/s $\color{#35bf28}+3.25\%$
test_dqn_speed[reduce-overhead-backward] 0.9482ms 0.9066ms 1.1031 KOps/s 1.0434 KOps/s $\textbf{\color{#35bf28}+5.72\%}$
test_ddpg_speed[False-None] 3.0512ms 2.8610ms 349.5254 Ops/s 314.3374 Ops/s $\textbf{\color{#35bf28}+11.19\%}$
test_ddpg_speed[False-backward] 5.1676ms 4.0279ms 248.2659 Ops/s 218.4734 Ops/s $\textbf{\color{#35bf28}+13.64\%}$
test_ddpg_speed[True-None] 1.2022ms 0.9930ms 1.0070 KOps/s 970.1460 Ops/s $\color{#35bf28}+3.80\%$
test_ddpg_speed[True-backward] 2.0989ms 1.9052ms 524.8849 Ops/s 469.3242 Ops/s $\textbf{\color{#35bf28}+11.84\%}$
test_ddpg_speed[reduce-overhead-None] 1.2327ms 0.9901ms 1.0100 KOps/s 916.3925 Ops/s $\textbf{\color{#35bf28}+10.21\%}$
test_ddpg_speed[reduce-overhead-backward] 1.9725ms 1.8958ms 527.4762 Ops/s 482.7329 Ops/s $\textbf{\color{#35bf28}+9.27\%}$
test_sac_speed[False-None] 9.0557ms 8.0525ms 124.1856 Ops/s 105.9572 Ops/s $\textbf{\color{#35bf28}+17.20\%}$
test_sac_speed[False-backward] 11.2959ms 10.7798ms 92.7660 Ops/s 74.7532 Ops/s $\textbf{\color{#35bf28}+24.10\%}$
test_sac_speed[True-None] 2.0363ms 1.8199ms 549.4892 Ops/s 544.2687 Ops/s $\color{#35bf28}+0.96\%$
test_sac_speed[True-backward] 3.5808ms 3.4898ms 286.5527 Ops/s 286.2179 Ops/s $\color{#35bf28}+0.12\%$
test_sac_speed[reduce-overhead-None] 2.4084ms 1.8203ms 549.3598 Ops/s 530.1212 Ops/s $\color{#35bf28}+3.63\%$
test_sac_speed[reduce-overhead-backward] 3.5751ms 3.4930ms 286.2895 Ops/s 285.6465 Ops/s $\color{#35bf28}+0.23\%$
test_redq_speed[False-None] 14.3431ms 12.7545ms 78.4037 Ops/s 74.2374 Ops/s $\textbf{\color{#35bf28}+5.61\%}$
test_redq_speed[False-backward] 23.9726ms 22.2395ms 44.9650 Ops/s 44.1056 Ops/s $\color{#35bf28}+1.95\%$
test_redq_speed[True-None] 5.1772ms 4.4558ms 224.4259 Ops/s 217.4942 Ops/s $\color{#35bf28}+3.19\%$
test_redq_speed[True-backward] 14.7121ms 13.2397ms 75.5302 Ops/s 82.9821 Ops/s $\textbf{\color{#d91a1a}-8.98\%}$
test_redq_speed[reduce-overhead-None] 5.5618ms 4.5920ms 217.7687 Ops/s 218.1407 Ops/s $\color{#d91a1a}-0.17\%$
test_redq_speed[reduce-overhead-backward] 13.6269ms 11.8244ms 84.5706 Ops/s 82.4057 Ops/s $\color{#35bf28}+2.63\%$
test_redq_deprec_speed[False-None] 13.3256ms 12.5589ms 79.6246 Ops/s 76.9419 Ops/s $\color{#35bf28}+3.49\%$
test_redq_deprec_speed[False-backward] 21.7933ms 18.3333ms 54.5454 Ops/s 53.0966 Ops/s $\color{#35bf28}+2.73\%$
test_redq_deprec_speed[True-None] 4.4685ms 3.6464ms 274.2450 Ops/s 278.2665 Ops/s $\color{#d91a1a}-1.45\%$
test_redq_deprec_speed[True-backward] 8.7883ms 7.9104ms 126.4154 Ops/s 101.9286 Ops/s $\textbf{\color{#35bf28}+24.02\%}$
test_redq_deprec_speed[reduce-overhead-None] 4.2941ms 3.6059ms 277.3238 Ops/s 278.3946 Ops/s $\color{#d91a1a}-0.38\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.0620ms 7.9204ms 126.2568 Ops/s 124.0715 Ops/s $\color{#35bf28}+1.76\%$
test_td3_speed[False-None] 9.7593ms 7.9862ms 125.2154 Ops/s 121.9579 Ops/s $\color{#35bf28}+2.67\%$
test_td3_speed[False-backward] 11.3488ms 10.2935ms 97.1486 Ops/s 94.4358 Ops/s $\color{#35bf28}+2.87\%$
test_td3_speed[True-None] 1.9241ms 1.6993ms 588.4702 Ops/s 569.7880 Ops/s $\color{#35bf28}+3.28\%$
test_td3_speed[True-backward] 3.5168ms 3.3049ms 302.5774 Ops/s 299.0518 Ops/s $\color{#35bf28}+1.18\%$
test_td3_speed[reduce-overhead-None] 1.8929ms 1.6993ms 588.4769 Ops/s 574.1245 Ops/s $\color{#35bf28}+2.50\%$
test_td3_speed[reduce-overhead-backward] 4.0318ms 3.3193ms 301.2728 Ops/s 296.2734 Ops/s $\color{#35bf28}+1.69\%$
test_cql_speed[False-None] 41.9013ms 36.8836ms 27.1123 Ops/s 27.4042 Ops/s $\color{#d91a1a}-1.07\%$
test_cql_speed[False-backward] 48.8005ms 46.5886ms 21.4645 Ops/s 21.1513 Ops/s $\color{#35bf28}+1.48\%$
test_cql_speed[True-None] 16.3547ms 15.4547ms 64.7051 Ops/s 63.7656 Ops/s $\color{#35bf28}+1.47\%$
test_cql_speed[True-backward] 24.3030ms 23.1295ms 43.2348 Ops/s 44.5307 Ops/s $\color{#d91a1a}-2.91\%$
test_cql_speed[reduce-overhead-None] 16.7783ms 15.5403ms 64.3488 Ops/s 64.0636 Ops/s $\color{#35bf28}+0.45\%$
test_cql_speed[reduce-overhead-backward] 22.9715ms 21.9700ms 45.5167 Ops/s 44.7754 Ops/s $\color{#35bf28}+1.66\%$
test_a2c_speed[False-None] 9.0899ms 7.1446ms 139.9657 Ops/s 137.1911 Ops/s $\color{#35bf28}+2.02\%$
test_a2c_speed[False-backward] 15.9016ms 14.0253ms 71.2998 Ops/s 69.2868 Ops/s $\color{#35bf28}+2.91\%$
test_a2c_speed[True-None] 5.0735ms 4.1835ms 239.0353 Ops/s 234.3548 Ops/s $\color{#35bf28}+2.00\%$
test_a2c_speed[True-backward] 11.1818ms 10.6479ms 93.9153 Ops/s 93.4893 Ops/s $\color{#35bf28}+0.46\%$
test_a2c_speed[reduce-overhead-None] 4.6647ms 4.1742ms 239.5658 Ops/s 196.2555 Ops/s $\textbf{\color{#35bf28}+22.07\%}$
test_a2c_speed[reduce-overhead-backward] 12.3952ms 11.8674ms 84.2641 Ops/s 80.0156 Ops/s $\textbf{\color{#35bf28}+5.31\%}$
test_ppo_speed[False-None] 11.8617ms 8.6057ms 116.2027 Ops/s 115.2816 Ops/s $\color{#35bf28}+0.80\%$
test_ppo_speed[False-backward] 17.3529ms 16.2938ms 61.3729 Ops/s 56.3138 Ops/s $\textbf{\color{#35bf28}+8.98\%}$
test_ppo_speed[True-None] 5.3914ms 3.9020ms 256.2762 Ops/s 247.8310 Ops/s $\color{#35bf28}+3.41\%$
test_ppo_speed[True-backward] 9.9200ms 9.5517ms 104.6936 Ops/s 94.0943 Ops/s $\textbf{\color{#35bf28}+11.26\%}$
test_ppo_speed[reduce-overhead-None] 4.2905ms 3.6566ms 273.4745 Ops/s 269.4861 Ops/s $\color{#35bf28}+1.48\%$
test_ppo_speed[reduce-overhead-backward] 9.9062ms 9.5514ms 104.6972 Ops/s 94.8977 Ops/s $\textbf{\color{#35bf28}+10.33\%}$
test_reinforce_speed[False-None] 9.2423ms 8.0806ms 123.7531 Ops/s 138.8836 Ops/s $\textbf{\color{#d91a1a}-10.89\%}$
test_reinforce_speed[False-backward] 10.8408ms 9.7530ms 102.5321 Ops/s 92.1113 Ops/s $\textbf{\color{#35bf28}+11.31\%}$
test_reinforce_speed[True-None] 2.9275ms 2.6066ms 383.6376 Ops/s 303.1559 Ops/s $\textbf{\color{#35bf28}+26.55\%}$
test_reinforce_speed[True-backward] 8.9200ms 8.5260ms 117.2881 Ops/s 99.3483 Ops/s $\textbf{\color{#35bf28}+18.06\%}$
test_reinforce_speed[reduce-overhead-None] 3.1595ms 2.6161ms 382.2541 Ops/s 300.5036 Ops/s $\textbf{\color{#35bf28}+27.20\%}$
test_reinforce_speed[reduce-overhead-backward] 8.8799ms 8.5489ms 116.9745 Ops/s 100.2323 Ops/s $\textbf{\color{#35bf28}+16.70\%}$
test_iql_speed[False-None] 33.3597ms 32.0029ms 31.2472 Ops/s 28.0071 Ops/s $\textbf{\color{#35bf28}+11.57\%}$
test_iql_speed[False-backward] 47.2143ms 45.2045ms 22.1217 Ops/s 20.0986 Ops/s $\textbf{\color{#35bf28}+10.07\%}$
test_iql_speed[True-None] 11.1928ms 10.4517ms 95.6778 Ops/s 84.7004 Ops/s $\textbf{\color{#35bf28}+12.96\%}$
test_iql_speed[True-backward] 22.2031ms 21.1708ms 47.2348 Ops/s 42.0054 Ops/s $\textbf{\color{#35bf28}+12.45\%}$
test_iql_speed[reduce-overhead-None] 12.4831ms 10.4541ms 95.6564 Ops/s 83.2311 Ops/s $\textbf{\color{#35bf28}+14.93\%}$
test_iql_speed[reduce-overhead-backward] 28.0962ms 23.8182ms 41.9846 Ops/s 42.1695 Ops/s $\color{#d91a1a}-0.44\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.7135ms 5.7225ms 174.7501 Ops/s 172.7264 Ops/s $\color{#35bf28}+1.17\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8175ms 0.5548ms 1.8026 KOps/s 1.7925 KOps/s $\color{#35bf28}+0.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8554ms 0.5288ms 1.8910 KOps/s 1.9297 KOps/s $\color{#d91a1a}-2.00\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.5602ms 4.9661ms 201.3659 Ops/s 190.1868 Ops/s $\textbf{\color{#35bf28}+5.88\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0055ms 0.4852ms 2.0611 KOps/s 619.2385 Ops/s $\textbf{\color{#35bf28}+232.85\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.8719ms 0.4797ms 2.0844 KOps/s 1.8874 KOps/s $\textbf{\color{#35bf28}+10.44\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 4.9855ms 1.6726ms 597.8539 Ops/s 580.5473 Ops/s $\color{#35bf28}+2.98\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.1784ms 1.5970ms 626.1812 Ops/s 591.3237 Ops/s $\textbf{\color{#35bf28}+5.89\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.2663ms 4.8332ms 206.9037 Ops/s 159.7381 Ops/s $\textbf{\color{#35bf28}+29.53\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.4705ms 0.6454ms 1.5495 KOps/s 1.4190 KOps/s $\textbf{\color{#35bf28}+9.19\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.2018ms 0.6101ms 1.6391 KOps/s 1.4885 KOps/s $\textbf{\color{#35bf28}+10.11\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.2297ms 4.6211ms 216.3974 Ops/s 168.4798 Ops/s $\textbf{\color{#35bf28}+28.44\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.7656ms 0.5109ms 1.9574 KOps/s 1.7838 KOps/s $\textbf{\color{#35bf28}+9.73\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6687ms 0.4807ms 2.0804 KOps/s 1.8191 KOps/s $\textbf{\color{#35bf28}+14.37\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.4488ms 4.6120ms 216.8261 Ops/s 170.6176 Ops/s $\textbf{\color{#35bf28}+27.08\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6419ms 0.4869ms 2.0536 KOps/s 1.8230 KOps/s $\textbf{\color{#35bf28}+12.65\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 7.7175ms 0.4791ms 2.0870 KOps/s 1.8583 KOps/s $\textbf{\color{#35bf28}+12.31\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.3972ms 4.7732ms 209.5039 Ops/s 160.1511 Ops/s $\textbf{\color{#35bf28}+30.82\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0388ms 0.6358ms 1.5728 KOps/s 1.4210 KOps/s $\textbf{\color{#35bf28}+10.68\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9003ms 0.6196ms 1.6140 KOps/s 1.4780 KOps/s $\textbf{\color{#35bf28}+9.21\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4497s 13.0476ms 76.6425 Ops/s 202.3568 Ops/s $\textbf{\color{#d91a1a}-62.13\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 7.7241ms 2.2669ms 441.1338 Ops/s 355.4115 Ops/s $\textbf{\color{#35bf28}+24.12\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.6583ms 1.1702ms 854.5548 Ops/s 653.5407 Ops/s $\textbf{\color{#35bf28}+30.76\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 7.6792ms 5.2742ms 189.6011 Ops/s 208.3553 Ops/s $\textbf{\color{#d91a1a}-9.00\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.2007ms 2.8123ms 355.5851 Ops/s 386.4461 Ops/s $\textbf{\color{#d91a1a}-7.99\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.5239ms 1.3518ms 739.7488 Ops/s 699.4047 Ops/s $\textbf{\color{#35bf28}+5.77\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.3804s 11.8669ms 84.2682 Ops/s 197.8730 Ops/s $\textbf{\color{#d91a1a}-57.41\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 8.7804ms 2.4676ms 405.2476 Ops/s 322.3807 Ops/s $\textbf{\color{#35bf28}+25.70\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 3.9338ms 1.4130ms 707.7377 Ops/s 632.1816 Ops/s $\textbf{\color{#35bf28}+11.95\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 13.3941ms 11.4102ms 87.6409 Ops/s 81.2649 Ops/s $\textbf{\color{#35bf28}+7.85\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 15.6518ms 14.7777ms 67.6697 Ops/s 65.2418 Ops/s $\color{#35bf28}+3.72\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 22.9409ms 20.6437ms 48.4410 Ops/s 47.7145 Ops/s $\color{#35bf28}+1.52\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 15.3403ms 14.9325ms 66.9679 Ops/s 65.0045 Ops/s $\color{#35bf28}+3.02\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 20.8919ms 20.1121ms 49.7212 Ops/s 47.8912 Ops/s $\color{#35bf28}+3.82\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 17.4471ms 16.0589ms 62.2709 Ops/s 59.8395 Ops/s $\color{#35bf28}+4.06\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}14$. Worsened: $\large\color{#d91a1a}9$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.7495s 0.7486s 1.3358 Ops/s 1.2789 Ops/s $\color{#35bf28}+4.45\%$
test_transformed 1.0988s 1.0194s 0.9810 Ops/s 0.9647 Ops/s $\color{#35bf28}+1.69\%$
test_serial 2.2362s 2.1574s 0.4635 Ops/s 0.4497 Ops/s $\color{#35bf28}+3.08\%$
test_parallel 2.2808s 2.0515s 0.4874 Ops/s 0.4939 Ops/s $\color{#d91a1a}-1.30\%$
test_step_mdp_speed[True-True-True-True-True] 0.1985ms 38.5669μs 25.9290 KOps/s 25.4453 KOps/s $\color{#35bf28}+1.90\%$
test_step_mdp_speed[True-True-True-True-False] 60.2630μs 22.4263μs 44.5905 KOps/s 43.9277 KOps/s $\color{#35bf28}+1.51\%$
test_step_mdp_speed[True-True-True-False-True] 56.2720μs 20.9446μs 47.7450 KOps/s 46.7145 KOps/s $\color{#35bf28}+2.21\%$
test_step_mdp_speed[True-True-True-False-False] 76.6230μs 12.6155μs 79.2678 KOps/s 79.7151 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[True-True-False-True-True] 0.4114ms 41.3795μs 24.1665 KOps/s 24.5097 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[True-True-False-True-False] 0.3916ms 24.5157μs 40.7902 KOps/s 40.0741 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[True-True-False-False-True] 0.1253ms 24.0791μs 41.5298 KOps/s 42.3660 KOps/s $\color{#d91a1a}-1.97\%$
test_step_mdp_speed[True-True-False-False-False] 0.4014ms 14.9065μs 67.0847 KOps/s 67.0998 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-False-True-True-True] 0.4272ms 44.0988μs 22.6763 KOps/s 22.7969 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-False-True-True-False] 0.4099ms 27.0410μs 36.9809 KOps/s 36.7786 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[True-False-True-False-True] 52.3520μs 23.4101μs 42.7167 KOps/s 41.5046 KOps/s $\color{#35bf28}+2.92\%$
test_step_mdp_speed[True-False-True-False-False] 0.3918ms 14.8230μs 67.4626 KOps/s 67.0373 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-False-False-True-True] 79.3230μs 45.7889μs 21.8393 KOps/s 21.5091 KOps/s $\color{#35bf28}+1.54\%$
test_step_mdp_speed[True-False-False-True-False] 0.4279ms 28.5875μs 34.9804 KOps/s 34.1858 KOps/s $\color{#35bf28}+2.32\%$
test_step_mdp_speed[True-False-False-False-True] 0.4085ms 25.9508μs 38.5344 KOps/s 38.8596 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[True-False-False-False-False] 0.4000ms 16.8922μs 59.1991 KOps/s 59.8023 KOps/s $\color{#d91a1a}-1.01\%$
test_step_mdp_speed[False-True-True-True-True] 77.5230μs 43.5324μs 22.9714 KOps/s 22.4811 KOps/s $\color{#35bf28}+2.18\%$
test_step_mdp_speed[False-True-True-True-False] 86.5140μs 25.9052μs 38.6023 KOps/s 37.2345 KOps/s $\color{#35bf28}+3.67\%$
test_step_mdp_speed[False-True-True-False-True] 72.5330μs 27.3247μs 36.5969 KOps/s 35.1455 KOps/s $\color{#35bf28}+4.13\%$
test_step_mdp_speed[False-True-True-False-False] 63.0030μs 16.2687μs 61.4677 KOps/s 61.3294 KOps/s $\color{#35bf28}+0.23\%$
test_step_mdp_speed[False-True-False-True-True] 90.8850μs 45.8892μs 21.7916 KOps/s 21.6203 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[False-True-False-True-False] 55.1120μs 28.7697μs 34.7588 KOps/s 34.2385 KOps/s $\color{#35bf28}+1.52\%$
test_step_mdp_speed[False-True-False-False-True] 3.0497ms 29.6386μs 33.7398 KOps/s 33.3709 KOps/s $\color{#35bf28}+1.11\%$
test_step_mdp_speed[False-True-False-False-False] 54.9030μs 18.4836μs 54.1020 KOps/s 53.2888 KOps/s $\color{#35bf28}+1.53\%$
test_step_mdp_speed[False-False-True-True-True] 95.8140μs 48.0530μs 20.8104 KOps/s 20.8922 KOps/s $\color{#d91a1a}-0.39\%$
test_step_mdp_speed[False-False-True-True-False] 66.6230μs 30.8924μs 32.3704 KOps/s 32.1179 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[False-False-True-False-True] 57.2730μs 29.6297μs 33.7499 KOps/s 33.7176 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-False-True-False-False] 54.2630μs 18.4495μs 54.2021 KOps/s 53.2816 KOps/s $\color{#35bf28}+1.73\%$
test_step_mdp_speed[False-False-False-True-True] 89.0540μs 49.3049μs 20.2820 KOps/s 20.6997 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[False-False-False-True-False] 64.2930μs 32.7993μs 30.4885 KOps/s 30.1544 KOps/s $\color{#35bf28}+1.11\%$
test_step_mdp_speed[False-False-False-False-True] 60.0830μs 30.6123μs 32.6667 KOps/s 32.4550 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-False-False-False-False] 47.6320μs 20.2352μs 49.4189 KOps/s 49.0163 KOps/s $\color{#35bf28}+0.82\%$
test_values[generalized_advantage_estimate-True-True] 25.1896ms 24.8323ms 40.2702 Ops/s 39.8002 Ops/s $\color{#35bf28}+1.18\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1082s 3.0722ms 325.5014 Ops/s 324.3565 Ops/s $\color{#35bf28}+0.35\%$
test_values[td0_return_estimate-False-False] 0.1082ms 81.7036μs 12.2394 KOps/s 12.0612 KOps/s $\color{#35bf28}+1.48\%$
test_values[td1_return_estimate-False-False] 56.0102ms 55.5878ms 17.9896 Ops/s 17.7720 Ops/s $\color{#35bf28}+1.22\%$
test_values[vec_td1_return_estimate-False-False] 1.3251ms 1.0953ms 912.9977 Ops/s 911.9201 Ops/s $\color{#35bf28}+0.12\%$
test_values[td_lambda_return_estimate-True-False] 88.9721ms 87.8617ms 11.3815 Ops/s 11.2191 Ops/s $\color{#35bf28}+1.45\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.2675ms 1.0839ms 922.6078 Ops/s 915.2173 Ops/s $\color{#35bf28}+0.81\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.7001ms 24.4694ms 40.8674 Ops/s 40.4513 Ops/s $\color{#35bf28}+1.03\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0507ms 0.7647ms 1.3077 KOps/s 1.2984 KOps/s $\color{#35bf28}+0.71\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8372ms 0.6812ms 1.4680 KOps/s 1.4601 KOps/s $\color{#35bf28}+0.53\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5401ms 1.4878ms 672.1472 Ops/s 670.1356 Ops/s $\color{#35bf28}+0.30\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8371ms 0.6988ms 1.4310 KOps/s 1.4339 KOps/s $\color{#d91a1a}-0.20\%$
test_dqn_speed[False-None] 7.0313ms 1.5354ms 651.3066 Ops/s 653.1294 Ops/s $\color{#d91a1a}-0.28\%$
test_dqn_speed[False-backward] 2.3018ms 2.1391ms 467.4930 Ops/s 465.9760 Ops/s $\color{#35bf28}+0.33\%$
test_dqn_speed[True-None] 0.7267ms 0.5580ms 1.7922 KOps/s 1.8515 KOps/s $\color{#d91a1a}-3.20\%$
test_dqn_speed[True-backward] 1.1906ms 1.1059ms 904.2600 Ops/s 888.9652 Ops/s $\color{#35bf28}+1.72\%$
test_dqn_speed[reduce-overhead-None] 0.5924ms 0.5482ms 1.8241 KOps/s 1.7977 KOps/s $\color{#35bf28}+1.47\%$
test_dqn_speed[reduce-overhead-backward] 1.0144ms 0.9692ms 1.0317 KOps/s 1.0220 KOps/s $\color{#35bf28}+0.96\%$
test_ddpg_speed[False-None] 3.1820ms 2.8565ms 350.0812 Ops/s 346.1736 Ops/s $\color{#35bf28}+1.13\%$
test_ddpg_speed[False-backward] 4.5845ms 4.0851ms 244.7939 Ops/s 242.3000 Ops/s $\color{#35bf28}+1.03\%$
test_ddpg_speed[True-None] 1.1584ms 1.0839ms 922.5747 Ops/s 916.0897 Ops/s $\color{#35bf28}+0.71\%$
test_ddpg_speed[True-backward] 2.2323ms 2.1699ms 460.8562 Ops/s 455.8433 Ops/s $\color{#35bf28}+1.10\%$
test_ddpg_speed[reduce-overhead-None] 1.2193ms 1.0950ms 913.2675 Ops/s 913.9543 Ops/s $\color{#d91a1a}-0.08\%$
test_ddpg_speed[reduce-overhead-backward] 1.7031ms 1.6447ms 608.0163 Ops/s 603.4860 Ops/s $\color{#35bf28}+0.75\%$
test_sac_speed[False-None] 8.5673ms 8.0731ms 123.8686 Ops/s 122.0558 Ops/s $\color{#35bf28}+1.49\%$
test_sac_speed[False-backward] 11.4789ms 11.0386ms 90.5911 Ops/s 89.7753 Ops/s $\color{#35bf28}+0.91\%$
test_sac_speed[True-None] 1.6368ms 1.5476ms 646.1492 Ops/s 626.7374 Ops/s $\color{#35bf28}+3.10\%$
test_sac_speed[True-backward] 3.5058ms 3.4341ms 291.1965 Ops/s 302.9998 Ops/s $\color{#d91a1a}-3.90\%$
test_sac_speed[reduce-overhead-None] 22.5495ms 12.4904ms 80.0617 Ops/s 80.1612 Ops/s $\color{#d91a1a}-0.12\%$
test_sac_speed[reduce-overhead-backward] 1.3910ms 1.3295ms 752.1388 Ops/s 664.9218 Ops/s $\textbf{\color{#35bf28}+13.12\%}$
test_redq_speed[False-None] 8.3257ms 7.5171ms 133.0294 Ops/s 130.7251 Ops/s $\color{#35bf28}+1.76\%$
test_redq_speed[False-backward] 12.0679ms 11.2933ms 88.5481 Ops/s 84.4957 Ops/s $\color{#35bf28}+4.80\%$
test_redq_speed[True-None] 2.0750ms 1.9911ms 502.2359 Ops/s 495.3118 Ops/s $\color{#35bf28}+1.40\%$
test_redq_speed[True-backward] 3.7416ms 3.6462ms 274.2566 Ops/s 266.8213 Ops/s $\color{#35bf28}+2.79\%$
test_redq_speed[reduce-overhead-None] 2.1989ms 1.9917ms 502.0770 Ops/s 491.5519 Ops/s $\color{#35bf28}+2.14\%$
test_redq_speed[reduce-overhead-backward] 4.3105ms 3.8687ms 258.4869 Ops/s 269.1644 Ops/s $\color{#d91a1a}-3.97\%$
test_redq_deprec_speed[False-None] 9.5768ms 9.0699ms 110.2548 Ops/s 108.1171 Ops/s $\color{#35bf28}+1.98\%$
test_redq_deprec_speed[False-backward] 12.8203ms 12.3023ms 81.2855 Ops/s 80.9740 Ops/s $\color{#35bf28}+0.38\%$
test_redq_deprec_speed[True-None] 2.4374ms 2.3326ms 428.7103 Ops/s 421.0254 Ops/s $\color{#35bf28}+1.83\%$
test_redq_deprec_speed[True-backward] 4.3448ms 4.2683ms 234.2827 Ops/s 234.0212 Ops/s $\color{#35bf28}+0.11\%$
test_redq_deprec_speed[reduce-overhead-None] 2.4234ms 2.3269ms 429.7535 Ops/s 422.3251 Ops/s $\color{#35bf28}+1.76\%$
test_redq_deprec_speed[reduce-overhead-backward] 4.2959ms 4.2278ms 236.5295 Ops/s 236.2205 Ops/s $\color{#35bf28}+0.13\%$
test_td3_speed[False-None] 8.0051ms 7.9350ms 126.0242 Ops/s 123.7782 Ops/s $\color{#35bf28}+1.81\%$
test_td3_speed[False-backward] 10.9593ms 10.4264ms 95.9102 Ops/s 94.2901 Ops/s $\color{#35bf28}+1.72\%$
test_td3_speed[True-None] 1.5954ms 1.5764ms 634.3539 Ops/s 628.7323 Ops/s $\color{#35bf28}+0.89\%$
test_td3_speed[True-backward] 3.4371ms 3.3308ms 300.2267 Ops/s 300.2673 Ops/s $\color{#d91a1a}-0.01\%$
test_td3_speed[reduce-overhead-None] 49.4877ms 25.2994ms 39.5266 Ops/s 37.5268 Ops/s $\textbf{\color{#35bf28}+5.33\%}$
test_td3_speed[reduce-overhead-backward] 1.4310ms 1.3817ms 723.7675 Ops/s 720.5824 Ops/s $\color{#35bf28}+0.44\%$
test_cql_speed[False-None] 16.6431ms 16.2791ms 61.4285 Ops/s 60.3394 Ops/s $\color{#35bf28}+1.80\%$
test_cql_speed[False-backward] 21.7330ms 21.4889ms 46.5357 Ops/s 45.2818 Ops/s $\color{#35bf28}+2.77\%$
test_cql_speed[True-None] 3.0761ms 2.9439ms 339.6892 Ops/s 332.6483 Ops/s $\color{#35bf28}+2.12\%$
test_cql_speed[True-backward] 5.2202ms 5.0737ms 197.0960 Ops/s 189.9997 Ops/s $\color{#35bf28}+3.73\%$
test_cql_speed[reduce-overhead-None] 21.4144ms 13.1104ms 76.2755 Ops/s 76.7011 Ops/s $\color{#d91a1a}-0.55\%$
test_cql_speed[reduce-overhead-backward] 1.5484ms 1.4890ms 671.5906 Ops/s 663.8899 Ops/s $\color{#35bf28}+1.16\%$
test_a2c_speed[False-None] 3.2999ms 3.2084ms 311.6798 Ops/s 304.3654 Ops/s $\color{#35bf28}+2.40\%$
test_a2c_speed[False-backward] 6.7036ms 6.1097ms 163.6731 Ops/s 162.2591 Ops/s $\color{#35bf28}+0.87\%$
test_a2c_speed[True-None] 1.0547ms 1.0020ms 998.0222 Ops/s 968.2812 Ops/s $\color{#35bf28}+3.07\%$
test_a2c_speed[True-backward] 2.7683ms 2.6382ms 379.0500 Ops/s 375.4209 Ops/s $\color{#35bf28}+0.97\%$
test_a2c_speed[reduce-overhead-None] 0.3915s 12.1566ms 82.2596 Ops/s 87.9235 Ops/s $\textbf{\color{#d91a1a}-6.44\%}$
test_a2c_speed[reduce-overhead-backward] 1.0670ms 0.9919ms 1.0082 KOps/s 1.0116 KOps/s $\color{#d91a1a}-0.33\%$
test_ppo_speed[False-None] 3.7655ms 3.6809ms 271.6735 Ops/s 266.7579 Ops/s $\color{#35bf28}+1.84\%$
test_ppo_speed[False-backward] 7.4546ms 6.8260ms 146.4995 Ops/s 144.9711 Ops/s $\color{#35bf28}+1.05\%$
test_ppo_speed[True-None] 0.9942ms 0.9421ms 1.0615 KOps/s 1.0378 KOps/s $\color{#35bf28}+2.28\%$
test_ppo_speed[True-backward] 2.6821ms 2.5891ms 386.2393 Ops/s 360.5675 Ops/s $\textbf{\color{#35bf28}+7.12\%}$
test_ppo_speed[reduce-overhead-None] 0.5486ms 0.4877ms 2.0504 KOps/s 1.9093 KOps/s $\textbf{\color{#35bf28}+7.39\%}$
test_ppo_speed[reduce-overhead-backward] 1.1524ms 1.1154ms 896.5470 Ops/s 1.0057 KOps/s $\textbf{\color{#d91a1a}-10.85\%}$
test_reinforce_speed[False-None] 2.3647ms 2.2559ms 443.2852 Ops/s 434.6759 Ops/s $\color{#35bf28}+1.98\%$
test_reinforce_speed[False-backward] 3.7804ms 3.4048ms 293.7000 Ops/s 302.9634 Ops/s $\color{#d91a1a}-3.06\%$
test_reinforce_speed[True-None] 0.9017ms 0.8350ms 1.1976 KOps/s 1.1918 KOps/s $\color{#35bf28}+0.49\%$
test_reinforce_speed[True-backward] 2.7427ms 2.6281ms 380.5092 Ops/s 382.6943 Ops/s $\color{#d91a1a}-0.57\%$
test_reinforce_speed[reduce-overhead-None] 21.8089ms 11.4877ms 87.0494 Ops/s 88.5768 Ops/s $\color{#d91a1a}-1.72\%$
test_reinforce_speed[reduce-overhead-backward] 1.1946ms 1.1541ms 866.4494 Ops/s 951.0708 Ops/s $\textbf{\color{#d91a1a}-8.90\%}$
test_iql_speed[False-None] 10.2325ms 9.5878ms 104.2997 Ops/s 107.1100 Ops/s $\color{#d91a1a}-2.62\%$
test_iql_speed[False-backward] 13.8121ms 13.3889ms 74.6886 Ops/s 76.1974 Ops/s $\color{#d91a1a}-1.98\%$
test_iql_speed[True-None] 1.8430ms 1.7710ms 564.6636 Ops/s 566.4683 Ops/s $\color{#d91a1a}-0.32\%$
test_iql_speed[True-backward] 4.6025ms 4.2565ms 234.9345 Ops/s 223.0936 Ops/s $\textbf{\color{#35bf28}+5.31\%}$
test_iql_speed[reduce-overhead-None] 19.8647ms 11.3913ms 87.7861 Ops/s 89.3427 Ops/s $\color{#d91a1a}-1.74\%$
test_iql_speed[reduce-overhead-backward] 1.6416ms 1.5954ms 626.8141 Ops/s 633.2116 Ops/s $\color{#d91a1a}-1.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.9135ms 6.3873ms 156.5611 Ops/s 154.6899 Ops/s $\color{#35bf28}+1.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.4915ms 0.2720ms 3.6761 KOps/s 2.8445 KOps/s $\textbf{\color{#35bf28}+29.23\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4980ms 0.2658ms 3.7623 KOps/s 3.0290 KOps/s $\textbf{\color{#35bf28}+24.21\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.3800ms 6.1469ms 162.6825 Ops/s 160.9257 Ops/s $\color{#35bf28}+1.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.0454ms 0.2572ms 3.8881 KOps/s 2.8315 KOps/s $\textbf{\color{#35bf28}+37.31\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4462ms 0.2417ms 4.1377 KOps/s 3.3288 KOps/s $\textbf{\color{#35bf28}+24.30\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.4751ms 1.2908ms 774.7057 Ops/s 741.9831 Ops/s $\color{#35bf28}+4.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6246ms 1.3122ms 762.0504 Ops/s 825.0888 Ops/s $\textbf{\color{#d91a1a}-7.64\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.4570ms 6.3148ms 158.3582 Ops/s 156.8996 Ops/s $\color{#35bf28}+0.93\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.0089ms 0.4580ms 2.1832 KOps/s 2.2446 KOps/s $\color{#d91a1a}-2.73\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7067ms 0.4312ms 2.3192 KOps/s 2.5714 KOps/s $\textbf{\color{#d91a1a}-9.81\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.2494ms 6.1274ms 163.2015 Ops/s 160.0929 Ops/s $\color{#35bf28}+1.94\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7234ms 0.3293ms 3.0369 KOps/s 3.0013 KOps/s $\color{#35bf28}+1.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5431ms 0.3004ms 3.3292 KOps/s 2.7734 KOps/s $\textbf{\color{#35bf28}+20.04\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.3600ms 6.1349ms 163.0026 Ops/s 161.9316 Ops/s $\color{#35bf28}+0.66\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.6682ms 0.2979ms 3.3571 KOps/s 3.1199 KOps/s $\textbf{\color{#35bf28}+7.60\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4424ms 0.2442ms 4.0946 KOps/s 4.1165 KOps/s $\color{#d91a1a}-0.53\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.4553ms 6.3234ms 158.1437 Ops/s 157.6695 Ops/s $\color{#35bf28}+0.30\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2030ms 0.4088ms 2.4460 KOps/s 2.4126 KOps/s $\color{#35bf28}+1.38\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.5861ms 0.4013ms 2.4920 KOps/s 2.5954 KOps/s $\color{#d91a1a}-3.98\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.9195ms 5.3052ms 188.4959 Ops/s 190.3005 Ops/s $\color{#d91a1a}-0.95\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 10.1175ms 2.0943ms 477.4796 Ops/s 432.8987 Ops/s $\textbf{\color{#35bf28}+10.30\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.6683ms 1.1950ms 836.8353 Ops/s 841.6320 Ops/s $\color{#d91a1a}-0.57\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.9286ms 5.3310ms 187.5808 Ops/s 191.2625 Ops/s $\color{#d91a1a}-1.92\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.4977s 12.0159ms 83.2230 Ops/s 477.5486 Ops/s $\textbf{\color{#d91a1a}-82.57\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.3450ms 1.1255ms 888.4831 Ops/s 792.2685 Ops/s $\textbf{\color{#35bf28}+12.14\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 8.6189ms 5.5847ms 179.0604 Ops/s 33.0529 Ops/s $\textbf{\color{#35bf28}+441.74\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 7.8192ms 2.1985ms 454.8562 Ops/s 498.6159 Ops/s $\textbf{\color{#d91a1a}-8.78\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 8.6407ms 1.4468ms 691.2041 Ops/s 730.3193 Ops/s $\textbf{\color{#d91a1a}-5.36\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 14.0284ms 13.3245ms 75.0498 Ops/s 74.0238 Ops/s $\color{#35bf28}+1.39\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.8867ms 17.8228ms 56.1078 Ops/s 58.0855 Ops/s $\color{#d91a1a}-3.40\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 19.0866ms 18.3358ms 54.5382 Ops/s 54.2431 Ops/s $\color{#35bf28}+0.54\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 18.1652ms 17.5123ms 57.1027 Ops/s 58.4937 Ops/s $\color{#d91a1a}-2.38\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 20.9923ms 18.3058ms 54.6275 Ops/s 55.3300 Ops/s $\color{#d91a1a}-1.27\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 19.8154ms 19.1494ms 52.2209 Ops/s 55.3504 Ops/s $\textbf{\color{#d91a1a}-5.65\%}$

@vmoens vmoens merged commit e3107b5 into gh/vmoens/57/base Dec 12, 2024
65 of 78 checks passed
vmoens added a commit that referenced this pull request Dec 12, 2024
@vmoens
[BugFix] Fix init_random_frames=0
19dfefc 
ghstack-source-id: 38a544ea15631f9affb4c385c09e7c4df94af55d
Pull Request resolved: #2645
@vmoens vmoens deleted the gh/vmoens/57/head branch December 12, 2024 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] SyncDataCollector Crashes when init_random_frames=0 with a policy that is NOT random
2 participants
@vmoens @facebook-github-bot