Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Tutorial] MCTS #2673

Open
wants to merge 1 commit into
base: gh/vmoens/63/base
Choose a base branch
from
Open

[Tutorial] MCTS #2673

wants to merge 1 commit into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Dec 19, 2024 •
edited
Loading

@vmoens
Update
1f0714c 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Dec 19, 2024
Open
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Dec 19, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2673

Note: Links to docs will display an error until the docs builds have been completed.

❌ 10 New Failures, 7 Unrelated Failures

As of commit 1f0714c with merge base 133d709 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 19, 2024
vmoens added a commit that referenced this pull request Dec 19, 2024
@vmoens
[Tutorial] MCTS
2d17ca7 
ghstack-source-id: 4d10603799b711c099db30ae2be4d46766f3adfc
Pull Request resolved: #2673
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}12$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.4257s 0.4235s 2.3612 Ops/s 2.2006 Ops/s $\textbf{\color{#35bf28}+7.30\%}$
test_transformed 0.6004s 0.5967s 1.6760 Ops/s 1.6290 Ops/s $\color{#35bf28}+2.88\%$
test_serial 1.4007s 1.3674s 0.7313 Ops/s 0.7185 Ops/s $\color{#35bf28}+1.79\%$
test_parallel 1.3057s 1.2097s 0.8267 Ops/s 0.8108 Ops/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[True-True-True-True-True] 0.2166ms 31.5298μs 31.7160 KOps/s 31.5442 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[True-True-True-True-False] 58.2080μs 18.1468μs 55.1063 KOps/s 53.9424 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-True-True-False-True] 82.9640μs 17.7137μs 56.4534 KOps/s 56.0634 KOps/s $\color{#35bf28}+0.70\%$
test_step_mdp_speed[True-True-True-False-False] 38.0800μs 10.2933μs 97.1501 KOps/s 95.6064 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[True-True-False-True-True] 80.1680μs 33.4484μs 29.8968 KOps/s 28.8454 KOps/s $\color{#35bf28}+3.64\%$
test_step_mdp_speed[True-True-False-True-False] 74.0250μs 19.9191μs 50.2032 KOps/s 48.5588 KOps/s $\color{#35bf28}+3.39\%$
test_step_mdp_speed[True-True-False-False-True] 0.1264ms 20.0067μs 49.9833 KOps/s 50.1995 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-True-False-False-False] 0.1194ms 12.6507μs 79.0473 KOps/s 80.1188 KOps/s $\color{#d91a1a}-1.34\%$
test_step_mdp_speed[True-False-True-True-True] 0.1087ms 35.7654μs 27.9600 KOps/s 27.5655 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[True-False-True-True-False] 57.7670μs 22.3471μs 44.7486 KOps/s 40.7790 KOps/s $\textbf{\color{#35bf28}+9.73\%}$
test_step_mdp_speed[True-False-True-False-True] 68.0760μs 19.5142μs 51.2447 KOps/s 49.7618 KOps/s $\color{#35bf28}+2.98\%$
test_step_mdp_speed[True-False-True-False-False] 34.7150μs 12.2864μs 81.3905 KOps/s 78.6008 KOps/s $\color{#35bf28}+3.55\%$
test_step_mdp_speed[True-False-False-True-True] 97.4510μs 37.2272μs 26.8621 KOps/s 25.6065 KOps/s $\color{#35bf28}+4.90\%$
test_step_mdp_speed[True-False-False-True-False] 70.5010μs 24.2528μs 41.2323 KOps/s 40.6187 KOps/s $\color{#35bf28}+1.51\%$
test_step_mdp_speed[True-False-False-False-True] 67.8360μs 21.4051μs 46.7179 KOps/s 45.7701 KOps/s $\color{#35bf28}+2.07\%$
test_step_mdp_speed[True-False-False-False-False] 68.6170μs 14.0004μs 71.4264 KOps/s 69.4410 KOps/s $\color{#35bf28}+2.86\%$
test_step_mdp_speed[False-True-True-True-True] 68.3070μs 35.6387μs 28.0594 KOps/s 27.5273 KOps/s $\color{#35bf28}+1.93\%$
test_step_mdp_speed[False-True-True-True-False] 76.2510μs 22.2635μs 44.9165 KOps/s 44.3145 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[False-True-True-False-True] 76.3420μs 22.4842μs 44.4756 KOps/s 43.0102 KOps/s $\color{#35bf28}+3.41\%$
test_step_mdp_speed[False-True-True-False-False] 0.1839ms 13.8930μs 71.9787 KOps/s 69.8340 KOps/s $\color{#35bf28}+3.07\%$
test_step_mdp_speed[False-True-False-True-True] 94.6160μs 37.2700μs 26.8312 KOps/s 25.9580 KOps/s $\color{#35bf28}+3.36\%$
test_step_mdp_speed[False-True-False-True-False] 66.6940μs 23.9105μs 41.8227 KOps/s 37.2414 KOps/s $\textbf{\color{#35bf28}+12.30\%}$
test_step_mdp_speed[False-True-False-False-True] 2.7430ms 24.7131μs 40.4643 KOps/s 39.8451 KOps/s $\color{#35bf28}+1.55\%$
test_step_mdp_speed[False-True-False-False-False] 50.6040μs 15.7475μs 63.5020 KOps/s 63.2942 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[False-False-True-True-True] 99.8150μs 39.1171μs 25.5642 KOps/s 25.0047 KOps/s $\color{#35bf28}+2.24\%$
test_step_mdp_speed[False-False-True-True-False] 79.1070μs 26.1787μs 38.1990 KOps/s 37.3119 KOps/s $\color{#35bf28}+2.38\%$
test_step_mdp_speed[False-False-True-False-True] 82.2630μs 24.9194μs 40.1294 KOps/s 40.6732 KOps/s $\color{#d91a1a}-1.34\%$
test_step_mdp_speed[False-False-True-False-False] 62.1360μs 15.7582μs 63.4589 KOps/s 62.7569 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[False-False-False-True-True] 78.6060μs 40.8688μs 24.4685 KOps/s 23.8290 KOps/s $\color{#35bf28}+2.68\%$
test_step_mdp_speed[False-False-False-True-False] 77.8140μs 27.6893μs 36.1151 KOps/s 35.5557 KOps/s $\color{#35bf28}+1.57\%$
test_step_mdp_speed[False-False-False-False-True] 63.4870μs 26.0673μs 38.3622 KOps/s 37.8469 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[False-False-False-False-False] 67.9660μs 17.3498μs 57.6374 KOps/s 56.1184 KOps/s $\color{#35bf28}+2.71\%$
test_values[generalized_advantage_estimate-True-True] 10.2362ms 9.8128ms 101.9075 Ops/s 100.3893 Ops/s $\color{#35bf28}+1.51\%$
test_values[vec_generalized_advantage_estimate-True-True] 40.0826ms 33.4828ms 29.8660 Ops/s 29.9075 Ops/s $\color{#d91a1a}-0.14\%$
test_values[td0_return_estimate-False-False] 0.2554ms 0.1904ms 5.2523 KOps/s 5.6030 KOps/s $\textbf{\color{#d91a1a}-6.26\%}$
test_values[td1_return_estimate-False-False] 28.1958ms 24.5261ms 40.7729 Ops/s 40.6081 Ops/s $\color{#35bf28}+0.41\%$
test_values[vec_td1_return_estimate-False-False] 36.4581ms 33.6157ms 29.7480 Ops/s 29.8371 Ops/s $\color{#d91a1a}-0.30\%$
test_values[td_lambda_return_estimate-True-False] 35.4658ms 34.5961ms 28.9050 Ops/s 28.0439 Ops/s $\color{#35bf28}+3.07\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.0163ms 33.5078ms 29.8438 Ops/s 29.8331 Ops/s $\color{#35bf28}+0.04\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.7022ms 8.4188ms 118.7812 Ops/s 116.3240 Ops/s $\color{#35bf28}+2.11\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2359ms 1.8633ms 536.6840 Ops/s 537.4082 Ops/s $\color{#d91a1a}-0.13\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4403ms 0.3537ms 2.8276 KOps/s 2.8247 KOps/s $\color{#35bf28}+0.10\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 41.8886ms 38.1423ms 26.2176 Ops/s 25.1253 Ops/s $\color{#35bf28}+4.35\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8296ms 3.0420ms 328.7353 Ops/s 329.9557 Ops/s $\color{#d91a1a}-0.37\%$
test_dqn_speed[False-None] 5.9706ms 1.4176ms 705.4291 Ops/s 698.2480 Ops/s $\color{#35bf28}+1.03\%$
test_dqn_speed[False-backward] 2.0156ms 1.8922ms 528.4957 Ops/s 524.5471 Ops/s $\color{#35bf28}+0.75\%$
test_dqn_speed[True-None] 0.8060ms 0.4895ms 2.0428 KOps/s 2.0450 KOps/s $\color{#d91a1a}-0.11\%$
test_dqn_speed[True-backward] 0.9518ms 0.9146ms 1.0934 KOps/s 812.8590 Ops/s $\textbf{\color{#35bf28}+34.51\%}$
test_dqn_speed[reduce-overhead-None] 0.6214ms 0.4912ms 2.0358 KOps/s 2.0583 KOps/s $\color{#d91a1a}-1.09\%$
test_dqn_speed[reduce-overhead-backward] 0.9765ms 0.9177ms 1.0897 KOps/s 1.0681 KOps/s $\color{#35bf28}+2.02\%$
test_ddpg_speed[False-None] 3.8865ms 2.9519ms 338.7593 Ops/s 336.5679 Ops/s $\color{#35bf28}+0.65\%$
test_ddpg_speed[False-backward] 4.8878ms 4.2368ms 236.0296 Ops/s 242.7510 Ops/s $\color{#d91a1a}-2.77\%$
test_ddpg_speed[True-None] 1.3472ms 1.0325ms 968.5056 Ops/s 935.9112 Ops/s $\color{#35bf28}+3.48\%$
test_ddpg_speed[True-backward] 2.0161ms 1.9465ms 513.7491 Ops/s 453.6334 Ops/s $\textbf{\color{#35bf28}+13.25\%}$
test_ddpg_speed[reduce-overhead-None] 1.2744ms 1.0247ms 975.8630 Ops/s 964.3105 Ops/s $\color{#35bf28}+1.20\%$
test_ddpg_speed[reduce-overhead-backward] 2.0232ms 1.9427ms 514.7513 Ops/s 496.7735 Ops/s $\color{#35bf28}+3.62\%$
test_sac_speed[False-None] 10.9277ms 8.2201ms 121.6530 Ops/s 117.5380 Ops/s $\color{#35bf28}+3.50\%$
test_sac_speed[False-backward] 12.2312ms 11.0499ms 90.4986 Ops/s 86.7765 Ops/s $\color{#35bf28}+4.29\%$
test_sac_speed[True-None] 2.5890ms 1.8735ms 533.7658 Ops/s 526.9212 Ops/s $\color{#35bf28}+1.30\%$
test_sac_speed[True-backward] 3.8576ms 3.6015ms 277.6640 Ops/s 276.4861 Ops/s $\color{#35bf28}+0.43\%$
test_sac_speed[reduce-overhead-None] 2.0652ms 1.8643ms 536.3965 Ops/s 516.1923 Ops/s $\color{#35bf28}+3.91\%$
test_sac_speed[reduce-overhead-backward] 4.4420ms 3.6390ms 274.7977 Ops/s 276.2167 Ops/s $\color{#d91a1a}-0.51\%$
test_redq_speed[False-None] 15.4562ms 13.0071ms 76.8812 Ops/s 76.2145 Ops/s $\color{#35bf28}+0.87\%$
test_redq_speed[False-backward] 23.2428ms 22.0948ms 45.2595 Ops/s 44.7861 Ops/s $\color{#35bf28}+1.06\%$
test_redq_speed[True-None] 5.8832ms 4.9929ms 200.2857 Ops/s 180.6563 Ops/s $\textbf{\color{#35bf28}+10.87\%}$
test_redq_speed[True-backward] 13.7058ms 12.8413ms 77.8739 Ops/s 77.6087 Ops/s $\color{#35bf28}+0.34\%$
test_redq_speed[reduce-overhead-None] 6.0649ms 5.0557ms 197.7970 Ops/s 197.2933 Ops/s $\color{#35bf28}+0.26\%$
test_redq_speed[reduce-overhead-backward] 13.5533ms 12.6495ms 79.0544 Ops/s 76.4264 Ops/s $\color{#35bf28}+3.44\%$
test_redq_deprec_speed[False-None] 15.4219ms 13.3789ms 74.7448 Ops/s 73.3805 Ops/s $\color{#35bf28}+1.86\%$
test_redq_deprec_speed[False-backward] 21.7956ms 19.2285ms 52.0061 Ops/s 50.6223 Ops/s $\color{#35bf28}+2.73\%$
test_redq_deprec_speed[True-None] 4.3743ms 3.6682ms 272.6114 Ops/s 278.9861 Ops/s $\color{#d91a1a}-2.28\%$
test_redq_deprec_speed[True-backward] 9.2675ms 8.5880ms 116.4414 Ops/s 124.0392 Ops/s $\textbf{\color{#d91a1a}-6.13\%}$
test_redq_deprec_speed[reduce-overhead-None] 4.4332ms 3.7259ms 268.3914 Ops/s 279.5016 Ops/s $\color{#d91a1a}-3.97\%$
test_redq_deprec_speed[reduce-overhead-backward] 9.5184ms 8.6326ms 115.8403 Ops/s 121.1524 Ops/s $\color{#d91a1a}-4.38\%$
test_td3_speed[False-None] 9.0116ms 8.2206ms 121.6449 Ops/s 120.7336 Ops/s $\color{#35bf28}+0.75\%$
test_td3_speed[False-backward] 11.5333ms 10.6184ms 94.1762 Ops/s 89.2165 Ops/s $\textbf{\color{#35bf28}+5.56\%}$
test_td3_speed[True-None] 2.2487ms 1.7710ms 564.6555 Ops/s 576.7560 Ops/s $\color{#d91a1a}-2.10\%$
test_td3_speed[True-backward] 3.3973ms 3.3360ms 299.7646 Ops/s 296.2012 Ops/s $\color{#35bf28}+1.20\%$
test_td3_speed[reduce-overhead-None] 2.0494ms 1.7419ms 574.0965 Ops/s 575.3586 Ops/s $\color{#d91a1a}-0.22\%$
test_td3_speed[reduce-overhead-backward] 3.5649ms 3.3653ms 297.1529 Ops/s 300.2297 Ops/s $\color{#d91a1a}-1.02\%$
test_cql_speed[False-None] 38.5727ms 36.4406ms 27.4419 Ops/s 27.0860 Ops/s $\color{#35bf28}+1.31\%$
test_cql_speed[False-backward] 49.0383ms 46.3025ms 21.5971 Ops/s 21.2023 Ops/s $\color{#35bf28}+1.86\%$
test_cql_speed[True-None] 25.0891ms 15.8310ms 63.1671 Ops/s 63.1911 Ops/s $\color{#d91a1a}-0.04\%$
test_cql_speed[True-backward] 23.3986ms 22.2069ms 45.0311 Ops/s 44.2060 Ops/s $\color{#35bf28}+1.87\%$
test_cql_speed[reduce-overhead-None] 17.1367ms 15.5705ms 64.2238 Ops/s 63.6737 Ops/s $\color{#35bf28}+0.86\%$
test_cql_speed[reduce-overhead-backward] 24.2240ms 22.8738ms 43.7181 Ops/s 44.3132 Ops/s $\color{#d91a1a}-1.34\%$
test_a2c_speed[False-None] 7.5097ms 7.2069ms 138.7552 Ops/s 139.3742 Ops/s $\color{#d91a1a}-0.44\%$
test_a2c_speed[False-backward] 17.1063ms 14.4145ms 69.3748 Ops/s 66.5055 Ops/s $\color{#35bf28}+4.31\%$
test_a2c_speed[True-None] 4.6555ms 4.2022ms 237.9710 Ops/s 235.4502 Ops/s $\color{#35bf28}+1.07\%$
test_a2c_speed[True-backward] 11.7939ms 10.7653ms 92.8908 Ops/s 93.0067 Ops/s $\color{#d91a1a}-0.12\%$
test_a2c_speed[reduce-overhead-None] 4.9532ms 4.2009ms 238.0424 Ops/s 236.0565 Ops/s $\color{#35bf28}+0.84\%$
test_a2c_speed[reduce-overhead-backward] 11.0664ms 10.6467ms 93.9260 Ops/s 93.5526 Ops/s $\color{#35bf28}+0.40\%$
test_ppo_speed[False-None] 9.0187ms 7.4185ms 134.7982 Ops/s 133.5617 Ops/s $\color{#35bf28}+0.93\%$
test_ppo_speed[False-backward] 15.8316ms 14.6855ms 68.0945 Ops/s 68.5412 Ops/s $\color{#d91a1a}-0.65\%$
test_ppo_speed[True-None] 4.2993ms 3.6921ms 270.8501 Ops/s 269.4814 Ops/s $\color{#35bf28}+0.51\%$
test_ppo_speed[True-backward] 11.4225ms 9.5818ms 104.3646 Ops/s 103.2854 Ops/s $\color{#35bf28}+1.04\%$
test_ppo_speed[reduce-overhead-None] 4.1916ms 3.6905ms 270.9664 Ops/s 270.5673 Ops/s $\color{#35bf28}+0.15\%$
test_ppo_speed[reduce-overhead-backward] 10.5308ms 9.5351ms 104.8761 Ops/s 103.3043 Ops/s $\color{#35bf28}+1.52\%$
test_reinforce_speed[False-None] 7.5392ms 6.5220ms 153.3261 Ops/s 152.7555 Ops/s $\color{#35bf28}+0.37\%$
test_reinforce_speed[False-backward] 9.9990ms 9.7561ms 102.5000 Ops/s 101.5466 Ops/s $\color{#35bf28}+0.94\%$
test_reinforce_speed[True-None] 5.3375ms 2.7525ms 363.3074 Ops/s 373.4697 Ops/s $\color{#d91a1a}-2.72\%$
test_reinforce_speed[True-backward] 9.6055ms 8.5901ms 116.4136 Ops/s 116.7846 Ops/s $\color{#d91a1a}-0.32\%$
test_reinforce_speed[reduce-overhead-None] 5.5655ms 2.7178ms 367.9421 Ops/s 375.0095 Ops/s $\color{#d91a1a}-1.88\%$
test_reinforce_speed[reduce-overhead-backward] 9.2675ms 8.6177ms 116.0400 Ops/s 115.4026 Ops/s $\color{#35bf28}+0.55\%$
test_iql_speed[False-None] 48.0808ms 33.8226ms 29.5661 Ops/s 30.3533 Ops/s $\color{#d91a1a}-2.59\%$
test_iql_speed[False-backward] 47.6458ms 45.5219ms 21.9674 Ops/s 21.4590 Ops/s $\color{#35bf28}+2.37\%$
test_iql_speed[True-None] 11.9121ms 10.7035ms 93.4277 Ops/s 91.3877 Ops/s $\color{#35bf28}+2.23\%$
test_iql_speed[True-backward] 22.9043ms 21.4602ms 46.5979 Ops/s 44.8380 Ops/s $\color{#35bf28}+3.93\%$
test_iql_speed[reduce-overhead-None] 12.6229ms 10.7221ms 93.2656 Ops/s 90.1786 Ops/s $\color{#35bf28}+3.42\%$
test_iql_speed[reduce-overhead-backward] 22.8956ms 21.5539ms 46.3954 Ops/s 44.7305 Ops/s $\color{#35bf28}+3.72\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.7516ms 4.9293ms 202.8669 Ops/s 201.8214 Ops/s $\color{#35bf28}+0.52\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8510ms 0.5120ms 1.9532 KOps/s 1.9220 KOps/s $\color{#35bf28}+1.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9988ms 0.4951ms 2.0199 KOps/s 1.9970 KOps/s $\color{#35bf28}+1.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.1541ms 4.6916ms 213.1466 Ops/s 210.8819 Ops/s $\color{#35bf28}+1.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.4635ms 0.5000ms 1.9999 KOps/s 1.9643 KOps/s $\color{#35bf28}+1.81\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8147ms 0.4860ms 2.0577 KOps/s 2.0704 KOps/s $\color{#d91a1a}-0.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.5227ms 1.6493ms 606.3232 Ops/s 604.3870 Ops/s $\color{#35bf28}+0.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.4143ms 1.5956ms 626.7367 Ops/s 622.0131 Ops/s $\color{#35bf28}+0.76\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.0719ms 4.8695ms 205.3583 Ops/s 205.9429 Ops/s $\color{#d91a1a}-0.28\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2463ms 0.6483ms 1.5424 KOps/s 1.5425 KOps/s $-0.01\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.1005ms 0.6260ms 1.5975 KOps/s 1.6062 KOps/s $\color{#d91a1a}-0.54\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.4119ms 4.7527ms 210.4067 Ops/s 212.5998 Ops/s $\color{#d91a1a}-1.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0604ms 0.5153ms 1.9406 KOps/s 1.9253 KOps/s $\color{#35bf28}+0.79\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8008ms 0.5002ms 1.9993 KOps/s 2.0114 KOps/s $\color{#d91a1a}-0.60\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.4940ms 4.7389ms 211.0176 Ops/s 213.9437 Ops/s $\color{#d91a1a}-1.37\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9497ms 0.5038ms 1.9848 KOps/s 1.9316 KOps/s $\color{#35bf28}+2.75\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7959ms 0.4843ms 2.0650 KOps/s 2.0808 KOps/s $\color{#d91a1a}-0.76\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.5299ms 4.7980ms 208.4181 Ops/s 198.7105 Ops/s $\color{#35bf28}+4.89\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.1371ms 0.6540ms 1.5291 KOps/s 1.5550 KOps/s $\color{#d91a1a}-1.66\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8074ms 0.6224ms 1.6066 KOps/s 1.6190 KOps/s $\color{#d91a1a}-0.77\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4192s 12.5592ms 79.6226 Ops/s 250.5059 Ops/s $\textbf{\color{#d91a1a}-68.22\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 3.2001ms 2.0839ms 479.8768 Ops/s 432.5649 Ops/s $\textbf{\color{#35bf28}+10.94\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.6750ms 1.3422ms 745.0680 Ops/s 749.7827 Ops/s $\color{#d91a1a}-0.63\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 5.5381ms 4.2636ms 234.5441 Ops/s 37.8323 Ops/s $\textbf{\color{#35bf28}+519.96\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.0712ms 2.4229ms 412.7207 Ops/s 409.0240 Ops/s $\color{#35bf28}+0.90\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.8660ms 1.4091ms 709.6716 Ops/s 794.4119 Ops/s $\textbf{\color{#d91a1a}-10.67\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.3720s 11.8459ms 84.4177 Ops/s 224.7837 Ops/s $\textbf{\color{#d91a1a}-62.44\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 8.5415ms 2.4542ms 407.4622 Ops/s 405.3015 Ops/s $\color{#35bf28}+0.53\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.7813ms 1.6323ms 612.6290 Ops/s 635.2015 Ops/s $\color{#d91a1a}-3.55\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 16.5433ms 13.2558ms 75.4387 Ops/s 72.3191 Ops/s $\color{#35bf28}+4.31\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 15.8954ms 14.5930ms 68.5260 Ops/s 64.4638 Ops/s $\textbf{\color{#35bf28}+6.30\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 24.2035ms 21.7581ms 45.9599 Ops/s 44.0888 Ops/s $\color{#35bf28}+4.24\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 16.3905ms 14.8889ms 67.1642 Ops/s 63.8496 Ops/s $\textbf{\color{#35bf28}+5.19\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 24.1307ms 21.7442ms 45.9894 Ops/s 45.1270 Ops/s $\color{#35bf28}+1.91\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 17.9033ms 16.1703ms 61.8416 Ops/s 57.7981 Ops/s $\textbf{\color{#35bf28}+7.00\%}$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.7161s 0.7094s 1.4095 Ops/s 1.3621 Ops/s $\color{#35bf28}+3.48\%$
test_transformed 0.9645s 0.9585s 1.0433 Ops/s 1.0416 Ops/s $\color{#35bf28}+0.16\%$
test_serial 2.2221s 2.1310s 0.4693 Ops/s 0.4772 Ops/s $\color{#d91a1a}-1.67\%$
test_parallel 1.9381s 1.8289s 0.5468 Ops/s 0.5521 Ops/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[True-True-True-True-True] 0.2500ms 40.1471μs 24.9084 KOps/s 25.1794 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[True-True-True-True-False] 0.2150ms 23.0365μs 43.4094 KOps/s 42.5832 KOps/s $\color{#35bf28}+1.94\%$
test_step_mdp_speed[True-True-True-False-True] 97.1220μs 22.4555μs 44.5324 KOps/s 44.1273 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-True-True-False-False] 38.2310μs 12.8132μs 78.0447 KOps/s 76.9015 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[True-True-False-True-True] 77.1410μs 42.3486μs 23.6136 KOps/s 23.7473 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[True-True-False-True-False] 58.6610μs 25.3759μs 39.4075 KOps/s 39.2852 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-True-False-False-True] 50.6710μs 24.5540μs 40.7265 KOps/s 39.8619 KOps/s $\color{#35bf28}+2.17\%$
test_step_mdp_speed[True-True-False-False-False] 42.6410μs 15.2041μs 65.7719 KOps/s 65.0013 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[True-False-True-True-True] 72.5520μs 45.0642μs 22.1905 KOps/s 21.8556 KOps/s $\color{#35bf28}+1.53\%$
test_step_mdp_speed[True-False-True-True-False] 57.7810μs 28.0220μs 35.6862 KOps/s 35.6952 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-False-True-False-True] 51.3810μs 24.7029μs 40.4810 KOps/s 40.3661 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[True-False-True-False-False] 39.1900μs 15.2292μs 65.6632 KOps/s 64.6902 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[True-False-False-True-True] 84.8610μs 47.2137μs 21.1803 KOps/s 21.0682 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[True-False-False-True-False] 62.4710μs 30.2747μs 33.0309 KOps/s 33.0907 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[True-False-False-False-True] 52.5410μs 26.4376μs 37.8249 KOps/s 37.2521 KOps/s $\color{#35bf28}+1.54\%$
test_step_mdp_speed[True-False-False-False-False] 45.6010μs 17.4845μs 57.1936 KOps/s 56.7537 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[False-True-True-True-True] 88.5020μs 44.9081μs 22.2677 KOps/s 21.9894 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[False-True-True-True-False] 55.6710μs 27.9904μs 35.7265 KOps/s 35.8423 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-True-True-False-True] 0.1086ms 28.3342μs 35.2931 KOps/s 34.8572 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-True-True-False-False] 43.1510μs 17.0190μs 58.7580 KOps/s 58.5519 KOps/s $\color{#35bf28}+0.35\%$
test_step_mdp_speed[False-True-False-True-True] 87.6320μs 47.1037μs 21.2298 KOps/s 21.0728 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[False-True-False-True-False] 66.1220μs 30.3630μs 32.9348 KOps/s 33.1986 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[False-True-False-False-True] 3.3559ms 30.8709μs 32.3929 KOps/s 32.3847 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[False-True-False-False-False] 57.6510μs 19.1893μs 52.1124 KOps/s 51.4428 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[False-False-True-True-True] 0.1135ms 49.3091μs 20.2802 KOps/s 20.2782 KOps/s $+0.01\%$
test_step_mdp_speed[False-False-True-True-False] 0.2077ms 32.8006μs 30.4873 KOps/s 30.8268 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[False-False-True-False-True] 61.1310μs 30.3383μs 32.9616 KOps/s 33.1200 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-False-True-False-False] 50.0410μs 19.1364μs 52.2565 KOps/s 52.7619 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[False-False-False-True-True] 88.6220μs 50.4483μs 19.8223 KOps/s 19.5849 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[False-False-False-True-False] 0.1856ms 34.7413μs 28.7842 KOps/s 28.7598 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-False-False-False-True] 0.2490ms 32.2182μs 31.0383 KOps/s 31.2330 KOps/s $\color{#d91a1a}-0.62\%$
test_step_mdp_speed[False-False-False-False-False] 0.2252ms 21.3345μs 46.8724 KOps/s 46.8081 KOps/s $\color{#35bf28}+0.14\%$
test_values[generalized_advantage_estimate-True-True] 25.5613ms 24.8463ms 40.2474 Ops/s 38.9478 Ops/s $\color{#35bf28}+3.34\%$
test_values[vec_generalized_advantage_estimate-True-True] 94.0573ms 2.7793ms 359.7980 Ops/s 334.8014 Ops/s $\textbf{\color{#35bf28}+7.47\%}$
test_values[td0_return_estimate-False-False] 0.1049ms 81.3233μs 12.2966 KOps/s 12.5881 KOps/s $\color{#d91a1a}-2.32\%$
test_values[td1_return_estimate-False-False] 57.0118ms 55.9497ms 17.8732 Ops/s 17.1258 Ops/s $\color{#35bf28}+4.36\%$
test_values[vec_td1_return_estimate-False-False] 1.2804ms 1.0845ms 922.0624 Ops/s 920.6365 Ops/s $\color{#35bf28}+0.15\%$
test_values[td_lambda_return_estimate-True-False] 91.3797ms 88.7979ms 11.2615 Ops/s 10.8281 Ops/s $\color{#35bf28}+4.00\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3938ms 1.0877ms 919.3779 Ops/s 924.4586 Ops/s $\color{#d91a1a}-0.55\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.3616ms 24.8675ms 40.2132 Ops/s 39.9438 Ops/s $\color{#35bf28}+0.67\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.1053ms 0.7628ms 1.3109 KOps/s 1.3169 KOps/s $\color{#d91a1a}-0.45\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8055ms 0.6669ms 1.4994 KOps/s 1.4894 KOps/s $\color{#35bf28}+0.67\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6399ms 1.4791ms 676.0989 Ops/s 677.4531 Ops/s $\color{#d91a1a}-0.20\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8496ms 0.6804ms 1.4697 KOps/s 1.4692 KOps/s $\color{#35bf28}+0.03\%$
test_dqn_speed[False-None] 7.6239ms 1.5311ms 653.1368 Ops/s 654.5016 Ops/s $\color{#d91a1a}-0.21\%$
test_dqn_speed[False-backward] 2.2614ms 2.1314ms 469.1768 Ops/s 469.2553 Ops/s $\color{#d91a1a}-0.02\%$
test_dqn_speed[True-None] 0.7103ms 0.5507ms 1.8159 KOps/s 1.8040 KOps/s $\color{#35bf28}+0.66\%$
test_dqn_speed[True-backward] 1.2821ms 1.2143ms 823.5256 Ops/s 882.4800 Ops/s $\textbf{\color{#d91a1a}-6.68\%}$
test_dqn_speed[reduce-overhead-None] 0.7338ms 0.5650ms 1.7700 KOps/s 1.7342 KOps/s $\color{#35bf28}+2.07\%$
test_dqn_speed[reduce-overhead-backward] 1.2648ms 1.0959ms 912.4951 Ops/s 1.0081 KOps/s $\textbf{\color{#d91a1a}-9.48\%}$
test_ddpg_speed[False-None] 3.1742ms 2.8610ms 349.5305 Ops/s 349.3022 Ops/s $\color{#35bf28}+0.07\%$
test_ddpg_speed[False-backward] 4.7449ms 4.2431ms 235.6771 Ops/s 244.6408 Ops/s $\color{#d91a1a}-3.66\%$
test_ddpg_speed[True-None] 1.2851ms 1.0852ms 921.4762 Ops/s 904.7779 Ops/s $\color{#35bf28}+1.85\%$
test_ddpg_speed[True-backward] 2.5140ms 2.3218ms 430.7000 Ops/s 458.9282 Ops/s $\textbf{\color{#d91a1a}-6.15\%}$
test_ddpg_speed[reduce-overhead-None] 1.2739ms 1.1050ms 904.9858 Ops/s 899.7230 Ops/s $\color{#35bf28}+0.58\%$
test_ddpg_speed[reduce-overhead-backward] 2.0305ms 1.8052ms 553.9550 Ops/s 586.6916 Ops/s $\textbf{\color{#d91a1a}-5.58\%}$
test_sac_speed[False-None] 8.4246ms 8.0195ms 124.6954 Ops/s 120.9346 Ops/s $\color{#35bf28}+3.11\%$
test_sac_speed[False-backward] 11.6675ms 11.2102ms 89.2046 Ops/s 90.3671 Ops/s $\color{#d91a1a}-1.29\%$
test_sac_speed[True-None] 1.7345ms 1.5374ms 650.4604 Ops/s 644.4002 Ops/s $\color{#35bf28}+0.94\%$
test_sac_speed[True-backward] 3.5401ms 3.2641ms 306.3593 Ops/s 291.4031 Ops/s $\textbf{\color{#35bf28}+5.13\%}$
test_sac_speed[reduce-overhead-None] 23.4372ms 12.8313ms 77.9346 Ops/s 77.8444 Ops/s $\color{#35bf28}+0.12\%$
test_sac_speed[reduce-overhead-backward] 1.4250ms 1.3549ms 738.0380 Ops/s 651.3138 Ops/s $\textbf{\color{#35bf28}+13.32\%}$
test_redq_speed[False-None] 8.2559ms 7.5288ms 132.8228 Ops/s 132.8116 Ops/s $+0.01\%$
test_redq_speed[False-backward] 12.1502ms 11.3189ms 88.3481 Ops/s 85.3143 Ops/s $\color{#35bf28}+3.56\%$
test_redq_speed[True-None] 2.1790ms 1.9795ms 505.1757 Ops/s 501.9673 Ops/s $\color{#35bf28}+0.64\%$
test_redq_speed[True-backward] 4.0109ms 3.6225ms 276.0497 Ops/s 257.9001 Ops/s $\textbf{\color{#35bf28}+7.04\%}$
test_redq_speed[reduce-overhead-None] 2.2241ms 2.0024ms 499.4107 Ops/s 497.9214 Ops/s $\color{#35bf28}+0.30\%$
test_redq_speed[reduce-overhead-backward] 4.0788ms 3.6194ms 276.2882 Ops/s 269.6391 Ops/s $\color{#35bf28}+2.47\%$
test_redq_deprec_speed[False-None] 9.6724ms 9.0329ms 110.7059 Ops/s 109.0575 Ops/s $\color{#35bf28}+1.51\%$
test_redq_deprec_speed[False-backward] 12.6664ms 12.0266ms 83.1494 Ops/s 82.2362 Ops/s $\color{#35bf28}+1.11\%$
test_redq_deprec_speed[True-None] 4.2336ms 2.3779ms 420.5367 Ops/s 417.7231 Ops/s $\color{#35bf28}+0.67\%$
test_redq_deprec_speed[True-backward] 4.2087ms 3.9655ms 252.1728 Ops/s 247.6336 Ops/s $\color{#35bf28}+1.83\%$
test_redq_deprec_speed[reduce-overhead-None] 2.5322ms 2.3345ms 428.3608 Ops/s 428.1675 Ops/s $\color{#35bf28}+0.05\%$
test_redq_deprec_speed[reduce-overhead-backward] 4.3774ms 4.0901ms 244.4951 Ops/s 239.0237 Ops/s $\color{#35bf28}+2.29\%$
test_td3_speed[False-None] 8.1591ms 7.9106ms 126.4123 Ops/s 127.7665 Ops/s $\color{#d91a1a}-1.06\%$
test_td3_speed[False-backward] 10.9611ms 10.3071ms 97.0201 Ops/s 96.4254 Ops/s $\color{#35bf28}+0.62\%$
test_td3_speed[True-None] 1.6095ms 1.5784ms 633.5608 Ops/s 608.4389 Ops/s $\color{#35bf28}+4.13\%$
test_td3_speed[True-backward] 3.2494ms 3.1041ms 322.1581 Ops/s 303.0686 Ops/s $\textbf{\color{#35bf28}+6.30\%}$
test_td3_speed[reduce-overhead-None] 86.1339ms 26.8265ms 37.2766 Ops/s 35.3746 Ops/s $\textbf{\color{#35bf28}+5.38\%}$
test_td3_speed[reduce-overhead-backward] 1.4722ms 1.3389ms 746.8566 Ops/s 740.0495 Ops/s $\color{#35bf28}+0.92\%$
test_cql_speed[False-None] 17.5027ms 16.8691ms 59.2801 Ops/s 58.7480 Ops/s $\color{#35bf28}+0.91\%$
test_cql_speed[False-backward] 22.5099ms 22.0105ms 45.4329 Ops/s 45.2979 Ops/s $\color{#35bf28}+0.30\%$
test_cql_speed[True-None] 3.2415ms 2.9333ms 340.9072 Ops/s 339.4758 Ops/s $\color{#35bf28}+0.42\%$
test_cql_speed[True-backward] 5.4831ms 5.3039ms 188.5393 Ops/s 189.2373 Ops/s $\color{#d91a1a}-0.37\%$
test_cql_speed[reduce-overhead-None] 22.2359ms 13.5072ms 74.0344 Ops/s 73.6840 Ops/s $\color{#35bf28}+0.48\%$
test_cql_speed[reduce-overhead-backward] 1.7275ms 1.5637ms 639.5194 Ops/s 646.9741 Ops/s $\color{#d91a1a}-1.15\%$
test_a2c_speed[False-None] 3.5229ms 3.2166ms 310.8896 Ops/s 308.6204 Ops/s $\color{#35bf28}+0.74\%$
test_a2c_speed[False-backward] 6.7253ms 6.0926ms 164.1325 Ops/s 161.7694 Ops/s $\color{#35bf28}+1.46\%$
test_a2c_speed[True-None] 1.3146ms 1.0153ms 984.9022 Ops/s 973.0695 Ops/s $\color{#35bf28}+1.22\%$
test_a2c_speed[True-backward] 3.0929ms 2.6610ms 375.7933 Ops/s 380.7397 Ops/s $\color{#d91a1a}-1.30\%$
test_a2c_speed[reduce-overhead-None] 22.3361ms 11.8436ms 84.4339 Ops/s 86.4423 Ops/s $\color{#d91a1a}-2.32\%$
test_a2c_speed[reduce-overhead-backward] 1.1240ms 0.9945ms 1.0055 KOps/s 991.3803 Ops/s $\color{#35bf28}+1.43\%$
test_ppo_speed[False-None] 4.1028ms 3.7044ms 269.9512 Ops/s 271.5700 Ops/s $\color{#d91a1a}-0.60\%$
test_ppo_speed[False-backward] 7.3521ms 6.8528ms 145.9261 Ops/s 146.7254 Ops/s $\color{#d91a1a}-0.54\%$
test_ppo_speed[True-None] 1.3349ms 0.9571ms 1.0448 KOps/s 1.0463 KOps/s $\color{#d91a1a}-0.14\%$
test_ppo_speed[True-backward] 2.6949ms 2.5439ms 393.0907 Ops/s 364.4391 Ops/s $\textbf{\color{#35bf28}+7.86\%}$
test_ppo_speed[reduce-overhead-None] 0.6852ms 0.5292ms 1.8898 KOps/s 1.8525 KOps/s $\color{#35bf28}+2.01\%$
test_ppo_speed[reduce-overhead-backward] 1.3002ms 1.1365ms 879.8870 Ops/s 851.7885 Ops/s $\color{#35bf28}+3.30\%$
test_reinforce_speed[False-None] 2.5086ms 2.2623ms 442.0263 Ops/s 442.4354 Ops/s $\color{#d91a1a}-0.09\%$
test_reinforce_speed[False-backward] 3.9452ms 3.3900ms 294.9847 Ops/s 297.8728 Ops/s $\color{#d91a1a}-0.97\%$
test_reinforce_speed[True-None] 1.0458ms 0.8402ms 1.1902 KOps/s 1.1653 KOps/s $\color{#35bf28}+2.14\%$
test_reinforce_speed[True-backward] 2.6843ms 2.5596ms 390.6801 Ops/s 383.1759 Ops/s $\color{#35bf28}+1.96\%$
test_reinforce_speed[reduce-overhead-None] 22.6972ms 11.8911ms 84.0965 Ops/s 85.4205 Ops/s $\color{#d91a1a}-1.55\%$
test_reinforce_speed[reduce-overhead-backward] 1.3291ms 1.1898ms 840.4901 Ops/s 932.2964 Ops/s $\textbf{\color{#d91a1a}-9.85\%}$
test_iql_speed[False-None] 9.7030ms 9.2675ms 107.9035 Ops/s 107.9756 Ops/s $\color{#d91a1a}-0.07\%$
test_iql_speed[False-backward] 14.2424ms 13.2999ms 75.1886 Ops/s 77.3216 Ops/s $\color{#d91a1a}-2.76\%$
test_iql_speed[True-None] 2.0722ms 1.7717ms 564.4315 Ops/s 568.5129 Ops/s $\color{#d91a1a}-0.72\%$
test_iql_speed[True-backward] 4.7805ms 4.4295ms 225.7599 Ops/s 223.3637 Ops/s $\color{#35bf28}+1.07\%$
test_iql_speed[reduce-overhead-None] 21.9251ms 11.7607ms 85.0288 Ops/s 86.0470 Ops/s $\color{#d91a1a}-1.18\%$
test_iql_speed[reduce-overhead-backward] 1.7494ms 1.6174ms 618.2796 Ops/s 643.7997 Ops/s $\color{#d91a1a}-3.96\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 8.6262ms 6.5270ms 153.2090 Ops/s 151.2601 Ops/s $\color{#35bf28}+1.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5876ms 0.3024ms 3.3066 KOps/s 3.5447 KOps/s $\textbf{\color{#d91a1a}-6.72\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7381ms 0.2817ms 3.5494 KOps/s 3.8005 KOps/s $\textbf{\color{#d91a1a}-6.61\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.8251ms 6.2569ms 159.8241 Ops/s 158.3614 Ops/s $\color{#35bf28}+0.92\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.2886ms 0.3061ms 3.2671 KOps/s 3.4074 KOps/s $\color{#d91a1a}-4.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7431ms 0.2894ms 3.4555 KOps/s 3.1461 KOps/s $\textbf{\color{#35bf28}+9.84\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6263ms 1.3595ms 735.5470 Ops/s 720.3560 Ops/s $\color{#35bf28}+2.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6147ms 1.2428ms 804.6423 Ops/s 819.2538 Ops/s $\color{#d91a1a}-1.78\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.9124ms 6.4368ms 155.3564 Ops/s 155.6912 Ops/s $\color{#d91a1a}-0.22\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.1427ms 0.4175ms 2.3953 KOps/s 2.0581 KOps/s $\textbf{\color{#35bf28}+16.38\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8226ms 0.4337ms 2.3060 KOps/s 2.5227 KOps/s $\textbf{\color{#d91a1a}-8.59\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.7043ms 6.2869ms 159.0599 Ops/s 158.6371 Ops/s $\color{#35bf28}+0.27\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.0424ms 0.3074ms 3.2530 KOps/s 3.4807 KOps/s $\textbf{\color{#d91a1a}-6.54\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6065ms 0.2930ms 3.4131 KOps/s 3.4877 KOps/s $\color{#d91a1a}-2.14\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.6534ms 6.2328ms 160.4423 Ops/s 158.8526 Ops/s $\color{#35bf28}+1.00\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7385ms 0.2729ms 3.6640 KOps/s 2.7450 KOps/s $\textbf{\color{#35bf28}+33.48\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6850ms 0.2575ms 3.8833 KOps/s 3.0399 KOps/s $\textbf{\color{#35bf28}+27.74\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.7113ms 6.4040ms 156.1519 Ops/s 156.6576 Ops/s $\color{#d91a1a}-0.32\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.0364ms 0.4163ms 2.4020 KOps/s 2.3814 KOps/s $\color{#35bf28}+0.87\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6102ms 0.3960ms 2.5250 KOps/s 2.5146 KOps/s $\color{#35bf28}+0.41\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 7.2732ms 5.5309ms 180.8019 Ops/s 180.8570 Ops/s $\color{#d91a1a}-0.03\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 10.9777ms 2.0913ms 478.1797 Ops/s 434.7700 Ops/s $\textbf{\color{#35bf28}+9.98\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.1712ms 1.1360ms 880.2436 Ops/s 816.8291 Ops/s $\textbf{\color{#35bf28}+7.76\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 9.0832ms 5.6162ms 178.0559 Ops/s 184.4633 Ops/s $\color{#d91a1a}-3.47\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 6.3550ms 1.9913ms 502.1956 Ops/s 435.8702 Ops/s $\textbf{\color{#35bf28}+15.22\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.5468s 12.2183ms 81.8442 Ops/s 846.6668 Ops/s $\textbf{\color{#d91a1a}-90.33\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 7.9751ms 5.7917ms 172.6620 Ops/s 30.7824 Ops/s $\textbf{\color{#35bf28}+460.91\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 9.4082ms 2.1545ms 464.1415 Ops/s 538.6750 Ops/s $\textbf{\color{#d91a1a}-13.84\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 8.4786ms 1.4114ms 708.5347 Ops/s 813.6171 Ops/s $\textbf{\color{#d91a1a}-12.92\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 15.7308ms 15.3224ms 65.2641 Ops/s 64.2209 Ops/s $\color{#35bf28}+1.62\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 20.2547ms 17.9108ms 55.8322 Ops/s 57.1656 Ops/s $\color{#d91a1a}-2.33\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 20.2697ms 19.7976ms 50.5113 Ops/s 47.5123 Ops/s $\textbf{\color{#35bf28}+6.31\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 20.2890ms 17.9920ms 55.5804 Ops/s 54.7992 Ops/s $\color{#35bf28}+1.43\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 20.1356ms 19.7920ms 50.5255 Ops/s 48.8209 Ops/s $\color{#35bf28}+3.49\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.4336ms 19.5349ms 51.1904 Ops/s 51.1978 Ops/s $\color{#d91a1a}-0.01\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot