Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix non-tensor passage in _StepMDP #2262

Merged
merged 4 commits into from
Jul 2, 2024
Merged

[BugFix] Fix non-tensor passage in _StepMDP #2262

merged 4 commits into from
Jul 2, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 2, 2024 •
edited
Loading

follow up on #2260

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 2, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2262

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 16 Unrelated Failures

As of commit 3b5e059 with merge base 39462f0 (image):

NEW FAILURE - The following job has failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 2, 2024
@vmoens vmoens added the bug Something isn't working label Jul 2, 2024
@vmoens vmoens merged commit 79fa8bf into main Jul 2, 2024
30 of 47 checks passed
@vmoens vmoens deleted the fix-nontensor-stepmdp branch July 2, 2024 08:51
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 2, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1196s 59.6861ms 16.7543 Ops/s 18.0719 Ops/s $\textbf{\color{#d91a1a}-7.29\%}$
test_sync 41.8159ms 31.7178ms 31.5280 Ops/s 32.5041 Ops/s $\color{#d91a1a}-3.00\%$
test_async 53.9892ms 28.5040ms 35.0827 Ops/s 34.3206 Ops/s $\color{#35bf28}+2.22\%$
test_simple 0.3797s 0.3781s 2.6449 Ops/s 2.6780 Ops/s $\color{#d91a1a}-1.23\%$
test_transformed 0.5356s 0.5320s 1.8797 Ops/s 1.8321 Ops/s $\color{#35bf28}+2.60\%$
test_serial 1.3040s 1.2470s 0.8019 Ops/s 0.7886 Ops/s $\color{#35bf28}+1.70\%$
test_parallel 1.1364s 1.0851s 0.9216 Ops/s 0.9232 Ops/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[True-True-True-True-True] 0.1578ms 22.4761μs 44.4918 KOps/s 44.2217 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[True-True-True-True-False] 37.9110μs 13.2989μs 75.1940 KOps/s 74.8077 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[True-True-True-False-True] 66.4640μs 13.1779μs 75.8845 KOps/s 76.7808 KOps/s $\color{#d91a1a}-1.17\%$
test_step_mdp_speed[True-True-True-False-False] 34.2640μs 7.6969μs 129.9220 KOps/s 128.7935 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[True-True-False-True-True] 57.0170μs 23.7509μs 42.1036 KOps/s 41.2916 KOps/s $\color{#35bf28}+1.97\%$
test_step_mdp_speed[True-True-False-True-False] 36.1670μs 14.5561μs 68.6998 KOps/s 68.6101 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[True-True-False-False-True] 49.3920μs 14.3485μs 69.6935 KOps/s 68.9411 KOps/s $\color{#35bf28}+1.09\%$
test_step_mdp_speed[True-True-False-False-False] 30.0060μs 9.0465μs 110.5398 KOps/s 111.3319 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[True-False-True-True-True] 60.9030μs 25.2439μs 39.6135 KOps/s 39.4146 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[True-False-True-True-False] 47.7690μs 15.9044μs 62.8756 KOps/s 62.4605 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[True-False-True-False-True] 48.8910μs 14.4150μs 69.3721 KOps/s 68.9517 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[True-False-True-False-False] 36.9290μs 9.0061μs 111.0353 KOps/s 111.0626 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-False-False-True-True] 52.1470μs 26.4837μs 37.7591 KOps/s 37.4675 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[True-False-False-True-False] 67.0750μs 17.1350μs 58.3601 KOps/s 57.9228 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[True-False-False-False-True] 51.3860μs 15.4165μs 64.8654 KOps/s 63.7719 KOps/s $\color{#35bf28}+1.71\%$
test_step_mdp_speed[True-False-False-False-False] 30.5470μs 10.1499μs 98.5236 KOps/s 97.7268 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[False-True-True-True-True] 63.3980μs 25.3488μs 39.4496 KOps/s 39.5598 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[False-True-True-True-False] 51.2960μs 16.0193μs 62.4249 KOps/s 61.8310 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[False-True-True-False-True] 42.2680μs 16.8020μs 59.5167 KOps/s 59.2579 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[False-True-True-False-False] 59.9500μs 10.2339μs 97.7148 KOps/s 98.1033 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-True-False-True-True] 56.2640μs 26.3190μs 37.9954 KOps/s 37.4671 KOps/s $\color{#35bf28}+1.41\%$
test_step_mdp_speed[False-True-False-True-False] 52.1170μs 17.0452μs 58.6676 KOps/s 58.2234 KOps/s $\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-True-False-False-True] 39.4230μs 17.9423μs 55.7342 KOps/s 55.9113 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-True-False-False-False] 38.8630μs 11.3983μs 87.7325 KOps/s 87.8085 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[False-False-True-True-True] 52.7590μs 27.6795μs 36.1279 KOps/s 35.8166 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[False-False-True-True-False] 44.1920μs 18.4593μs 54.1733 KOps/s 53.8143 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-False-True-False-True] 46.6070μs 17.9348μs 55.7577 KOps/s 55.7056 KOps/s $\color{#35bf28}+0.09\%$
test_step_mdp_speed[False-False-True-False-False] 38.3320μs 11.3238μs 88.3099 KOps/s 86.5730 KOps/s $\color{#35bf28}+2.01\%$
test_step_mdp_speed[False-False-False-True-True] 40.1050μs 29.0495μs 34.4240 KOps/s 33.9157 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-False-False-True-False] 44.6130μs 19.5687μs 51.1021 KOps/s 50.5259 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[False-False-False-False-True] 41.3170μs 18.7723μs 53.2701 KOps/s 53.0796 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-False-False-False] 40.5360μs 12.4646μs 80.2273 KOps/s 79.7348 KOps/s $\color{#35bf28}+0.62\%$
test_values[generalized_advantage_estimate-True-True] 11.1917ms 9.7697ms 102.3576 Ops/s 103.3322 Ops/s $\color{#d91a1a}-0.94\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.9192ms 35.2319ms 28.3834 Ops/s 27.9297 Ops/s $\color{#35bf28}+1.62\%$
test_values[td0_return_estimate-False-False] 0.2396ms 0.1664ms 6.0108 KOps/s 6.0570 KOps/s $\color{#d91a1a}-0.76\%$
test_values[td1_return_estimate-False-False] 25.8330ms 24.1286ms 41.4446 Ops/s 42.6380 Ops/s $\color{#d91a1a}-2.80\%$
test_values[vec_td1_return_estimate-False-False] 38.3449ms 35.4434ms 28.2140 Ops/s 28.2804 Ops/s $\color{#d91a1a}-0.23\%$
test_values[td_lambda_return_estimate-True-False] 35.8490ms 34.6608ms 28.8510 Ops/s 29.4852 Ops/s $\color{#d91a1a}-2.15\%$
test_values[vec_td_lambda_return_estimate-True-False] 52.3876ms 36.0314ms 27.7536 Ops/s 28.2273 Ops/s $\color{#d91a1a}-1.68\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 10.8839ms 8.5162ms 117.4237 Ops/s 117.8969 Ops/s $\color{#d91a1a}-0.40\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 3.1340ms 2.2666ms 441.1832 Ops/s 491.8495 Ops/s $\textbf{\color{#d91a1a}-10.30\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5704ms 0.3560ms 2.8089 KOps/s 2.7787 KOps/s $\color{#35bf28}+1.09\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.1351ms 46.4985ms 21.5061 Ops/s 21.4184 Ops/s $\color{#35bf28}+0.41\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.4149ms 3.0305ms 329.9732 Ops/s 328.5353 Ops/s $\color{#35bf28}+0.44\%$
test_dqn_speed 1.7319ms 1.3347ms 749.2226 Ops/s 743.3229 Ops/s $\color{#35bf28}+0.79\%$
test_ddpg_speed 3.6098ms 2.8286ms 353.5268 Ops/s 352.6461 Ops/s $\color{#35bf28}+0.25\%$
test_sac_speed 10.0331ms 8.4486ms 118.3623 Ops/s 117.6941 Ops/s $\color{#35bf28}+0.57\%$
test_redq_speed 15.3618ms 13.6091ms 73.4801 Ops/s 72.8697 Ops/s $\color{#35bf28}+0.84\%$
test_redq_deprec_speed 15.1748ms 13.3338ms 74.9974 Ops/s 67.8762 Ops/s $\textbf{\color{#35bf28}+10.49\%}$
test_td3_speed 17.3419ms 8.4618ms 118.1783 Ops/s 118.2437 Ops/s $\color{#d91a1a}-0.06\%$
test_cql_speed 38.6158ms 37.1390ms 26.9259 Ops/s 26.8793 Ops/s $\color{#35bf28}+0.17\%$
test_a2c_speed 8.2000ms 7.5119ms 133.1227 Ops/s 133.6776 Ops/s $\color{#d91a1a}-0.42\%$
test_ppo_speed 8.8276ms 7.7834ms 128.4782 Ops/s 128.4907 Ops/s $-0.01\%$
test_reinforce_speed 7.3323ms 6.8118ms 146.8031 Ops/s 148.8907 Ops/s $\color{#d91a1a}-1.40\%$
test_iql_speed 35.0114ms 33.3984ms 29.9416 Ops/s 30.0722 Ops/s $\color{#d91a1a}-0.43\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.5055ms 3.5390ms 282.5664 Ops/s 278.1836 Ops/s $\color{#35bf28}+1.58\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0102ms 0.5143ms 1.9443 KOps/s 1.7786 KOps/s $\textbf{\color{#35bf28}+9.31\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8089ms 0.4703ms 2.1264 KOps/s 2.1131 KOps/s $\color{#35bf28}+0.63\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8413ms 3.4780ms 287.5202 Ops/s 284.0612 Ops/s $\color{#35bf28}+1.22\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0987ms 0.4905ms 2.0389 KOps/s 2.0513 KOps/s $\color{#d91a1a}-0.60\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7440ms 0.4706ms 2.1251 KOps/s 2.1207 KOps/s $\color{#35bf28}+0.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.0312ms 1.7209ms 581.0915 Ops/s 579.1739 Ops/s $\color{#35bf28}+0.33\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 4.7091ms 1.6436ms 608.4303 Ops/s 612.8945 Ops/s $\color{#d91a1a}-0.73\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.5814ms 3.6851ms 271.3625 Ops/s 273.8588 Ops/s $\color{#d91a1a}-0.91\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9806ms 0.6332ms 1.5792 KOps/s 1.5869 KOps/s $\color{#d91a1a}-0.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9810ms 0.6071ms 1.6472 KOps/s 1.6568 KOps/s $\color{#d91a1a}-0.58\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.8361ms 3.6495ms 274.0073 Ops/s 281.6982 Ops/s $\color{#d91a1a}-2.73\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5979ms 0.4959ms 2.0165 KOps/s 1.9958 KOps/s $\color{#35bf28}+1.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7567ms 0.4764ms 2.0992 KOps/s 2.1142 KOps/s $\color{#d91a1a}-0.71\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.7923ms 3.5134ms 284.6273 Ops/s 281.5313 Ops/s $\color{#35bf28}+1.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1393ms 0.4918ms 2.0335 KOps/s 2.0439 KOps/s $\color{#d91a1a}-0.50\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8376ms 0.4746ms 2.1072 KOps/s 2.1035 KOps/s $\color{#35bf28}+0.18\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.0686ms 3.7145ms 269.2187 Ops/s 274.5295 Ops/s $\color{#d91a1a}-1.93\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7489ms 0.6332ms 1.5793 KOps/s 1.5772 KOps/s $\color{#35bf28}+0.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.8330ms 0.6129ms 1.6315 KOps/s 1.6418 KOps/s $\color{#d91a1a}-0.63\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1283s 6.3449ms 157.6075 Ops/s 123.2663 Ops/s $\textbf{\color{#35bf28}+27.86\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.9133ms 12.6013ms 79.3568 Ops/s 80.5324 Ops/s $\color{#d91a1a}-1.46\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.6758ms 1.1015ms 907.8508 Ops/s 871.2174 Ops/s $\color{#35bf28}+4.20\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1171s 8.2730ms 120.8748 Ops/s 170.9686 Ops/s $\textbf{\color{#d91a1a}-29.30\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.2054ms 12.6837ms 78.8416 Ops/s 80.1092 Ops/s $\color{#d91a1a}-1.58\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.7723ms 1.0651ms 938.8720 Ops/s 902.6991 Ops/s $\color{#35bf28}+4.01\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1241s 6.3186ms 158.2623 Ops/s 164.8073 Ops/s $\color{#d91a1a}-3.97\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 20.3105ms 12.9560ms 77.1844 Ops/s 78.8988 Ops/s $\color{#d91a1a}-2.17\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.8741ms 1.2213ms 818.7921 Ops/s 814.9078 Ops/s $\color{#35bf28}+0.48\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 2, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1624s 0.1096s 9.1230 Ops/s 9.6847 Ops/s $\textbf{\color{#d91a1a}-5.80\%}$
test_sync 91.7233ms 88.3603ms 11.3173 Ops/s 11.2766 Ops/s $\color{#35bf28}+0.36\%$
test_async 0.1711s 85.5422ms 11.6901 Ops/s 11.5519 Ops/s $\color{#35bf28}+1.20\%$
test_single_pixels 0.1139s 0.1137s 8.7958 Ops/s 8.7441 Ops/s $\color{#35bf28}+0.59\%$
test_sync_pixels 69.9243ms 68.3983ms 14.6202 Ops/s 14.6760 Ops/s $\color{#d91a1a}-0.38\%$
test_async_pixels 0.1197s 56.3438ms 17.7482 Ops/s 17.4925 Ops/s $\color{#35bf28}+1.46\%$
test_simple 0.7273s 0.7267s 1.3761 Ops/s 1.3457 Ops/s $\color{#35bf28}+2.26\%$
test_transformed 1.0576s 0.9919s 1.0081 Ops/s 1.0220 Ops/s $\color{#d91a1a}-1.36\%$
test_serial 2.2706s 2.2022s 0.4541 Ops/s 0.4607 Ops/s $\color{#d91a1a}-1.43\%$
test_parallel 1.9455s 1.8646s 0.5363 Ops/s 0.5390 Ops/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[True-True-True-True-True] 69.1110μs 34.1463μs 29.2858 KOps/s 29.5996 KOps/s $\color{#d91a1a}-1.06\%$
test_step_mdp_speed[True-True-True-True-False] 49.8410μs 19.8272μs 50.4358 KOps/s 51.7770 KOps/s $\color{#d91a1a}-2.59\%$
test_step_mdp_speed[True-True-True-False-True] 51.5210μs 19.6863μs 50.7967 KOps/s 53.0211 KOps/s $\color{#d91a1a}-4.20\%$
test_step_mdp_speed[True-True-True-False-False] 34.7310μs 11.3498μs 88.1072 KOps/s 90.8226 KOps/s $\color{#d91a1a}-2.99\%$
test_step_mdp_speed[True-True-False-True-True] 62.0510μs 36.8445μs 27.1411 KOps/s 28.8085 KOps/s $\textbf{\color{#d91a1a}-5.79\%}$
test_step_mdp_speed[True-True-False-True-False] 50.0210μs 21.6528μs 46.1834 KOps/s 47.7539 KOps/s $\color{#d91a1a}-3.29\%$
test_step_mdp_speed[True-True-False-False-True] 39.3510μs 21.5402μs 46.4249 KOps/s 48.8554 KOps/s $\color{#d91a1a}-4.97\%$
test_step_mdp_speed[True-True-False-False-False] 38.3710μs 13.0772μs 76.4687 KOps/s 78.2276 KOps/s $\color{#d91a1a}-2.25\%$
test_step_mdp_speed[True-False-True-True-True] 56.4610μs 38.4514μs 26.0069 KOps/s 27.0339 KOps/s $\color{#d91a1a}-3.80\%$
test_step_mdp_speed[True-False-True-True-False] 57.9710μs 23.5307μs 42.4978 KOps/s 43.7952 KOps/s $\color{#d91a1a}-2.96\%$
test_step_mdp_speed[True-False-True-False-True] 47.6210μs 21.6185μs 46.2567 KOps/s 48.8715 KOps/s $\textbf{\color{#d91a1a}-5.35\%}$
test_step_mdp_speed[True-False-True-False-False] 29.5100μs 13.1373μs 76.1189 KOps/s 78.6494 KOps/s $\color{#d91a1a}-3.22\%$
test_step_mdp_speed[True-False-False-True-True] 57.7610μs 40.2770μs 24.8280 KOps/s 26.2869 KOps/s $\textbf{\color{#d91a1a}-5.55\%}$
test_step_mdp_speed[True-False-False-True-False] 41.4100μs 25.3574μs 39.4363 KOps/s 40.4949 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[True-False-False-False-True] 73.4620μs 23.4015μs 42.7323 KOps/s 45.6288 KOps/s $\textbf{\color{#d91a1a}-6.35\%}$
test_step_mdp_speed[True-False-False-False-False] 68.9710μs 14.9376μs 66.9454 KOps/s 69.7740 KOps/s $\color{#d91a1a}-4.05\%$
test_step_mdp_speed[False-True-True-True-True] 66.0910μs 38.5473μs 25.9421 KOps/s 27.4342 KOps/s $\textbf{\color{#d91a1a}-5.44\%}$
test_step_mdp_speed[False-True-True-True-False] 40.6700μs 23.7341μs 42.1335 KOps/s 43.7114 KOps/s $\color{#d91a1a}-3.61\%$
test_step_mdp_speed[False-True-True-False-True] 43.3910μs 25.2312μs 39.6335 KOps/s 41.2033 KOps/s $\color{#d91a1a}-3.81\%$
test_step_mdp_speed[False-True-True-False-False] 31.9100μs 14.8651μs 67.2718 KOps/s 69.4411 KOps/s $\color{#d91a1a}-3.12\%$
test_step_mdp_speed[False-True-False-True-True] 67.7210μs 40.2466μs 24.8468 KOps/s 26.2818 KOps/s $\textbf{\color{#d91a1a}-5.46\%}$
test_step_mdp_speed[False-True-False-True-False] 49.1210μs 25.2697μs 39.5731 KOps/s 40.8416 KOps/s $\color{#d91a1a}-3.11\%$
test_step_mdp_speed[False-True-False-False-True] 45.7110μs 27.2412μs 36.7090 KOps/s 38.3898 KOps/s $\color{#d91a1a}-4.38\%$
test_step_mdp_speed[False-True-False-False-False] 33.2500μs 16.7161μs 59.8227 KOps/s 61.7816 KOps/s $\color{#d91a1a}-3.17\%$
test_step_mdp_speed[False-False-True-True-True] 0.2006ms 42.1587μs 23.7199 KOps/s 25.2139 KOps/s $\textbf{\color{#d91a1a}-5.93\%}$
test_step_mdp_speed[False-False-True-True-False] 51.3200μs 27.7756μs 36.0028 KOps/s 37.8600 KOps/s $\color{#d91a1a}-4.91\%$
test_step_mdp_speed[False-False-True-False-True] 51.9610μs 27.4632μs 36.4124 KOps/s 38.7346 KOps/s $\textbf{\color{#d91a1a}-6.00\%}$
test_step_mdp_speed[False-False-True-False-False] 33.2710μs 16.7247μs 59.7918 KOps/s 62.0086 KOps/s $\color{#d91a1a}-3.58\%$
test_step_mdp_speed[False-False-False-True-True] 77.4410μs 44.3146μs 22.5660 KOps/s 23.4030 KOps/s $\color{#d91a1a}-3.58\%$
test_step_mdp_speed[False-False-False-True-False] 52.4710μs 29.5527μs 33.8379 KOps/s 35.2299 KOps/s $\color{#d91a1a}-3.95\%$
test_step_mdp_speed[False-False-False-False-True] 60.2610μs 28.5639μs 35.0093 KOps/s 36.2355 KOps/s $\color{#d91a1a}-3.38\%$
test_step_mdp_speed[False-False-False-False-False] 42.4910μs 18.5817μs 53.8164 KOps/s 55.4904 KOps/s $\color{#d91a1a}-3.02\%$
test_values[generalized_advantage_estimate-True-True] 24.3059ms 23.8802ms 41.8756 Ops/s 41.4200 Ops/s $\color{#35bf28}+1.10\%$
test_values[vec_generalized_advantage_estimate-True-True] 90.1319ms 2.7012ms 370.2061 Ops/s 375.6475 Ops/s $\color{#d91a1a}-1.45\%$
test_values[td0_return_estimate-False-False] 91.3420μs 65.9864μs 15.1546 KOps/s 15.0087 KOps/s $\color{#35bf28}+0.97\%$
test_values[td1_return_estimate-False-False] 54.0447ms 53.7496ms 18.6048 Ops/s 18.4992 Ops/s $\color{#35bf28}+0.57\%$
test_values[vec_td1_return_estimate-False-False] 1.3559ms 1.0822ms 924.0116 Ops/s 925.4512 Ops/s $\color{#d91a1a}-0.16\%$
test_values[td_lambda_return_estimate-True-False] 91.8395ms 86.2820ms 11.5899 Ops/s 11.6204 Ops/s $\color{#d91a1a}-0.26\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.4188ms 1.0827ms 923.6467 Ops/s 926.7681 Ops/s $\color{#d91a1a}-0.34\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 26.1639ms 25.3757ms 39.4077 Ops/s 40.9831 Ops/s $\color{#d91a1a}-3.84\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9545ms 0.7116ms 1.4052 KOps/s 1.4074 KOps/s $\color{#d91a1a}-0.16\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7445ms 0.6646ms 1.5047 KOps/s 1.5147 KOps/s $\color{#d91a1a}-0.66\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4954ms 1.4626ms 683.6936 Ops/s 684.6861 Ops/s $\color{#d91a1a}-0.14\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7256ms 0.6793ms 1.4722 KOps/s 1.4823 KOps/s $\color{#d91a1a}-0.68\%$
test_dqn_speed 7.7477ms 1.4629ms 683.5629 Ops/s 691.7536 Ops/s $\color{#d91a1a}-1.18\%$
test_ddpg_speed 3.1787ms 2.9413ms 339.9857 Ops/s 343.0522 Ops/s $\color{#d91a1a}-0.89\%$
test_sac_speed 9.2849ms 8.4116ms 118.8839 Ops/s 119.4424 Ops/s $\color{#d91a1a}-0.47\%$
test_redq_speed 0.1036s 11.6076ms 86.1504 Ops/s 94.4188 Ops/s $\textbf{\color{#d91a1a}-8.76\%}$
test_redq_deprec_speed 12.0607ms 11.3994ms 87.7241 Ops/s 79.0638 Ops/s $\textbf{\color{#35bf28}+10.95\%}$
test_td3_speed 8.4700ms 8.3456ms 119.8241 Ops/s 119.8804 Ops/s $\color{#d91a1a}-0.05\%$
test_cql_speed 26.6621ms 25.6975ms 38.9143 Ops/s 39.0957 Ops/s $\color{#d91a1a}-0.46\%$
test_a2c_speed 6.4382ms 5.6280ms 177.6840 Ops/s 177.6682 Ops/s $+0.01\%$
test_ppo_speed 6.5367ms 5.9335ms 168.5349 Ops/s 166.7690 Ops/s $\color{#35bf28}+1.06\%$
test_reinforce_speed 4.8022ms 4.6298ms 215.9936 Ops/s 215.1430 Ops/s $\color{#35bf28}+0.40\%$
test_iql_speed 20.2516ms 19.4168ms 51.5019 Ops/s 51.1620 Ops/s $\color{#35bf28}+0.66\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.8277ms 4.6691ms 214.1747 Ops/s 216.4766 Ops/s $\color{#d91a1a}-1.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.1082s 0.6175ms 1.6193 KOps/s 1.8622 KOps/s $\textbf{\color{#d91a1a}-13.04\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7853ms 0.5153ms 1.9406 KOps/s 1.9437 KOps/s $\color{#d91a1a}-0.16\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.7920ms 4.6202ms 216.4392 Ops/s 215.9365 Ops/s $\color{#35bf28}+0.23\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6646ms 0.5333ms 1.8753 KOps/s 1.8689 KOps/s $\color{#35bf28}+0.34\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.6924ms 0.5156ms 1.9395 KOps/s 1.9521 KOps/s $\color{#d91a1a}-0.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1390ms 1.9276ms 518.7723 Ops/s 513.2364 Ops/s $\color{#35bf28}+1.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 5.9279ms 1.8450ms 542.0189 Ops/s 544.0627 Ops/s $\color{#d91a1a}-0.38\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.9605ms 4.8065ms 208.0533 Ops/s 209.7619 Ops/s $\color{#d91a1a}-0.81\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7237ms 0.6862ms 1.4573 KOps/s 1.4733 KOps/s $\color{#d91a1a}-1.09\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8397ms 0.6650ms 1.5037 KOps/s 1.5191 KOps/s $\color{#d91a1a}-1.02\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.8611ms 4.6707ms 214.0984 Ops/s 215.8393 Ops/s $\color{#d91a1a}-0.81\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.6222ms 0.5381ms 1.8584 KOps/s 1.8544 KOps/s $\color{#35bf28}+0.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6772ms 0.5197ms 1.9243 KOps/s 1.9130 KOps/s $\color{#35bf28}+0.59\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.8192ms 4.6176ms 216.5628 Ops/s 217.0998 Ops/s $\color{#d91a1a}-0.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6613ms 0.5319ms 1.8801 KOps/s 1.8569 KOps/s $\color{#35bf28}+1.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 5.0933ms 0.5198ms 1.9238 KOps/s 1.9435 KOps/s $\color{#d91a1a}-1.02\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.9655ms 4.8449ms 206.4014 Ops/s 207.8397 Ops/s $\color{#d91a1a}-0.69\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8009ms 0.6876ms 1.4543 KOps/s 1.4652 KOps/s $\color{#d91a1a}-0.75\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8028ms 0.6636ms 1.5070 KOps/s 1.4922 KOps/s $\color{#35bf28}+0.99\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1422s 7.5461ms 132.5189 Ops/s 138.9337 Ops/s $\color{#d91a1a}-4.62\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.0998ms 15.2750ms 65.4664 Ops/s 64.9335 Ops/s $\color{#35bf28}+0.82\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.4254ms 1.1336ms 882.1151 Ops/s 880.0233 Ops/s $\color{#35bf28}+0.24\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1257s 9.6057ms 104.1053 Ops/s 137.5298 Ops/s $\textbf{\color{#d91a1a}-24.30\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.8301ms 15.4354ms 64.7860 Ops/s 56.5203 Ops/s $\textbf{\color{#35bf28}+14.62\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.2278ms 1.1179ms 894.4955 Ops/s 862.6461 Ops/s $\color{#35bf28}+3.69\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1250s 7.3087ms 136.8228 Ops/s 134.0514 Ops/s $\color{#35bf28}+2.07\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.9781ms 15.5451ms 64.3289 Ops/s 64.9573 Ops/s $\color{#d91a1a}-0.97\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.4294ms 1.3852ms 721.9116 Ops/s 747.4698 Ops/s $\color{#d91a1a}-3.42\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot