[Question] Is it possible to use custom LSTM and transformer models with MlpPolicy ActorCriticPolicy?

[HaakonFlaaronning]

❓ Question

I want to use PPO and A2C with a custom LSTM and transformer network. PPO only have native support for "MlpPolicy" "CnnPolicy" and "MultiInputPolicy". Can I still use "MlpPolicy" but specify a custom LSTM or transformer network, or can it purely be used with networks that only have linear layers? Does it mess up the training if I specify a LSTM network?

Checklist

• I have checked that there is no similar issue in the repo
• I have read the documentation
• If code there is, it is minimal and working
• If code there is, it is formatted using the markdown code blocks for both code and stack traces.

[araffin]

I have checked that there is no similar issue in the repo

Duplicate of #1387, #1077 #177 and Stable-Baselines-Team/stable-baselines3-contrib#165

For RecurrentPPO, please take a look at SB3 contrib (as written in the doc): https://sb3-contrib.readthedocs.io/en/master/modules/ppo_recurrent.html

[Pythoniasm]

I can recommend to use a custom features extractor that does everything you want. However, you might want to adjust the MLP heads for critic and actor accordingly - but still using the MlpPolicy.