[Bug]: DDPG seems unable to solve the MountainCarContinuous-v0 problem.

[sunweice]

🐛 Bug

import gymnasium as gym

from stable_baselines3 import DDPG

env = gym.make("MountainCarContinuous-v0", render_mode="rgb_array")

model = DDPG("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=10_0000)

vec_env = model.get_env()
obs = vec_env.reset()
for i in range(1000):
action, _state = model.predict(obs, deterministic=True)
obs, reward, done, info = vec_env.step(action)
vec_env.render("human")

The average reward seems unable to exceed 0

To Reproduce

python train.py --algo ...

Relevant log output / Error message

No response

System Info

No response

Checklist

• I have checked that there is no similar issue in the repo
• I have read the SB3 documentation
• I have read the RL Zoo documentation
• I have provided a minimal and working example to reproduce the bug
• I've used the markdown code blocks for both code and stack traces.

[araffin]

hello,
why are you not using the RL Zoo and the tuned hyperparameters?