it stuck in selu?

[ouyangzhuzhu]

hi~, i run CartPole-v1 , and it is ok.
But, when i run with other env-name, they all stuck in the same place:

here in model.py , i add some print to help check where they stuck:
def forward(self, inputs):
if self.small_net:
x = selu(self.linear1(inputs))
x = selu(self.linear2(x))
return self.actor_linear(x)
else:
print('model: !!!forward!!! big-net(4conv+1lstm)')
inputs, (hx, cx) = inputs
print('model: !!!after update: input, (hx,cx) = inputs')
x = selu(self.conv1(inputs))
x = selu(self.conv2(x))
x = selu(self.conv3(x))
x = selu(self.conv4(x))
print('model: !!!after 4conv end selu process')
x = x.view(-1, 3233)
print('model: !!!after x reshape: x.view(-1,3233)')
......

and here below is the output of the " python3 main.py --env-name PongDeterministic-v4 --n 10 --lr 0.01 --useAdam" command:
(venv_openai-es) l00221575@F0817-S05:~/venv_openai-es/pytorch-es$ python3 main.py --env-name PongDeterministic-v4 --n 10 --lr 0.01 --useAdam
[2018-10-23 22:23:10,929] Making new env: PongDeterministic-v4
Preprocessing env
Num params in network 588710
/home/l00221575/venv_openai-es/pytorch-es/train.py:50: UserWarning: volatile was removed and now has no effect. Use with torch.no_grad(): instead.
(Variable(state.unsqueeze(0), volatile=True),
model: !!!forward!!! big-net(4conv+1lstm)
model: !!!after update: input, (hx,cx) = inputs
/home/l00221575/venv_openai-es/pytorch-es/train.py:50: UserWarning: volatile was removed and now has no effect. Use with torch.no_grad(): instead.
(Variable(state.unsqueeze(0), volatile=True),
model: !!!forward!!! big-net(4conv+1lstm)
model: !!!after update: input, (hx,cx) = inputs
/home/l00221575/venv_openai-es/pytorch-es/train.py:50: UserWarning: volatile was removed and now has no effect. Use with torch.no_grad(): instead.
(Variable(state.unsqueeze(0), volatile=True),

 
and I guess they stuck in selu, and  i add some print in selu and run PongDeterministic-v4 again, but the output stay the same as above, and other env-name like Kangaroo-ram-v0, Skiing-v0, Freeway-v0 and Gravitar-v0 , they all stuck in the same place like I run PongDeterministic-v4.

Please help~~~

def selu(x):
    print('selu begin')
    alpha = 1.6732632423543772848170429916717
    scale = 1.0507009873554804934193349852946
    print('selu ends')
    return scale * F.elu(x, alpha)