Skip to content

dropout和batchnorm层已经传了training,outputs._uses_learning_phase = training is not None的作用是什么? #506

@Daemoonn

Description

@Daemoonn

Describe the question(问题描述)
既然DNN中的bn_layers和dropout_layers已经传入了training状态参数来区分是在训练还是在预测阶段,手动设置_uses_learning_phase的作用是什么?
我的tf版本2.10.0,应该是执行outputs._uses_learning_phase = training is not None

attention_score = self.local_att([queries, keys], training=training)
outputs = tf.transpose(attention_score, (0, 2, 1))
if self.weight_normalization:
paddings = tf.ones_like(outputs) * (-2 ** 32 + 1)
else:
paddings = tf.zeros_like(outputs)
outputs = tf.where(key_masks, outputs, paddings)
if self.weight_normalization:
outputs = softmax(outputs)
if not self.return_score:
outputs = tf.matmul(outputs, keys)
if tf.__version__ < '1.13.0':
outputs._uses_learning_phase = attention_score._uses_learning_phase
else:
outputs._uses_learning_phase = training is not None
return outputs

在求attention_score的时候,不是已经把training状态传给DNN里的dropout和batchnorm层了吗?

attention_score = self.local_att([queries, keys], training=training)

att_out = self.dnn(att_input, training=training)

fc = self.bn_layers[i](fc, training=training)

fc = self.dropout_layers[i](fc, training=training)

为什么还需要对_uses_learning_phase手动设置?

if tf.__version__ < '1.13.0':
outputs._uses_learning_phase = attention_score._uses_learning_phase
else:
outputs._uses_learning_phase = training is not None

这里手动设置_uses_learning_phase是否发挥实际作用?去掉会怎么样?对于outputs这个tf.Tensor来说,好像是没有_uses_learning_phase这个类属性,这么写好像是临时新建个了_uses_learning_phase属性,然后赋值成xxx

outputs._uses_learning_phase = xxx

求大佬指点

Additional context
Add any other context about the problem here.

Operating environment(运行环境):

  • python version 3.9.12
  • tensorflow version 2.10.0
  • deepctr version 0.9.3

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions