🤙 Update gradient Accumulator multi-gpu aggregation.

dathudeptrai · dathudeptrai · commit b301fbb79aef · 2020-11-22T00:29:43.000+07:00
diff --git a/tensorflow_tts/optimizers/gradient_accumulate.py b/tensorflow_tts/optimizers/gradient_accumulate.py
@@ -60,7 +60,6 @@ def __call__(self, gradients):
                         tf.zeros_like(gradient),
                         trainable=False,
                         synchronization=tf.VariableSynchronization.ON_READ,
-                        aggregation=tf.VariableAggregation.ONLY_FIRST_REPLICA,
                     )
                     if gradient is not None
                     else gradient

Original file line number	Diff line number	Diff line change
`@@ -60,7 +60,6 @@ def __call__(self, gradients):`
`60`	`60`	`tf.zeros_like(gradient),`
`61`	`61`	`trainable=False,`
`62`	`62`	`synchronization=tf.VariableSynchronization.ON_READ,`
`63`		`- aggregation=tf.VariableAggregation.ONLY_FIRST_REPLICA,`
`64`	`63`	`)`
`65`	`64`	`if gradient is not None`
`66`	`65`	`else gradient`