diff --git a/docs/deep_learning/practical_pytorch/pytorch_feedforward_neuralnetwork.md b/docs/deep_learning/practical_pytorch/pytorch_feedforward_neuralnetwork.md index b7499e6dc..bea287b0f 100755 --- a/docs/deep_learning/practical_pytorch/pytorch_feedforward_neuralnetwork.md +++ b/docs/deep_learning/practical_pytorch/pytorch_feedforward_neuralnetwork.md @@ -86,7 +86,7 @@ LogisticRegressionModel( - A scaled sigmoid function - Input number $\rightarrow$ [-1, 1] - Cons: - 1. Activation saturates at 0 or 1 with **gradients $\approx$ 0** + 1. Activation saturates at -1 or 1 with **gradients $\approx$ 0** - No signal to update weights $\rightarrow$ **cannot learn** - **Solution**: Have to carefully initialize weights to prevent this