I believe layer_1_delta should be calculated below
layer_1_input = np.dot(layer_0,weights_0_1)
layer_1_delta = layer_2_delta.dot(weights_1_2.T) * relu2deriv(layer_1_input)
instead of
layer_1 = relu(np.dot(layer_0,weights_0_1))
layer_1_delta = layer_2_delta.dot(weights_1_2.T) * relu2deriv(layer_1)
even though they return the same value.
I believe layer_1_delta should be calculated below
layer_1_input = np.dot(layer_0,weights_0_1)layer_1_delta = layer_2_delta.dot(weights_1_2.T) * relu2deriv(layer_1_input)instead of
layer_1 = relu(np.dot(layer_0,weights_0_1))layer_1_delta = layer_2_delta.dot(weights_1_2.T) * relu2deriv(layer_1)even though they return the same value.