XOR test broke

[copernicus365]

Hello. First off, amazing job! Thanks a million for this well-written code and for the article!

So the problem: Running the XOR test, I get inconsistent results. I changed the code to see different results based on number of epochs for training it, default in the code was 2000, so below results are me rebuilding it every time with an incrementing epoch count starting out 1000.

Note: While I made some big refactors in my own clone mostly for the sake of clarity, when I saw this error I went and re-cloned a version of this repo with no edits made at all, and had the same problem. Also, it looks like I do have the handwritten example working.

Creating neural network...
Train new network, epochs count: 1200

Networks answer for true_1: 0.9692815440274506 which is True
Networks answer for true_2: 0.9690442760246527 which is True
Networks answer for false_1: 0.0321651473684387 which is False
Networks answer for false_2: 0.5007013220216159 which is True

Train new network, epochs count: 1400

Networks answer for true_1: 0.9685193840647762 which is True
Networks answer for true_2: 0.4993329024352548 which is False
Networks answer for false_1: 0.03209017050659973 which is False
Networks answer for false_2: 0.5011027951729532 which is True

Train new network, epochs count: 1600

Networks answer for true_1: 0.9672918464610395 which is True
Networks answer for true_2: 0.9715019693799224 which is True
Networks answer for false_1: 0.0331260208413061 which is False
Networks answer for false_2: 0.028849972181367856 which is False

Train new network, epochs count: 1800

Networks answer for true_1: 0.9701267565700543 which is True
Networks answer for true_2: 0.9770661838514405 which is True
Networks answer for false_1: 0.028995994304228372 which is False
Networks answer for false_2: 0.027506925600888365 which is False

Train new network, epochs count: 2000

Networks answer for true_1: 0.9744159714377177 which is True
Networks answer for true_2: 0.9703829011508596 which is True
Networks answer for false_1: 0.02970457995816969 which is False
Networks answer for false_2: 0.026475506582866536 which is False

Train new network, epochs count: 2200

Networks answer for true_1: 0.9718185400203787 which is True
Networks answer for true_2: 0.9750812820920768 which is True
Networks answer for false_1: 0.02869325363009985 which is False
Networks answer for false_2: 0.0247742819484018 which is False

Train new network, epochs count: 2400

Networks answer for true_1: 0.9757579432657709 which is True
Networks answer for true_2: 0.9751078120864166 which is True
Networks answer for false_1: 0.027656712090314686 which is False
Networks answer for false_2: 0.5004648871902403 which is True

Train new network, epochs count: 2600

Networks answer for true_1: 0.9753137836134012 which is True
Networks answer for true_2: 0.9762910390121077 which is True
Networks answer for false_1: 0.02627798396677237 which is False
Networks answer for false_2: 0.500369817832474 which is True

Train new network, epochs count: 2800

Networks answer for true_1: 0.9725728335491508 which is True
Networks answer for true_2: 0.9778730455181609 which is True
Networks answer for false_1: 0.026051632803920973 which is False
Networks answer for false_2: 0.025979921505266856 which is False

Train new network, epochs count: 3000

Networks answer for true_1: 0.9756331382423711 which is True
Networks answer for true_2: 0.9789445875785799 which is True
Networks answer for false_1: 0.024228957833893304 which is False
Networks answer for false_2: 0.022120057882401706 which is False

Train new network, epochs count: 3200

Networks answer for true_1: 0.967992236012789 which is True
Networks answer for true_2: 0.49851393590212323 which is False
Networks answer for false_1: 0.01862645518904485 which is False
Networks answer for false_2: 0.039309859237290225 which is False

Train new network, epochs count: 3400

Networks answer for true_1: 0.9767192478995378 which is True
Networks answer for true_2: 0.9792451394029276 which is True
Networks answer for false_1: 0.02354968730362952 which is False
Networks answer for false_2: 0.02075132052928394 which is False

Train new network, epochs count: 3600

Networks answer for true_1: 0.9788653427441534 which is True
Networks answer for true_2: 0.9793612093023636 which is True
Networks answer for false_1: 0.022749756512084714 which is False
Networks answer for false_2: 0.500243815949135 which is True

Train new network, epochs count: 3800

Networks answer for true_1: 0.9781994648824733 which is True
Networks answer for true_2: 0.9804575172661157 which is True
Networks answer for false_1: 0.022077032072795794 which is False
Networks answer for false_2: 0.019456010204163664 which is False

Train new network, epochs count: 4000

Networks answer for true_1: 0.9787679309607947 which is True
Networks answer for true_2: 0.9808240362598755 which is True
Networks answer for false_1: 0.02158805405458328 which is False
Networks answer for false_2: 0.018874148223281692 which is False

Train new network, epochs count: 4200

Networks answer for true_1: 0.9823738423792663 which is True
Networks answer for true_2: 0.9823455155851335 which is True
Networks answer for false_1: 0.01822290856635264 which is False
Networks answer for false_2: 0.5001000455718403 which is True

Train new network, epochs count: 4400

Networks answer for true_1: 0.9827133385640879 which is True
Networks answer for true_2: 0.9826941240537013 which is True
Networks answer for false_1: 0.017860931060893637 which is False
Networks answer for false_2: 0.5000858880953554 which is True

Train new network, epochs count: 4600

Networks answer for true_1: 0.9811221835842906 which is True
Networks answer for true_2: 0.9813151645754703 which is True
Networks answer for false_1: 0.020489472166540377 which is False
Networks answer for false_2: 0.5001660593664615 which is True

Train new network, epochs count: 4800

Networks answer for true_1: 0.9800860545711273 which is True
Networks answer for true_2: 0.9819843729163426 which is True
Networks answer for false_1: 0.020176823852188296 which is False
Networks answer for false_2: 0.017845858665023823 which is False

Train new network, epochs count: 5000

Networks answer for true_1: 0.9807921392440822 which is True
Networks answer for true_2: 0.9825300611486817 which is True
Networks answer for false_1: 0.0195093654079766 which is False
Networks answer for false_2: 0.017193875707453118 which is False

Train new network, epochs count: 5200

....

(P.S. Unfortunately I'm still struggling to see how the "magic" works, which was my purpose in experimenting with this. It's just shocking that so little code pulls off working neural network results! I can't say I quite get how the arrays actually got trained. Perhaps I need to find further articles on the power of backpropagation.)