Unix |
---|
- A self-contained neural network testing and studying platform.
- It has less than 500 lines of code.
- It is not meant for building a full blown application. You may want to try tiny-dnn for a comprehensive platform.
- Concepts you may learn with this code:
- Forward propagation
- Back propagation
- Gradient descent
- Regularization
- ReLU activation
- Metrics to track convergence
- Effects of floating point precision
- Effects of weight and bias initialization
- Automatic differentiation
- The code is a translation of the Python code available in the excellent and freely available neural network cs231n course by Andrej Karpathy. You are encouraged to check his web-page. It has all the theory in great detail. Also, you may try his original Python code instead.
It segments a group of U=300 points belonging to K=3 classes. There are 100 points in each class.
Data | Segmented Data |
---|---|
This code was developed in Linux (Ubuntu 16.04.1LTS) but it should be easy to build it in any platform with a C++11 compiler. It has no dependencies and just one cpp file and two header files.
- In Linux, type
make
to build the executablepiconn
, which will be then found in thebuild
folder. - To run the executable type
./piconn
The output should look like:
iteration 0: loss 1.098764 data_loss 1.098738 reg_loss 0.000026 training accuracy 33.67%
iteration 1000: loss 0.301150 data_loss 0.195622 reg_loss 0.105528 training accuracy 91.67%
iteration 2000: loss 0.257442 data_loss 0.132855 reg_loss 0.124587 training accuracy 96.33%
iteration 3000: loss 0.249705 data_loss 0.121657 reg_loss 0.128048 training accuracy 97.33%
iteration 4000: loss 0.247016 data_loss 0.117306 reg_loss 0.129711 training accuracy 98.33%
iteration 5000: loss 0.245445 data_loss 0.115239 reg_loss 0.130206 training accuracy 98.67%
iteration 6000: loss 0.244663 data_loss 0.114385 reg_loss 0.130277 training accuracy 98.67%
iteration 7000: loss 0.244248 data_loss 0.113716 reg_loss 0.130532 training accuracy 98.67%
iteration 8000: loss 0.243881 data_loss 0.113107 reg_loss 0.130774 training accuracy 98.67%
iteration 9000: loss 0.243553 data_loss 0.112449 reg_loss 0.131104 training accuracy 99.00%
training accuracy 99.00%
The last line shows that the segmentation was performed with 99% accuracy. The result will not perfectly match the original Python code. This is because of differences in random number generation.
The input was also tested with the support vector machine (SVM) classification method for reference. SVM with radial kernel works pretty well with this particular data (R script included). It has four kernels available (radial basis, linear, polynomial and sigmoid). The results are suboptimal since all kernels were tested using default parameters.
Radial Basis | Linear |
---|---|
Polynomial | Sigmoid |