Final project for my Engineering Python course (CPE 551). A real time finger digit classifier for determining how many fingers are held up on a hand. Uses a convolutional neural network trained on a dataset I created to determine how many fingers are being held up on a hand in an image.
"I pledge my honor that I have abided by the Stevens Honor System" -Ryan O'Shea
- cv2
- numpy
- keras
- tensorflow
- matplotlib
- glob
- skimage
- sklearn
- random
- git lfs (NOT A PYTHON PACKAGE)
There were existing datasets out there for hands holding up different numbers of digits but I couldn't seem to get good results using existing datasets so I decided to generate my own. I created 2000 images for each number of fingers being held up to train my neural network on. I uploaded my dataset to kaggle here (https://www.kaggle.com/roshea6/finger-digits-05) so others could use it. My dataset already has the images segmented to fully isolate the hand in the image to make it more beginner friendly. The dataset was used to train a convolutional neural network using Keras with a Tensorflow back end to create a 6 class classifier network. The classifier network is used to classify how many fingers are held up in a video stream through a laptop camera using OpenCV.
Please run all scripts from the source directory. The code uses relative paths to load models and if you run the script from outside the src directory it will break.
- src
- Contains the python files for the various classes and driver code for the real time classifier. Launch classifier is the main file to actually start the code.
- models
- Contains the models I trained ad a .h5 model. These models can be loaded using the keras load_model function. They are stored both using git lfs and on the google drive listed below.
- training_images
- Contains the images used to train the models in the models folder. These can be used to train your own model. There are 12006 128x128 images of a hand showing the numbers 0 through 5 as fingers held up.
- media
- Contains the demo images and videos for the repository and kaggle dataset.
- Clone this repository
- Install git lfs and activate it for this repository
- OR download the models from my google drive here:
- Install the required packages listed above
- Test any of the functionality you want to by running the file directly
- Make sure your camera is facing a wall or something that will be easy to build an average background with
- Run launch_classifier.py from within the src directory!
- Wait until the words above the box say "Place hand in box"
- Place your hand in the box on the video feed to see what the classifier outputs!
- Run either realtime_classifier.py or model_training.py. Their main functions contain test code to test all the different functions of the classes.
- Ensure that all the functionality is working as intended. There should be printed instructions and images that pop up to make sure everything is working
- If everything looks good then you should be good to go!
I used a large number of online tutorials for specific parts of the code. Primarily the code that dealt with making a good convolutional neural network and the image processing techniques.
- OpenCV - https://opencv.org/
- Tensorflow - https://www.tensorflow.org/
- Keras - https://keras.io/
- pyimagesearch - https://www.pyimagesearch.com/
- Kaggle - https://www.kaggle.com/