Object Detection

Updated merges with Master as of May 22, 2020

Current State: There are currently separate functions made for image cleaning, cropping, color detection, shape detection, and letter detection.

Next Steps: We will combine the image cleaning with color detection, and cropping with shape detection before removing the image cleaning branch. Shape detection and letter detection will work on using Keras to increase accuracy.

# Image Cleaning

Current State: Cable to remove small markings from an image with a shape. (isolates the shape). Stil having some trouble with identiying larged bodies since they are disconnected pixels.

Latest Change: Using a dictionary to calculate the number of pixels (using highest R or G or B value)

How it works:

We will take advantage of a supervised learning technique known as KNearestNeighbor (KNN)

Step 1: Pass image to morphology library

Filters the uneccessary pixels but is unable to detect larger bodies

Kernel can be manipulated to have a threshold for size but can be innaccurate the smaller the threshold

Step 2: Pass Pixels into dictionary"

Counts the number of highest taken RGB value for each pixel and places into a dictionary

Filter the values with a number greater than 600 and smaller than 200

Step 2: Delete Pixels"

Iterate through the pixels and store the ones that are beyond the threshold.

Shape Detection

Current State: Prepares images for shape detection, utilizes basic edge counting for identifying shapes

Latest Change: Crops images to prepare for shape detection

Next Goals: Implement KNN, implement a shape detector to parse through all images return by color detector

How it works:

Currently uses tools in the OpenCV library, planning to implement KNearestNeighbor (KNN)

Step 1: Fill in the hole left by the letter

Reads selected image from color detection and converts to grayscale

Threshold the image to change to black and white

Floodfills the image and inverts to pull out the letter hole

Overlay the two images and invert

Save the new filled in image in filled folder

Step 2: Crop the Shape

Crops the shape using a bounding recetangle around each contour detected

Save the cropped shapes in cropped folder

Note: works for images that may have more than one shape

Step 3: Shape Detection

Detects the corners and edges of the shapes

Classifies the shape based on the number of corners and edges

Note: only works for a very limited amount of shapes,so need to improve that

Letter Recognition

Current State: Can recognize numbers and letters but we need to figure out how to make it more accurate

Latest Change: Now using HOG Features instead of pixel value features for improved accuracy

How it works:

Step 1: Collect Data & Save It

Read from the thousands of training images stored in our TrainingImages folder

Extract HOG features from these training images

Organize image features into 1d array and create a corresponding 1d label array

Save the training data in .npz file for easy and faster access later

Note: Training can take up to 3 minutes depending on the number of training images chosen

Step 2: Train KNN "Model"

Load the training information stored in .npz file

Create a KNN object and train it (built in OpenCV class and methods)

Step 3: Reap the reward

Once the training is done, we can read our test image, extract its HOG features, and find its nearest feature neighbors (built in OpenCV method)

Example Input Image:

Example Input HOG Features:

Example Output: