Skip to content

Recognizing actors in a movie clip or image, using OpenCV, DeepLearning and Python.

Notifications You must be signed in to change notification settings

R1im/Actor-Recognition-In-Movies

 
 

Repository files navigation

Actor Recognition In Movie Clips and Images

ex1

Recognizing actors/celebs in a clip or image from any media, using DeepLearning with Python. Can use either CNN or HOG for face detection and then compare the face with our dataset of faces.

I have used the mostly comprehensive dataset available here. It is only updated with celebrity faces till 2021, so we might need to update it further if required.

Tons of help from ageitgey's face_recognition library.

Inspired by this wonderful article.

Process

  • Setup

    Install cmake, as it is required for the dlib library. For linux, run sudo apt-get install cmake For Windows, download the installer from here For macOS, run brew install cmake

    Also, install pipenv for managing the virtual environment. For linux, run sudo pip install pipenv For Windows, run pip install pipenv For macOS, run brew install pipenv

    Then, run pipenv shell to activate the virtual environment. Finally, run pipenv install to install all the dependencies.

    Note: the face-recognition library does not officially support Windows, but it still might work, as it says in its README

  • Dataset

    The dataset has the following structure.

    screenshot from 2019-01-11 19-19-13

    For my implementation, each actor has 25 images. More will do better, but this number seems to work fine.

  • Training

    For every image in the dataset, we first get a square enclosing the face in the image, then generate a 128d vector for that face, which is dumped to the 'encodings.pickle' file.

    We can either use CNN(slower, more accurate) or HOG(faster, less accurate) for the face detection process. Here I've used the face_recognition library, which gives me both the options.

    For a big dataset, techniques like MapReduce or Spark can be used to parallelize the process over a cluster of machines.

    Moreover, use the -fnn flag in case you want to use the KDTree method for searching, which is much faster than the linear search.

  • Face Recognition

    Consider an image, be it a still from the movie, or a frame of a video clip. First, we identify the faces in the image using the same method as above (CNN or HOG), generate an encoding for it(128d vector), and then compare it with our collected encodings. The actors with the most matched encodings is the actor in the image.

    This search can either be linear, or using a KDTree. I've used the KDTree method, which is much faster. This can be done by passing the -fnn flag to the python file.

Usage

Read the first few lines of the Python file involved to understand the parameters used in each case

  • Making encodings

    python faceEncode.py --dataset dataset/actors --encodings encodings/encodings.pickle -d hog -c 8
    

    -c flag is the number of cores to use for parallel processing.

    Can also use the -fnn flag to later use the KDTree method for searching.

  • Face Recognition in Image

    python faceRecImage.py -e encodings.pickle -i examples/ex6.png -d hog -o out/
    

    Use the -fnn flag to use the KDTree method for searching.

  • Face Recognition in Video File

    python faceRecVideoFile.py -e encodings/encodings.pickle -i input_vids/ex2.mp4 -o output_vids/ex2.avi -y 0 -d hog 
    

    Outputs a video with the faces marked.

Feel free to fork the repository and use it on your own dataset. The encodings/encodings/encodings_fnn_big.pickle file is already trained on a big dataset (1100 celebs with 25 images each), so you can use it directly for face recognition.

About

Recognizing actors in a movie clip or image, using OpenCV, DeepLearning and Python.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%