Determining the similarity between abstract scenes is a difficult problem in cognitive computing. Humans can look at two images or videos and immediately tell if they are similar, but this intuition is difficult to encode.
We present an application of 2D human pose estimation for accessing similarity between motions in the space of weightlifting. Given input video of a person performing an exercise, our system will access their form. That is, it will give a similarity score between the user's motion and that motion performed "ideally".
Clone the public repository
git clone https://github.com/RFinkelberg/Stance.git
Use the package manager pip to install the dependencies
pip install -r requirements.txt
Download the neural net
python get_models.py
cd stance/
python main.py -f example/squat.mp4
usage: main.py [-h] [-f VIDEO_PATH] [-v] [-p]
optional arguments:
-h, --help show this help message and exit
-f VIDEO_PATH, --video_path VIDEO_PATH
filepath to video containing the user performing a
motion
-v, --verbose Verbosity level. 1 (v) displays info, 2 (vv) displays
debug logs
-p, --use_pickle uses the pickle file corresponding to the video file
given
-s, --save_pickle makes a pickle file corresponding to the video file
given
There are three pickle files each containing precomputed skeletons for each frame of the videos. To use, simply attach a "-p" to the end of the command with the relevant video file.
- example/squat.mp4
- example/squatbad.mp4
- example/squatpoop.mp4
python main.py -f example/squatbad.mp4 -p
If you would like to save the output of the neural net to pickle file so the system spends less time computing the user skeletons of the video in future runs, simply attach a "-s" to the end of the command.
python main.py -f example/squat.mp4 -s
This project is licensed under the MIT License - see the LICENSE.md file for details
For more information on how we developed this project, check out our stance website.