Skip to content

Latest commit

 

History

History
87 lines (65 loc) · 3.7 KB

README.md

File metadata and controls

87 lines (65 loc) · 3.7 KB
Raw

DeepSign: A Deep Learning Architecture for Sign-Language-Recognition

DeepSign is a new deep-learning architecture which achieves comparable results with limited training data for Sign Language Recognition.

Paper Link

https://rc.library.uta.edu/uta-ir/bitstream/handle/10106/27803/SHAH-THESIS-2018.pdf?sequence=1&isAllowed=y

Medium Post

https://medium.com/@jayshah_84248/deepsign-a-deep-learning-pipeline-for-sign-language-recognition-a51a8f116dfc

core:

  • This is the folder where the training code is and the testing code is.
  • It has following files:

    1. train_auto_encoder_1.py

      • This file consist the code which trains the auto-encoder.
      • This auto encoder is Model-1.

    2. train_bi_lstm.py

      • This file consist the code which trains the bi-directional LSTM.
      • This bi-directio2nal LSTM is Model-2.
      • This file also loads the Model-1 and only takes output from enocder of Model-1 to bi-directional LSTM.

    3. train_lstm.py

      • This file consist the code which trains the uni-directional LSTM.
      • This uni-directio2nal LSTM is Model-2.
      • This file also loads the Model-1 and only takes output from enocder of Model-1 to uni-directional LSTM.

    4. train_vae.py

      • This file consist the code which trains the variational auto-encoder.
      • This auto encoder is Model-1.

    5. test_bi_lstm.py

      • This file consists of the code which does inference of bi-directional LSTM.
      • This file loads the freezed model and does predictions on test data.

    6. test_lstm.py

      • This file consists of the code which does inference of uni-directional LSTM.
      • This file loads the freezed model and does predictions on test data.

models:

  1. auto_encoder_1.py

    • This file consist the architecture of auto-encoder.
    • This auto-encoder is Model-1.
    • It is 10 Layered encoder and 15 layered decoder.
    • The file also defines the cost function and the optimizer.
    • This file is used by train_auto_encoder_1.py of core module.

  2. bi_lstm.py

    • This file consist the architecture of bi-directional lstm.
    • This bi-directional lstm is Model-2.
    • The file also defines the cost function and the optimizer.
    • This file is used by train_bi_lstm.py of core module.

  3. lstm.py

    • This file consist the architecture of uni-directional lstm.
    • This bi-directional lstm is Model-2.
    • The file also defines the cost function and the optimizer.
    • This file is used by train_lstm.py of core module.

  4. vae.py

    • This file consist the architecture of auto-encoder.
    • This variational auto-encoder is Model-1.
    • The file also defines the cost function and the optimizer.
    • This file is used by train_auto_encoder_1.py of core module.

utils:

  1. constants.py

    • This file consists of constant.py
    • It has defined path to data folder and models
    • Only this file needs to be changed if you want to use a custom path with in the project

  2. cv_utils.py

    • contains all the OPENCV functions that are commonly used by files in the core module.
    • functions like reading frames, converting image to black and white, resizinng video frame.

  3. os_utils.py

    • contains all the os module functions that are commonly used by files in the core module.
    • functions like iteratng a directory, creating a folder, joining paths.

  4. utility.py

    • contains all the functions that are commonly used by files in the core module.
    • functions like freeze_model, prepare_batch_frames_from_bg_data, load_a_frozen_model.