8. Future Work

Get Involved!

There are still many ways we hope Signapse can be improved and taken forward by the community, please feel free to make a fork of the project and take things in your own direction. Alternatively, you can suggest features to the team by adding an issue.

Platforms and Deployment

From gauging interaction with our userbase on Social Media, it seems many people are eager for a version of Signapse they could use on an Android or IOS device or a web-based portal. Signpase could easily be modified for running on a server and serving over the web, the video processing pipeline could be modified so that Camera is replaced by a stream from the client's webcam and the Gui element could be replaced with a web-frontend for serving results. Writing a web interface would allow Signapse to become compatible with a wider range of devices while the compute restrictions for the CNN are constrained by the server only; this may allow the use of much larger, more accurate deep networks.

Intelligence

Another feature requested on our Social Media channels is support for more signs and sign language from different countries. To address this, more networks could be trained and research effort could be expended to find out which architectures respond best to learning Sign Language. Our CNN approach should also be checked against CV approaches like vision transformers or maybe a method operating on video stream rather than per-image.

Boosting CNN Throughput

Some additions for future variations on the video processing pipeline are included in our branch future-work. This has mainly involved experimentation with multi-threaded acceleration of the video pipeline CNN execution. An object which intuitively extends SchedulableLink has been written, MultiThreadedSchedulableLink, this allows the latency bound stage to be executed with multiple threads, which can boost CNN execution throughput if the ProcessScene method is guaranteed threadsafe. This feature is still experimental but may come soon to a release of Signapse. We have also experimented with increasing the batch size shown to the OpenCV execution runtime for model inference, it is thought that executing multiple images at once with OpenCV may boost throughput, although this is not yet shown to be the case. Experimentation with different OpenCV back-ends may provide insight in the future. Find out about OpenCV's dnn backends here.