Jetsy

Personal assistant robot that interacts with the human being on a physical and emotional level.

Table of Contents

• What is this?
• Demo
• Requirements
• How to use
• Components
• Hardware Scheme
• Software Architecture
• Software Modules
• Eyes
  • Affirmation
  • Loved
  • Suspicious
  • Angry
  • Happy
  • Normal
  • Sad
• Arms
• Movements
  • Front
  • Left
  • Right
  • Back
• Proximity Sensors
• Command Voice
  • Voice Controller
  • Dance
  • Tell me a joke
  • Object Classification
• Additional Implementations
• Built With
• License
• Use-case
• Amazing Contribution
• How to contribute
• Citing
• Support
• Authors
• Bibliography

What is this?

This project seeks to create an autonomous robot completely focused on emotional interaction with the user. It will be carried out by means of a table assistant with already existing typical functionalities, but thanks to artificial intelligence you will be able to interact with it through voice and video and thus enhance robot-human interaction seeking the maximum fluidity possible.

We also believe that the aesthetic design of the robot is very important to be able to transmit emotions to the user, so it will be very well worked.

Another point that we want is that all software has to be open source, from deep learning models to used libraries. You do not need an internet connection to use most of its features.

Demo

Requirements

For running each sample code:

• Python 3.9.x

• pytorch

• opencv

• pipwin

• pyaudio

• pygame

• torchaudio

• omegaconf

• sox

• noisereduce

• Levenshtein

• fuzzywuzzy

How to use

1. Clone this repo.

git clone https://github.com/juanmacaaz/Jetsy

2. Go to the directory where the code is located.

cd Jesty

3. Install the required libraries.

using pip :

pip install -r Requirements/requirements.txt

4. Execute the following command to run the sample code.

python3 run.py

5. Add star to this repo if you like it 😃.

Components

SainSmart Dysplay	Raspberry Pi Camera	Sg90	GA12-N20 DC
infrared barrier module	infrared barrier module	L298N	Plastic AA Battery Holder
Arduino Uno	Power Bank	ALT-37	USB Microphone
Front cover	Back cover	Arms	Battery Cover
Hardware Scheme Software Architecture Software Modules Eyes Affirmation Loved Suspicious Angry Happy Normal Sad Arms Movements Front Left Right Back Proximity Sensors Frontal Command Voice Voice Controller Dance Tell me a joke Object Classification Additional Implementations Where I am Emotion Detection Built With TinkerCard - Model Design Program. Arduino - IDE used for the development of the servos. VSCode - Code editor to program hardware components. Python - Language used for programming. Adoble Suit - For visual content creation. License This project is under the MIT License - see the LICENSE file for details Use-Case If this project helps your robotics project, please let us know with creating an issue. Amazing Contribution An emotional human-robot interaction never seen before. Table assistant 2.0 equipped with artificial intelligence. An assistant with next-level computer vision. Leaving current trade assistants behind. An easy-to-program framework to add new functionality to the robot. All the code is open-source and does not require the internet to work. How to contribute New estates In order to create new states you can follow these steps: Adds the initial state of the new functionality in the Default state, this is found in state/StateMove.py. class Default(State): next_states = { -1: 'Default', 0: 'Default', 1: 'InitObject', 2: 'InitEmotion', 3: 'InitRepite', 4: 'InitJoke', 5: 'InitWhereAmI' 6: 'Ejemplo1' } def run(self, kwargs): #self.state_machine.global_data['audio'] = None audio_state = self.state_machine.global_data['audio'] kwargs = {} self.go_to(self.next_states[audio_state], kwargs) Create a new voice instruction in VoiceDetector/StringMaching.py inside list FRASES. FRASES = [ ("hola detecta objeto", 1), ... ("hola, esto es un ejemplo", 6) ] Note: that the second value of the tuple corresponds to the identifier of the new state created in step 1. You can now create the different states necessary to carry out the new functionality: class Ejemplo1(State): def run(self, kwargs): kwargs = {} kwargs['variable'] = ‘Vengo del estado uno’ print(f"Inicializando sequencia de test") self.go_to("Ejemplo2", kwargs) class Ejemplo2(State): def run(self, kwargs): kwargs = {} print(f’Mostramos variable {kwargs['variable']}’) print(‘Modificamos variable y creamos otra’) kwargs['variable'] = ‘He pasado por el estado 2’ kwargs['variable2'] = ‘He sido creado por el estado 2’ print(f"Volvemos al estado 1") self.go_to("Ejemplo1", kwargs) Any contribution is welcome!! Contact us via email. Citing If you use this project's code for your academic work or in industry, we'd love to hear from you as well; feel free to reach out to the developers directly. Support Escola d'Enginyeria - UAB Barcelona UAB Open Labs Authors Juan Manuel Camara - juanmacaaz Miguel del Arco - migueldemollet Daniel Suarez - Danny-8 Christian Ferre - christian-ferre Bibliography Controlar un servo con arduino, Luis Llamas, 2016 Modulo sensor de llama, Andrés Cortés, 2021 Tutorial de uso del modulo l298n, Naylamp Mechatronics, 2017 CSI Camera, Jetson Hacks Nano, 2019