Ripple Live

RippleLive is the ultimate virtual karaoke experience designed to faithfully emulate the atmosphere of a real karaoke room. This web app brings the authenticity of in-person karaoke to the digital realm, requiring participants to use both a face cam and microphone for the most immersive experience. With the capacity to host up to four individuals per room, RippleLive facilitates a genuine sense of camaraderie and interaction, where users can see and hear each other in real-time. The app integrates a lively chat feature, allowing participants to engage in banter, song suggestions, and applause, just like they would in a physical karaoke setting. Adding to the realism, RippleLive incorporates entertaining face filters that users can apply to their live performances, enhancing the visual appeal of each singing session. The pivotal inclusion of a sophisticated pitch detection algorithm ensures that participants receive immediate, accurate feedback on their pitch accuracy, contributing to the genuine karaoke room experience. Each karaoke room is under the control of a designated room owner who dictates the song selection, mirroring the traditional setup of a physical karaoke venue. As users sing in unison, they can view live scores that dynamically update, allowing them to compete and compare their singing prowess. RippleLive is a virtual stage where friends can harmonize, compete, and celebrate the joy of singing together in a lifelike, competitive, and entertaining environment.

Development

The Frontend server and Backend server are separated in their respective folders within this repo.

Frontend is built using Next.js and React. It is styled using TailwindCSS with NextUI components to simplify the frontend web pages. We utilized the Next.js built-in router feature to present an effect that seemed as if the app was entirely single page. The following libraries were used:

• axios: Sending and fetching from API endpoint
• socket.io-client: Connecting client side to backend websocket host, allows client synchronization with all other individuals in the same room
• peerjs: Client to Client WebRTC connection to initiate the video calling on room join. Note: the peerjs signalling server is used, thus if the signalling server is down that means the video calling will no longer work.
• facefilter/JeelizFaceFilter: Facefilter library that allows for easy application of face effects by utilizing a computer vision/AI + face tracking technique
• Web Audio API: Client side Audio processing, allows us to display song audio graph, user vocal audio graph, and also score computation using our own pitch detection comparison algorithm

The frontend has its .env file that only specifies the api endpoint.

Backend is built using Node.js and Express.js following a RestAPI server. Uses MongoDB for storage and is the backend hosting the room logic using sockets. The following libraries were used:

• socket.io: Server-side socket library for hosting the actual web sockets for the client to connect to. Responsible for all Karaoke room synchronization control, room logic, and most of the functionality presented on the room page.
• express-session: Simple session-based authentication with express
• express-validator: Validating user inputs and cleaning them before allowing any POST requests to proceed
• bcrypt: Salt Hashing password for extra security during auth
• dotenv: Loading content from the .env file for backend server. Nothing private is inside the .env. Only basic server parameters are in there for easier configuration.
• cors: Allowing managing cross-domain requests easier
• uuid: Small library for managing room IDs
• connect-mongo: Change express-session to use the MongoDB store instead of its MemoryStore to prevent memory leaks
• mongodb: MongoDB connector for the backend to the MongoDB database server. The database contains auth info and room info

Deployment

Used Google Cloud VMs to deploy the application and each server is set up within its docker containers.

• Nginx is running on the actual VM (not containerized)
  • Certbot (Let's Encrypt) is used to provide the SSL certificate enabling HTTPS connections
  • Nginx is the reverse proxy that directs traffic to the API server and frontend server
    • The config file for the reverse proxy is located in conf.d/ripplelive.me.conf
• docker-compose.yml is the docker-compose file that allows easy setup and easy deployment of the servers
  • backend/Dockerfile the Dockerfile to generate the image for the backend server
  • frontend/Dockerfile the Dockerfile to generate the image for the frontend server
  • Also sets up a separate container for the MongoDB database server

All it takes is to install Docker, Nginx, and Certbot inside the VM and run the docker-compose file to get the backend, frontend, and database server up and running. Then the config is loaded by moving the config file into the conf.d folder of Nginx and then running Certbot to obtain the SSL certificate for our domain.