Worship Lyrics AI - AI-Powered Song Detector

An intelligent offline-first system that automatically detects worship songs in real-time using AI speech-to-text and fuzzy matching, then displays lyrics on screen for presentations.

Features

• 🎤 Real-time Audio Detection: Captures audio from microphone and transcribes to text
• 🧠 AI-Powered Matching: Uses OpenAI Whisper for speech-to-text and fuzzy matching for lyrics
• 📚 Offline-First: Works without internet for songs in your library
• 🌐 Online Fallback: Automatically searches online if song not found locally
• 🎯 Smart Section Detection: Detects verses, choruses, bridges automatically
• 📺 Dual Display Modes: Control panel for operators, full-screen presentation for audience
• ⚡ Real-time Updates: Continuous listening and automatic slide updates

Architecture

• Backend: Python Flask API with Whisper STT, RapidFuzz matching,
• Frontend: React web application with dual-mode display
• Offline-First: Local song library with automatic online expansion

Quick Start

Prerequisites

• Python 3.10+
• Node.js 16+ and npm
• Microphone or audio input device

Installation

1. Set up backend

cd backend
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

2. Set up frontend

cd ../frontend
pnpm install

3. Create .env file (copy from .env.example)

cd ..
cp .env.example .env
# Edit .env  

4. Generate metadata from song library

cd backend
python song_parser.py

Running the Application

Terminal 1 - Backend:

cd backend; .\venv\Scripts\python.exe app.py

Terminal 2 - Frontend:

cd frontend
pnpm start

The application will be available at:

• Control Panel: http://localhost:3000/control
• Presentation Mode: http://localhost:3000/presentation

Usage

1. Start the backend server (Flask API)
2. Open the Control Panel in your browser
3. Add songs to songs_library/ directory (see format in guide.mdx)
4. Generate metadata using python backend/song_parser.py
5. Set up multi-window views:
   • Control Panel - Main operator interface
   • Preview Output - See what will be projected
   • Presentation Mode - Send to projector (fullscreen)
   • AI Monitoring - Detailed detection status
6. Start listening from the Control Panel
7. Sing or play a song - the system will detect and display lyrics automatically

Multi-Window Projection Setup

The system supports multiple views that can be opened in separate windows:

• Control Panel (/control) - Main interface with controls
• Preview Output (/preview) - Preview what will be sent to projector
• Presentation Mode (/presentation) - Fullscreen display for projector/audience
• AI Monitoring (/monitoring) - Detailed AI detection dashboard

See PROJECTION_SETUP.md for detailed multi-window setup instructions.

Project Structure

worship-lyrics-ai/
├── backend/              # Python Flask backend
│   ├── app.py           # Flask API server
│   ├── stt_engine.py    # Speech-to-text engine
│   ├── lyric_matcher.py # Fuzzy matching engine
│   ├── online_fallback.py # Online lyrics search
│   ├── song_parser.py   # Song file parser
│   └── config.py        # Configuration
├── frontend/            # React frontend
│   ├── src/
│   │   ├── components/  # React components
│   │   └── services/    # API client
│   └── public/
├── config/              # Configuration files
│   └── songs_metadata.json
├── songs_library/       # Song lyrics files (.txt)
└── guide.mdx           # Complete system guide

Song File Format

Songs are stored as .txt files in songs_library/:

TITLE: Song Name
ARTIST: Artist Name
LANGUAGE: English
DURATION: 3:45

[Verse 1]
Line 1 lyrics
Line 2 lyrics

[Chorus]
Chorus line 1
Chorus line 2

Configuration

Edit .env file for configuration:

• STT_ENGINE: whisper (default)
• WHISPER_MODEL: base, small, medium, large
• CONFIDENCE_THRESHOLD: 75 (default, 0-100)
• GENIUS_API_KEY: Your Genius API key (optional)
• ENABLE_ONLINE_FALLBACK: true/false

API Endpoints

• GET /api/health - Health check
• POST /api/start-listening - Start audio capture
• POST /api/stop-listening - Stop audio capture
• GET /api/current-song - Get currently detected song
• GET /api/presentation-data - Get data for presentation display
• GET /api/songs - List all songs
• GET /api/search-songs?q=query - Search songs
• POST /api/manual-select - Manually select a song
• POST /api/reload-metadata - Reload song metadata

Troubleshooting

Audio Issues

• Check microphone permissions in Windows settings
• Verify audio device index in .env (AUDIO_DEVICE_INDEX)
• List devices: Check backend logs on startup

STT Not Working

• Ensure Whisper model is downloaded (first run downloads automatically)
• Check audio quality (too quiet/noisy)
• Try shorter audio chunks (adjust CHUNK_DURATION_SECONDS)

No Songs Found

• Ensure songs are in songs_library/ directory
• Run python backend/song_parser.py to regenerate metadata
• Check config/songs_metadata.json exists and has songs

Frontend Not Connecting

• Verify backend is running on port 5000
• Check CORS settings in backend/app.py
• Check browser console for errors

Development

Adding Songs

1. Create .txt file in songs_library/ with proper format
2. Run python backend/song_parser.py to update metadata
3. Or use online fallback to automatically cache songs

Testing

• Test audio capture: Use Control Panel "Start Listening" button
• Test matching: Speak song lyrics and verify detection
• Test presentation: Open Presentation Mode and verify display

License

This project is open source and available for use.

Contributing

Contributions are welcome! Please read the guide.mdx for detailed system architecture and implementation details.

Support

For detailed documentation, see guide.mdx for complete system overview, architecture, and implementation details.