DubSync

DubSync exclusively orchestrates the OpenAI API for speech-to-text transcription, speaker diarization, and context-aware film-quality translation, then compiles export-ready .srt and .vtt files for localization, post-production, and distribution workflows.

Supported languages: English, Spanish, Italian, Portuguese, German, Japanese, Russian, Polish, Catalan, French, Dutch, Swedish, Ukrainian, Norwegian, Finnish, Turkish, Korean, Greek, Czech, Indonesian, Hungarian, Chinese, Arabic, Hindi, Vietnamese, Thai, Danish, Hebrew, Romanian, Bulgarian.

Screenshots

Pipeline & Export

Architecture

Data Flow

1. Browser: User selects a video. The ffmpeg.wasm module safely extracts 32kbps mono audio purely on the client-side (video data never actually leaves your machine).
2. Upload: The audio blob + metadata is sent to FastAPI and saved in Supabase Storage.
3. Transcribe & Diarize: The Celery worker sends the audio to OpenAI's Whisper models for full speech-to-text with speaker diarization returning heavily structured timestamps.
4. Translate: The worker processes transcript chunks entirely through OpenAI models via custom localization prompting logic.
5. Generate: The system dynamically merges the language translations back with the exact timestamp hashes to produce robust .srt and .vtt files.
6. Deliver: Supabase Realtime pushes status updates live to your browser Dashboard and Transcript Editor so the user can easily download final outputs.

Tech Stack & Features

Core Platform Capabilities

• Enterprise Security & Client-Side Extraction: Your video stays on your device. We use ffmpeg.wasm to extract a 32kbps mono audio track entirely in your browser. Only the extracted audio track travels to our secure servers for AI processing, saving bandwidth and guaranteeing total privacy.
• Lightning Fast API & Cloud Processing: Offload heavy video processing to scalable, distributed Celery background workers. Transcribe hours of video in just minutes.
• Contextual Dubbing & 30+ Languages: Powered by optimized Whisper models and OpenAI's semantic translation. The AI understands the context of your video, delivering lifelike emotional subtitles with extreme accuracy across global languages. Features automatic source-language detection (Spark AI Auto-detect).
• Export Anywhere (Rich Transcript Editor): Live-preview the synchronized translation separated logically by timecodes right in the neo-brutalist dashboard natively, then download in standard .SRT or .VTT formats directly to your editing software.
• Realtime Pipeline Analytics: Watch your project evolve instantaneously with Supabase WebSockets mapping the strict worker phases: Queued → Transcribing (Extracting audio and generating transcript) → Translating (Translating to target language) → Generating Files.
• Striking Neo-Brutalist Interface: Completely custom, monochromatic, strictly black-and-white theme utilizing stark brutalist shadows (8px_8px_0px_#000), heavy typography, and interactive WebGL Three.js particle waves.

Stack Details

Layer	Technology	Role / Feature
Frontend UI	Next.js 15, React 19, Tailwind CSS	Ultra-responsive monochrome Neo-Brutalist design language. Featuring live Dashboard monitoring, mobile accessibility, interactive Three.js Particle Wave background, and an integrated Subtitle Editor preview.
Backend	Python FastAPI	Core API orchestrator managing secure uploads and Celery handoffs.
AI Processing	OpenAI API	Responsible for both Audio-to-Transcript diarization and Contextual Semantic Translation.
Database	Supabase (PostgreSQL)	Stores Projects, transcriptions mapping, and generated subtitle assets.
Auth	Supabase Auth	Highly secure Email/Password session handoffs native to Next.js routes.
Task Queue	Celery + Redis	Industrial-grade asynchronous AI task processing pipeline.
DevOps	Docker & Compose	Turnkey 5-service localized container orchestration.

Getting Started

Prerequisites

• Docker & Docker Compose
• A standard Supabase project
• An active OpenAI API key

(Note: DubSync no longer requires an Anthropic API Key as routing has been unified to OpenAI.)

1. Clone & Configure

git clone <repo-url>
cd DubSync
cp .env.example .env

Edit your fresh .env with your secure credentials:

SUPABASE_URL=https://xxxxx.supabase.co
SUPABASE_ANON_KEY=eyJ...
SUPABASE_SERVICE_ROLE_KEY=eyJ...
NEXT_PUBLIC_SUPABASE_URL=https://xxxxx.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJ...
OPENAI_API_KEY=sk-...
CELERY_BROKER_URL=redis://redis:6379/0
CELERY_RESULT_BACKEND=redis://redis:6379/0
NEXT_PUBLIC_API_URL=http://localhost:8000

2. Set up Supabase

1. Run the migration in your Supabase SQL Editor:
   • Open supabase/migrations/001_initial_schema.sql and execute it to generate core project routes.
2. Initialize your Storage space by creating two buckets (set both to private):
   • audio-uploads
   • subtitle-files
3. Lastly, enable strict Realtime observation on the projects table (Database → Replication → toggle projects).

3. Start all services

Spin up the entire local infrastructure via Docker:

docker compose up

This brings 5 unified services to life:

Service	URL	Description
Frontend	http://localhost:3000	The user-facing Next.js App / UI
Backend	http://localhost:8000	FastAPI server (+ Swagger docs at /docs)
Worker	(Background)	Active Celery polling (4 concurrent processing blocks)
Redis	localhost:6379	Fast message broker for tasks
Flower	http://localhost:5555	A live dashboard tracking your active Celery workers

4. Use the app

1. Sign up at http://localhost:3000/signup
2. Click New Project → select a video file
3. Choose target language → click Start Processing
4. Watch real-time progress: Queued → Transcribing → Translating → Generating → Completed
5. Download your .srt or .vtt subtitle file

Project Structure

DubSync/
├── docker-compose.yml              # 5-service orchestration
├── .env.example                    # Environment variables template
│
├── frontend/                       # Next.js 15 (App Router)
│   ├── src/app/                    # Pages: login, signup, dashboard, upload
│   ├── src/components/             # JobStatusBadge, SubtitlePreview, VideoPlayer
│   ├── src/hooks/                  # useAuth, useJobStatus (Supabase Realtime)
│   └── src/lib/                    # supabase-browser, api client, audio extraction
│
├── backend/                        # Python FastAPI + Celery
│   ├── app/main.py                 # FastAPI entry point
│   ├── app/routers/                # projects, subtitles endpoints
│   ├── app/services/               # transcription, translation, subtitle generation
│   ├── app/tasks/                  # Celery app + process_video pipeline
│   └── tests/                      # Unit tests
│
└── supabase/migrations/            # Database schema + RLS policies

API Endpoints

Method	Path	Description
GET	/api/health	Health check
POST	/api/projects	Create project + upload audio
GET	/api/projects	List user's projects
GET	/api/projects/{id}	Get project details
DELETE	/api/projects/{id}	Delete project + cleanup
GET	/api/projects/{id}/subtitles	Get translated subtitle JSON
GET	/api/projects/{id}/subtitles/download?format=srt	Download .srt or .vtt

Supported Languages

English, Spanish, Italian, Portuguese, German, Japanese, Russian, Polish, Catalan, French, Dutch, Swedish, Ukrainian, Norwegian, Finnish, Turkish, Korean, Greek, Czech, Indonesian, Hungarian, Chinese, Arabic, Hindi, Vietnamese, Thai, Danish, Hebrew, Romanian, Bulgarian

Running Tests

# Backend unit tests (inside container)
docker compose exec backend pytest -v

# Or locally with a virtualenv
cd backend
pip install -r requirements.txt
pytest -v

Cost Estimates

Component	Pricing	Example (2hr movie)
OpenAI Transcription (Whisper)	$0.006/min	~$0.72
OpenAI Translation (GPT-4o mini)	~$0.15/1M input tokens	~$0.10-0.30
Total per video		~$0.80-1.00