🦀 CrabMusic

CrabMusic turns audio, images, videos, old TV, and 3D scenes into terminal art.

✨ Current Capabilities

• 🖼️ Turn images into Braille art in the terminal
• 🔁 Morph between two images
• 🎮 Pause, reverse, speed up, tweak, resize, and save image mode live
• 🎞️ Play local videos or YouTube in terminal art
• 📼 Press U to jump through random Internet Archive channels and let the next show roll automatically
• 📹 Use your webcam as a live terminal feed
• 🎵 Visualize your mic or your system audio
• 🔊 Hear the audio while you watch
• 🧊 Load .obj files and spin, zoom, and inspect them
• 🌌 Fly through Primitives, Grid Tunnel, Gravity Well, and Starfield and steer them live

🎨 Gallery

Braille Art Output

Different Rendering Modes

Audio Visualization & Effects

3D OBJ Viewer (Wireframe & Solid)

🚀 Quick Start

Installation

# Clone the repository
git clone https://github.com/newjordan/crabmusic.git
cd crabmusic

# Build the project
cargo build --release

# Run with default settings (microphone input)
cargo run --release

Quick Start: Images

# Open a single image (Braille art)
cargo run --release -- --image ".\media\viper.jpg"

# Start a morph that ping‑pongs between two images (A↔B)
cargo run --release -- --morph-a ".\media\viper.jpg" --morph-b ".\media\tiger.jpg"

# Optional: set morph duration per leg (ms)
cargo run --release -- --morph-a ".\media\viper.jpg" --morph-b ".\media\tiger.jpg" --morph-duration 4000

# Drag-and-paste mode: start, then paste paths to view
cargo run --release -- --image-drop

Quick Start: Audio

# Windows system audio (WASAPI loopback)
cargo run --release -- --loopback

# Mic input, pick devices
cargo run --release -- --device "Microphone" --output-device "Speakers"

Quick Start: Video (feature-gated)

# Play a local video file as Braille
cargo run --release -- --video ".\media\clip.mp4"

cargo run --release --features video -- --video "media/clip.mp4"

# Or play a YouTube URL/channel/playlist via yt-dlp resolution
cargo run --release --features video -- --video "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

Notes:

• Local file playback still works as before.
• YouTube watch/short/channel/playlist URLs are resolved through yt-dlp into a direct stream URL before FFmpeg playback.
• For channel/playlist URLs, CrabMusic samples a random video from the first slice of results.

Quick Start: 3D OBJ Viewer

• Put your .obj files in the models/ folder (we ignore .mtl).
• Run the app (cargo run) and switch to the “OBJ Viewer” channel (use ←/→ to change channels; the status bar lists keys).
• Use Up/Down to switch between OBJ files.

Controls (OBJ Viewer):

• W: toggle Wireframe/Solid
• A/D: yaw left/right
• J/K: pitch down/up
• ,/.: roll CCW/CW
• Z/X: zoom in/out
• F: focus (fit to view)
• G/H: line thickness (wireframe)
• T/Y: vertex dot size (wireframe)
• R: auto‑rotate ON/OFF

Notes:

• Wireframe rendering shows real mesh edges and vertices with hidden‑line removal (depth‑tested) so back/occluded edges don’t clutter.
• Solid mode uses simple diffuse lighting; if normals are missing in the OBJ, we fall back to flat shading safely.
• OBJ loader supports 1‑based and negative indices and triangulates n‑gons by fan. Texture/MTL are ignored for now.

Quick Start: Retro 3D Channels

• Switch to Primitives, Grid Tunnel, Gravity Well, or Starfield with ←/→.
• Each mode stays beat-reactive, but now you can steer/tune it live too.

Controls:

• Primitives: A/D yaw, J/K pitch, ,/. roll, Z/X zoom, R auto-rotate
• Grid Tunnel: A/D twist, J/K speed, ,/. glow, Z/X zoom, R auto-roll
• Gravity Well: audio-reactive dual-view channel with a warped top grid plus a falling-depth side view
• Starfield: A/D twist, J/K speed, ,/. trail gain, Z/X FOV, R auto-twist

More

# List audio devices
cargo run --release -- --list-devices

# Show all options
cargo run --release -- --help

⌨️ Image Mode Controls

Global quality bank

• F1 - color mode: Off → Grayscale → Full RGB
• F2 - cycle named quality presets
• F3 - cycle dither mode
• F4 - gamma preset
• F5 - contrast preset
• F6 - exposure preset
• F7 - reset quality controls

These work consistently in the main audio visualizer app, image mode, and video mode.

• m - start/stop morph (prompt for second image when starting from single image)
• Space - pause/unpause morph
• r - reverse morph direction instantly
• [ / ] - faster / slower (shorter/longer duration per leg)
• l - letterbox ON/OFF (preserve aspect vs fill)
• Legacy aliases still work in image mode: c, p, d, g, v, z, 0
• + / - - manual threshold up/down; a - toggle auto-threshold
• x - attempt to maximize canvas (some terminals may not allow programmatic resize)
• s - save current Braille art to <image_stem>.braille.txt next to the image
• Esc - clears typed input/morph prompt; Esc again (empty) quits; q also quits

🎞️ Video Mode Controls

• Space - pause / resume playback
• l - letterbox ON/OFF
• + / - - manual threshold up/down; a - toggle auto-threshold
• F1-F7 - shared quality bank (image/video/main-app parity)
• Legacy aliases still work in video mode: c, p, d, g, v, z, 0
• t - cycle temporal blend (frame memory) presets
• y - cycle temporal hysteresis presets for steadier motion
• u - open/retune a random Internet Archive stream (where supported)
• q / Esc - quit

📝 Configuration

CrabMusic uses YAML configuration files. The default configuration is loaded from config.default.yaml.

Key configuration options:

• Audio sample rate and buffer sizes
• DSP processing parameters (FFT size, smoothing, frequency ranges)
• Visualization settings (amplitude scale, frequency scale, wave count)
• Character set selection
• Target FPS
• Braille live/image/video presets, color mode defaults, dither overrides, gamma/contrast/exposure overrides
• Video temporal blend + temporal hysteresis defaults for motion smoothing

See config/default.yaml for all available options and detailed comments.

Example braille-quality config:

rendering:
  braille_live: { preset: motion, color_mode: full }
  braille_image: { preset: cinema }
  braille_video: { preset: motion, temporal_blend_preset: 2 }

🛠️ Development

Prerequisites

• Rust 1.75 or later
• System audio libraries:
  • Linux: ALSA development files (libasound2-dev on Debian/Ubuntu)
  • macOS: CoreAudio (included with Xcode Command Line Tools)
  • Windows: WASAPI (included with Windows)

macOS Setup

On macOS, you'll need Xcode Command Line Tools installed:

xcode-select --install

Audio Device Access: macOS may prompt for microphone permissions on first run. Grant access in System Preferences → Security & Privacy → Privacy → Microphone.

Terminal Compatibility: Best performance with:

• iTerm2 (recommended for Unicode Braille support)
• Terminal.app (built-in, works well)
• Alacritty, kitty (modern GPU-accelerated terminals)

Building

# Debug build
cargo build

# Release build (optimized)
cargo build --release

# Run tests
cargo test

# Run benchmarks
cargo bench

# Check code quality
cargo clippy
cargo fmt

Platform-Specific Testing

Visual Test Suite (Unix/macOS/Linux):

# Run the sacred geometry visual test suite
./test-sacred-geometry.sh

Visual Test Suite (Windows):

# Run the sacred geometry visual test suite
.\test-sacred-geometry.ps1

Both scripts run interactive demos for manual visual verification of rendering quality.

Project Structure

crabmusic/
├── src/
│   ├── main.rs              # Application entry point
│   ├── audio/               # Audio capture and playback
│   │   ├── cpal_device.rs   # CPAL-based audio capture
│   │   ├── wasapi_loopback.rs  # Windows WASAPI loopback (system audio)
│   │   ├── output_device.rs # Audio output/passthrough
│   │   └── ring_buffer.rs   # Lock-free ring buffer
│   ├── dsp/                 # DSP processing (FFT, frequency analysis)
│   ├── visualization/       # Visualization engine
│   │   ├── sine_wave.rs     # Sine wave visualizer
│   │   └── character_sets.rs # ASCII/Unicode character sets
│   ├── rendering/           # Terminal rendering with differential updates
│   └── config/              # Configuration management
├── tests/                   # Integration tests
├── benches/                 # Performance benchmarks
└── config.default.yaml      # Default configuration

🏗️ Architecture

CrabMusic uses a pipeline architecture with lock-free ring buffers for thread communication:

┌─────────────────┐
│  Audio Capture  │ (CPAL or WASAPI)
│   (Thread 1)    │
└────────┬────────┘
         │ Ring Buffer
         ▼
┌─────────────────┐
│ DSP Processing  │ (FFT, Frequency Analysis)
│   (Main Loop)   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Visualization  │ (Sine Wave, Character Mapping)
│   (Main Loop)   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│    Rendering    │ (Differential Terminal Updates)
│   (Main Loop)   │
└─────────────────┘

Key Design Decisions:

• Lock-free ring buffer for audio data transfer between threads
• Trait-based audio devices for polymorphic capture (CPAL vs WASAPI)
• Differential rendering to minimize terminal I/O
• Hot-reload configuration for live parameter tuning

🤝 Contributing

Contributions are welcome! Areas for improvement:

• Additional visualizer modes (spectrum analyzer, oscilloscope, waveform)
• Color support and themes
• Beat detection and rhythm analysis
• More character sets and visual effects
• Performance optimizations

📄 License

MIT License - see LICENSE file for details

🙏 Acknowledgments

Built with these excellent Rust crates:

• cpal - Cross-platform audio I/O
• wasapi - Windows WASAPI bindings
• rustfft - Fast Fourier Transform
• spectrum-analyzer - Frequency analysis
• ratatui - Terminal UI framework
• crossterm - Terminal manipulation

---

📊 Current Status

Version: 0.1.0

Best stuff right now:

• ✅ Open images, videos, YouTube links, webcam feed, mic input, or system audio
• ✅ Morph images, save text art, and tweak the look live while it runs
• ✅ Channel-surf random old TV and weird archive finds
• ✅ Load .obj files and control them in the terminal
• ✅ Jump into retro 3D scenes and steer them live

Next up (roadmap):

• 🔁 Image playlists (3+ images) with selectable transitions
• ✨ Additional transitions (noise dissolve, wipe/slide, radial)
• 📊 Simpler, accurate spectrum analyzer (Spectrum 2) and beat‑reactive effects
• 🧭 XY oscilloscope refinements