Skip to content

Latest commit

 

History

History
926 lines (743 loc) · 29.8 KB

File metadata and controls

926 lines (743 loc) · 29.8 KB

Wingman - AI DAW Control Assistant

Project Overview

Wingman is an AI-powered music production assistant that provides deep Ableton Live control through a streamlined two-component architecture:

  • Standalone App (Electron): React UI with AI that runs independently on your desktop
  • Remote Script (Python): Ableton Live API integration for total DAW control

Goal: AI can see and control EVERYTHING in Ableton Live - tempo, tracks, clips, plugins, automation, mixer - providing intelligent creative assistance with robust safety guardrails.


Architecture

Simplified Architecture (Current Approach)

┌─────────────────────────────────────────────┐
│  Wingman Standalone App (Electron)          │
│  - Full React UI (WingmanMinimal)           │
│  - AI logic (OpenAI, Anthropic, etc.)       │
│  - Chat interface                           │
│  - WebSocket server :8765                   │
│  - Safety controls & guardrails             │
│  - Connection status monitoring             │
└──────────────────┬──────────────────────────┘
                   │
                   ↕ WebSocket (JSON Messages)
                   │
┌──────────────────┴──────────────────────────┐
│  Ableton Remote Script (Python 3)           │
│  - WebSocket client (connects to App)       │
│  - Full Live API access (LOM)               │
│  - Track/clip/device control                │
│  - Scene launching                          │
│  - Parameter automation                     │
│  - MIDI clip creation                       │
│  - Real-time observers for state changes    │
└─────────────────────────────────────────────┘

Why This Architecture?

Simplicity: Only 2 components = easier debugging, faster development Reliability: WebSocket provides ordered, reliable message delivery Complete Control: Remote Script has full LOM (Live Object Model) access Safety-First: Guardrails built into app from day one No Plugin Required: Works entirely outside the VST3 ecosystem

What We Eliminated (and Why)

Bridge VST3: Unnecessary - Remote Script provides all DAW control we need ❌ UDP Communication: Unreliable (packet loss/reordering) - WebSocket is better for control messages ❌ Complex Port Management: Single WebSocket connection instead of 4+ UDP ports ❌ UID System: Fragile hash-based IDs that break on track renames - use observers instead ❌ Max for Live: Deferred to future phase - not critical for core functionality

Remote Script Capabilities

The Ableton Remote Script (Python 3 in Live 11+) provides everything we need:

✅ Full Read Access:

  • All track properties (name, color, type, arm state, mute, solo, volume, pan)
  • Clip matrix (has_clip, is_playing, is_triggered, length, name)
  • Device chains and all parameter values
  • Transport state (playing, tempo, time signature, position)
  • Scene list and properties

✅ Full Control:

  • Create/delete tracks and scenes
  • Launch clips and scenes
  • Arm/mute/solo tracks
  • Create MIDI clips and add notes
  • Set device parameters (with automation)
  • Control transport (play/stop/tempo/locate)

✅ Real-Time Observers:

  • Detect any parameter changes instantly
  • Track user actions vs. AI actions
  • Respond to set changes (tracks added/removed)

Tech Stack

Standalone App

  • Framework: Electron 28+ (Windows first, Mac later)
  • UI: React 18 + TypeScript (existing WingmanMinimal)
  • Build: Vite 5
  • AI: OpenAI GPT-4, Anthropic Claude, Perplexity, OpenRouter
  • Communication: WebSocket server using ws library (port 8765)
  • Node Version: 18+ LTS

Remote Script

  • Language: Python 3.x (Live 11+)
  • API: Ableton Live Object Model (LOM) v2 framework
  • Location: %USERPROFILE%\Documents\Ableton\User Library\Remote Scripts\WingmanControl
  • Communication: WebSocket client using built-in Python modules
  • Threading: Python threading for non-blocking WebSocket I/O

Important: Ableton Live 11+ uses Python 3, but it's a stripped-down interpreter:

  • No subprocess, msvcrt, win32api modules
  • Limited standard library access
  • socket and threading modules ARE available

Communication Protocol

WebSocket Connection

Port: 8765 (app listens, script connects) Format: JSON messages over WebSocket Reliability: TCP-based, ordered delivery, automatic reconnection

Message Types

1. Commands (App → Script)

AI or user triggers an action in Ableton Live.

{
  "type": "command",
  "id": "cmd-a3f91b",
  "cmd": "clip.launch",
  "args": {
    "track_index": 0,
    "scene_index": 2,
    "quantization": 1
  },
  "meta": {
    "origin": "ai",
    "timestamp": 1730512345.123
  }
}

2. Events (Script → App)

Ableton Live state changes (from observers or user actions).

{
  "type": "event",
  "event": "tempo_changed",
  "data": {
    "tempo": 128.0,
    "timestamp": 1730512346.456
  }
}

3. Acknowledgments (Script → App)

Response to a command (success or error).

{
  "type": "ack",
  "id": "cmd-a3f91b",
  "success": true,
  "result": {
    "clip_launched": true,
    "is_playing": true
  }
}

4. Snapshots (Script → App)

Full DAW state (sent on connection and on-demand).

{
  "type": "snapshot",
  "data": {
    "tempo": 120.0,
    "time_signature_numerator": 4,
    "time_signature_denominator": 4,
    "playing": false,
    "current_song_time": 0.0,
    "tracks": [
      {
        "index": 0,
        "name": "1-MIDI",
        "type": "midi",
        "color": 12,
        "arm": false,
        "mute": false,
        "solo": false,
        "volume": 0.85,
        "pan": 0.0,
        "clips": [
          {"index": 0, "has_clip": false},
          {"index": 1, "has_clip": true, "name": "Bass", "length": 4.0, "playing": false}
        ]
      }
    ],
    "scenes": [
      {"index": 0, "name": "Scene 1"},
      {"index": 1, "name": "Breakdown"}
    ]
  }
}

Command Verbs

Transport:

  • transport.play - Start playback
  • transport.stop - Stop playback
  • transport.continue - Continue from current position
  • transport.locate - Jump to position (args: beats)
  • transport.set_tempo - Change BPM (args: tempo)
  • transport.record - Toggle record mode

Clips:

  • clip.launch - Fire clip (args: track_index, scene_index, quantization)
  • clip.stop - Stop clip (args: track_index)
  • clip.create - Create new MIDI clip (args: track_index, scene_index, length)
  • clip.delete - Remove clip (args: track_index, scene_index)
  • clip.duplicate - Duplicate clip (args: track_index, scene_index)
  • clip.add_notes - Add MIDI notes (args: track_index, scene_index, notes)
  • clip.clear_notes - Remove all notes (args: track_index, scene_index)

Tracks:

  • track.create_midi - Create MIDI track (args: name)
  • track.create_audio - Create audio track (args: name)
  • track.delete - Remove track (args: track_index)
  • track.arm - Set arm state (args: track_index, armed)
  • track.mute - Set mute state (args: track_index, muted)
  • track.solo - Set solo state (args: track_index, soloed)
  • track.rename - Change track name (args: track_index, name)
  • track.set_volume - Set volume (args: track_index, volume [0.0-1.0])
  • track.set_pan - Set pan (args: track_index, pan [-1.0-1.0])

Scenes:

  • scene.launch - Fire scene (args: scene_index)
  • scene.create - Create new scene (args: name)
  • scene.delete - Remove scene (args: scene_index)
  • scene.duplicate - Duplicate scene (args: scene_index)

Devices:

  • device.param.set - Set device parameter (args: track_index, device_index, param_index, value)
  • device.add - Load device on track (args: track_index, device_name)
  • device.remove - Remove device (args: track_index, device_index)

Event Types

Transport Events:

  • tempo_changed - BPM changed
  • time_signature_changed - Time sig changed
  • playing_status_changed - Play/stop state
  • position_changed - Playhead moved (throttled to 10Hz)

Clip Events:

  • clip_playing_status_changed - Clip started/stopped
  • clip_added - New clip created
  • clip_removed - Clip deleted

Track Events:

  • track_added - Track created
  • track_removed - Track deleted
  • track_name_changed - Track renamed
  • track_arm_changed - Arm state changed
  • track_mute_changed - Mute state changed

Parameter Events:

  • device_param_changed - Device parameter value changed
  • user_touched_param - User manually changed parameter (AI should pause)

Data Model - Index-Based with Observers

The Index Problem (and Solution)

Reality: Ableton uses indices that shift when tracks/scenes are reordered or deleted.

Previous Approach (REMOVED): Generate stable UIDs via hashing track properties Why It Failed: UIDs break when users rename tracks, change colors, or modify devices - which happens constantly in music production

New Approach: Accept index-based model and use observers to track changes

How It Works

  1. Initial Snapshot: On connection, Remote Script sends full DAW state with indices
  2. Observer Registration: Script registers observers for all relevant properties
  3. Change Detection: When anything changes, script sends event with current indices
  4. App State Sync: Electron app maintains synchronized state model
  5. Command Validation: Before executing, script validates indices against current state

Example Flow

User renames track 2 → Observer fires → Script sends event:
{
  "type": "event",
  "event": "track_name_changed",
  "data": {
    "track_index": 2,
    "old_name": "Drums",
    "new_name": "Drum Bus"
  }
}

App updates its state → AI now knows track 2 is "Drum Bus"

AI sends command:
{
  "cmd": "track.mute",
  "args": {"track_index": 2, "muted": true}
}

Script validates track 2 exists → Executes → Sends ack

Handling Destructive Changes

Track Deletion:

User deletes track 3 (of 5 tracks)
→ Track 4 becomes track 3, track 5 becomes track 4
→ Script sends: { "event": "track_removed", "data": {"track_index": 3} }
→ App rebuilds track list
→ In-flight commands to old track 4/5 fail gracefully with error

Track Reordering:

User drags track 1 to position 3
→ Script detects track list change
→ Sends full track list update event
→ App rebuilds state

Safety & Guardrails (Phase 1 Priority)

Critical Safety Features (Implemented First)

1. AI Pause / Kill Switch

UI: Large red "PAUSE AI" button always visible Function: Immediately stops all AI-generated commands Scope: User-initiated commands still work

2. Access Control List (ACL)

Default-deny for destructive operations:

const DEFAULT_ACL = {
  ALLOW_TRACK_DELETE: false,      // Requires explicit enable
  ALLOW_CLIP_DELETE: false,       // Requires explicit enable
  ALLOW_SCENE_DELETE: false,      // Requires explicit enable
  ALLOW_DEVICE_REMOVE: false,     // Requires explicit enable
  ALLOW_TRANSPORT_CONTROL: true,  // Safe operations
  ALLOW_CLIP_LAUNCH: true,
  ALLOW_PARAMETER_CHANGE: true
};

UI: Settings panel with checkboxes for each permission Storage: Persisted to user config file

3. Command Rate Limiting

  • Global: Max 10 commands/second from AI
  • Per-Parameter: Max 10 parameter changes/second
  • Burst Protection: Max 5 commands in 100ms window

Behavior: Excess commands queued or rejected with error

4. User Touch Detection

When user manually touches a control:

  • Script detects parameter change from user (not from script)
  • Sends user_touched_param event to app
  • App freezes AI control of that parameter for 5 seconds
  • Visual indicator in UI: "User controlling [param name]"

5. Command Validation

Every command validated before execution:

  • Track/clip/scene indices checked against current state
  • Parameter values clamped to min/max ranges
  • Required arguments presence check
  • Invalid commands rejected with detailed error

6. Undo Stack

  • All AI commands logged with timestamps
  • "Undo Last AI Action" button
  • Undo stack persisted across sessions
  • Max 50 actions in history

7. Dry Run Mode

  • UI: "Preview Mode" toggle
  • Function: Shows what AI would do without executing
  • Use Case: Test AI understanding before committing

Error Recovery Patterns

WebSocket Disconnection

// App side (Electron)
ws.on('close', () => {
  connectionStatus = 'disconnected';
  showNotification('Lost connection to Ableton Live');
  attemptReconnect();
});

function attemptReconnect() {
  let retries = 0;
  const maxRetries = 10;
  const retryInterval = setInterval(() => {
    if (retries >= maxRetries) {
      clearInterval(retryInterval);
      showError('Could not reconnect to Ableton Live');
      return;
    }
    // Try to reconnect...
    retries++;
  }, 2000); // Retry every 2 seconds
}

Command Timeout

  • Default timeout: 5 seconds per command
  • If no ack received, command considered failed
  • UI shows timeout error with retry option

State Desync

  • App requests snapshot every 30 seconds (heartbeat)
  • If snapshot differs from app state, full resync
  • User notified of desync with details

Script Crash Recovery

  • Script logs errors to Live's Log.txt
  • On restart, script sends connection event
  • App detects reconnection, requests fresh snapshot
  • Previous command queue cleared

Current Status

✅ Working (Legacy WebView2 VST3)

  1. VST3 Plugin Loading: Loads in Ableton Live successfully
  2. WebView2 Integration: Renders React UI in plugin window
  3. Basic DAW Context: Reads BPM, time signature, playback state
  4. AI Integration: OpenAI GPT-4 with DAW context in prompts
  5. Build System: Automated CMake + Vite build

🚧 Migration to New Architecture

  • Legacy VST3: Kept for backward compatibility during transition
  • New Architecture: Electron + Remote Script (actively developing)
  • Dual Support: Both architectures will coexist during migration

⛔ Deprecated Components

  • Bridge VST3 (minimal plugin) - Not needed
  • UDP communication layer - Replaced by WebSocket
  • UID generation system - Replaced by index + observers
  • Max for Live integration - Deferred to future

Implementation Roadmap (6-8 Weeks)

Week 1-2: Foundation & Safety

Goal: Electron app with safety controls

  • Create Electron project structure
  • Port React UI from actual ui/vocal-muse-sidecar-main
  • Implement WebSocket server (port 8765)
  • Add connection status UI
  • Build safety controls:
    • AI Pause / Kill Switch button
    • ACL configuration panel
    • Rate limiting engine
    • Command validation framework
  • Add settings persistence
  • Test AI chat works standalone (without DAW)

Week 3-4: Remote Script Core

Goal: Basic Ableton Live control

  • Create Remote Script folder structure
  • Implement WebSocket client in Python
  • Add automatic reconnection logic
  • Implement command dispatcher
  • Add basic command handlers:
    • Transport control (play/stop/tempo)
    • Clip launching
    • Track mute/solo/arm
  • Send snapshot on connection
  • Add error handling and logging
  • Test with Electron app

Week 5: Observers & Events

Goal: Real-time state synchronization

  • Register observers for:
    • Tempo changes
    • Transport state
    • Track list changes
    • Clip slots
    • Parameter changes
  • Implement user touch detection
  • Add position update (throttled to 10Hz)
  • Test state sync during live editing

Week 6: Advanced Commands

Goal: Full DAW control

  • Track creation/deletion
  • MIDI clip creation
  • Add notes to clips
  • Device parameter control
  • Scene creation/launching
  • Volume/pan control
  • Test complex workflows

Week 7: Polish & Testing

Goal: Robust, production-ready

  • Comprehensive error messages
  • Undo stack implementation
  • Dry run / preview mode
  • Connection status heartbeat
  • State desync detection & recovery
  • Performance optimization
  • User documentation
  • Video tutorials

Week 8: Packaging & Deployment

Goal: Easy installation

  • Electron app installer (.exe for Windows)
  • Auto-updater integration
  • Remote Script installer script
  • Troubleshooting guide
  • Beta testing with real users
  • Bug fixes based on feedback

File Structure

Current Project

C:\wingman\
├── CLAUDE.md                        # This file
├── CMakeLists.txt                   # Legacy VST3 build config
├── JUCE/                            # JUCE 8.0.9 (legacy)
├── Source/                          # Legacy WebView2 VST3 code
├── actual ui/vocal-muse-sidecar-main/  # React UI (will be ported)
└── build/                           # Legacy VST3 build output

New Architecture (To Be Created)

C:\wingman\
├── CLAUDE.md                        # This file
│
├── wingman-app/                     # Electron standalone app
│   ├── package.json
│   ├── electron-builder.json        # Installer config
│   ├── src/
│   │   ├── main/                    # Electron main process
│   │   │   ├── index.ts             # App entry, window management
│   │   │   ├── websocketServer.ts   # WebSocket server
│   │   │   ├── safetyManager.ts     # ACL, rate limiting, kill switch
│   │   │   ├── commandValidator.ts  # Command validation
│   │   │   └── stateManager.ts      # DAW state synchronization
│   │   ├── renderer/                # React UI
│   │   │   ├── App.tsx              # Main app component
│   │   │   ├── components/
│   │   │   │   ├── WingmanMinimal.tsx  # Main UI
│   │   │   │   ├── SafetyControls.tsx  # Kill switch, ACL panel
│   │   │   │   ├── ConnectionStatus.tsx
│   │   │   │   └── UndoPanel.tsx
│   │   │   └── hooks/
│   │   │       ├── useWebSocket.ts  # WebSocket hook
│   │   │       └── useDAWState.ts   # DAW state hook
│   │   └── preload/
│   │       └── preload.ts           # Context bridge
│   ├── .env                         # API keys
│   └── build/                       # Build output
│       └── Wingman Setup.exe
│
├── remote-script/                   # Ableton Python control surface
│   └── WingmanControl/
│       ├── __init__.py              # Entry point
│       ├── WingmanControl.py        # Main Remote Script class
│       ├── websocket_client.py      # WebSocket connection
│       ├── command_handler.py       # Command dispatcher
│       ├── observer_manager.py      # LOM observers
│       ├── state_snapshot.py        # Snapshot generator
│       └── config.py                # Configuration
│
└── installer/                       # Installation tools
    ├── install_remote_script.py     # Auto-install script
    └── README.md                    # Setup instructions

Installation Paths

Electron App:

C:\Program Files\Wingman\Wingman.exe
C:\Users\[User]\AppData\Roaming\Wingman\config.json
C:\Users\[User]\AppData\Roaming\Wingman\logs\

Remote Script:

C:\Users\[User]\Documents\Ableton\User Library\Remote Scripts\WingmanControl\

Logs:

C:\Users\[User]\AppData\Roaming\Wingman\logs\app.log
C:\Users\[User]\AppData\Roaming\Ableton\Live X.X\Preferences\Log.txt  (Remote Script)

Environment Variables

.env file (Electron App)

# AI Providers
OPENAI_API_KEY=sk-proj-...
ANTHROPIC_API_KEY=sk-ant-...
PERPLEXITY_API_KEY=pplx-...
OPENROUTER_API_KEY=sk-or-...

# WebSocket Configuration
WEBSOCKET_PORT=8765
WEBSOCKET_HOST=localhost

# Safety Defaults
DEFAULT_RATE_LIMIT=10
DEFAULT_ACL_DESTRUCTIVE=false

# Development
NODE_ENV=production
LOG_LEVEL=info

Build Instructions

Electron App

Development:

cd C:\wingman\wingman-app
npm install
npm run dev          # Run with hot reload

Production Build:

cd C:\wingman\wingman-app
npm run build        # Build renderer (Vite)
npm run compile      # Build main (TypeScript)
npm run dist         # Create installer

Output: C:\wingman\wingman-app\build\Wingman Setup.exe

Remote Script Installation

Automated:

cd C:\wingman\installer
python install_remote_script.py

Manual:

xcopy "C:\wingman\remote-script\WingmanControl" "%USERPROFILE%\Documents\Ableton\User Library\Remote Scripts\WingmanControl" /E /I /Y

Enable in Ableton:

  1. Open Ableton Live preferences
  2. Go to Link/Tempo/MIDI tab
  3. In Control Surface dropdown, select "WingmanControl"
  4. Restart Ableton Live

Key Technical Decisions

Why WebSocket Over UDP?

Reliability: TCP ensures ordered, guaranteed delivery - critical for control messages Simplicity: Single connection instead of multiple UDP ports Error Handling: Built-in connection state management Debugging: Easy to inspect with browser dev tools or Wireshark Firewall: More likely to work through firewalls than UDP

Performance: WebSocket latency (~1-2ms local) is acceptable for control messages (not audio)

Why No VST3 Plugin?

Remote Script is Sufficient: LOM provides complete DAW control Complexity Reduction: Fewer components = easier to maintain Cross-Platform: No C++ compilation issues Safer: Runs in separate process, can't crash DAW

Future Consideration: Could add VST3 for visual integration (status widget in DAW), but not for control

Why Index-Based Model?

Ableton's Reality: Live uses indices internally, fighting this creates problems Observer Pattern: Detect changes in real-time rather than maintain fragile mappings Simplicity: No UID generation/translation logic Performance: No hashing overhead

User Impact: App UI shows track names, AI refers to tracks by name, but commands use validated indices

Python 3 in Remote Scripts

Live 11+ Requirement: Must use Python 3 syntax Limited Standard Library: Ableton strips out many modules for safety Available Modules: socket, threading, json, re, math, time Not Available: subprocess, urllib, http, msvcrt, win32api

WebSocket Solution: Use socket module directly with WebSocket protocol implementation


Troubleshooting

Standalone App Won't Start

  1. Check Node.js installed: node --version (v18+ required)
  2. Check port 8765 not in use: netstat -ano | findstr 8765
  3. Check firewall allows localhost connections
  4. Check logs: %APPDATA%\Wingman\logs\app.log
  5. Try running from terminal to see errors: Wingman.exe

Remote Script Not Connecting

  1. Verify installed in correct folder:
    %USERPROFILE%\Documents\Ableton\User Library\Remote Scripts\WingmanControl\
    
  2. Check selected in Live preferences (Link/Tempo/MIDI → Control Surface)
  3. Check Python errors in Live's Log.txt:
    %APPDATA%\Ableton\Live X.X\Preferences\Log.txt
    
  4. Verify Wingman app is running first (script connects to app)
  5. Restart Ableton Live after installing script

Connection Status Shows "Disconnected"

  1. Ensure Wingman app started before Ableton Live
  2. Check firewall not blocking localhost:8765
  3. Try restarting both app and Live
  4. Check app logs for WebSocket errors
  5. Verify no other program using port 8765

Commands Not Executing

  1. Check connection status is "Connected" in app
  2. Verify ACL permissions enabled for command type (Settings → Safety)
  3. Check AI not paused (red "PAUSE AI" button should not be active)
  4. Try simple command first: "play the transport"
  5. Check Live's Log.txt for Python errors
  6. Verify command targets valid track/clip (check indices)

AI Not Seeing DAW Changes

  1. Verify connection status is "Connected"
  2. Check "Last Update" timestamp in app (should update in real-time)
  3. Try requesting snapshot manually (Settings → Refresh State)
  4. Check app logs for event parsing errors
  5. Verify observers are registered (check Live's Log.txt on startup)

Performance Issues / Lag

  1. Check event rate throttling (Settings → Performance)
  2. Disable position updates if not needed (high frequency)
  3. Reduce AI command rate limit (Settings → Safety)
  4. Check system resources (Electron app memory usage)
  5. Clear undo history if very large (> 1000 actions)

Performance Targets

  • Command Latency: < 20ms (app → script → DAW execution)
  • Event Rate:
    • Transport: 10 Hz (position updates)
    • Parameters: 20 Hz max per parameter
    • Other events: Instant (no throttling)
  • WebSocket Overhead: < 5ms for round-trip localhost communication
  • Snapshot Generation: < 100ms for typical set (50 tracks, 200 clips)
  • Memory: < 200 MB for Electron app, < 20 MB for Remote Script

Security & Privacy

Local-Only Communication

  • WebSocket binds to localhost only (not 0.0.0.0)
  • No external network access required
  • All data stays on local machine

API Key Security

  • Stored in .env file (not checked into git)
  • Electron app uses dotenv to load securely
  • Never sent to Remote Script or exposed in UI

AI Safety

  • All AI commands validated before execution
  • Destructive operations disabled by default
  • User can audit all AI actions in undo stack
  • "Dry run" mode for testing without executing

No Telemetry

  • No analytics or tracking
  • No crash reporting to external servers
  • Logs stored locally only

Known Limitations

Ableton Live Only

  • Currently only works with Ableton Live 11+ (Python 3)
  • No support for other DAWs yet
  • Future: Could adapt Remote Script pattern to other DAWs with scripting APIs

Limited Audio Analysis

  • Cannot read audio file waveforms (would require Max for Live)
  • Cannot analyze audio in real-time (frequency spectrum, etc.)
  • Can only control parameters, not process audio

Python Module Restrictions

  • Cannot use most Python standard library (Ableton limitation)
  • Cannot spawn subprocesses or access Win32 API
  • Limited to modules available in Live's Python build

Windows First

  • Initial version Windows only
  • Mac support planned after Windows stabilized
  • Linux not supported (Electron works, but Ableton is Windows/Mac only)

Future Enhancements

Short-Term (Next 3 Months)

  • Mac support (test Remote Script on Mac Live)
  • Advanced MIDI generation (using Magenta, transformers)
  • Preset library (common AI commands saved as templates)
  • Multi-language support (UI localization)
  • Dark/light theme toggle

Medium-Term (3-6 Months)

  • Max for Live device (audio analysis, waveform display)
  • VST3 widget (show connection status in DAW, optional)
  • Cloud sync for presets and settings
  • Collaborative sessions (multiple users controlling one DAW)
  • Voice control integration (speech-to-text commands)

Long-Term (6-12 Months)

  • Support for other DAWs (FL Studio, Logic Pro, Cubitase, Studio One)
  • Advanced AI features (style transfer, arrangement suggestions)
  • Marketplace for community presets and templates
  • Mobile app (control DAW from phone/tablet)
  • Plugin parameter learning (AI watches user, suggests improvements)

Git Repository

Current Branch: claude/code-review-analysis-011CUyxzyvT4tboSrg6C4PEy

Recent Commits:

c77d2a0 Merge pull request #6 (project review feedback)
8cd7fa2 Merge pull request #5 (investigate p variable)
a8a09da Merge pull request #4 (check p)
2579d1d feat: Implement per-chat MIDI isolation and auto-features
c39d3f2 docs: Add comprehensive feature verification reports

Contact & Support

  • Developer: Sylor
  • Project Path: C:\wingman\
  • Logs: %APPDATA%\Wingman\logs\
  • Target Platform: Windows 10/11 (Mac later)
  • Node Version: 18+ LTS
  • Ableton Version: Live 11+ (Python 3)

FAQ

Q: Do I need to keep the Wingman app running?

A: Yes, the Electron app must run alongside Ableton Live for the Remote Script to connect.

Q: Can I use this with other DAWs besides Ableton?

A: Not yet. The Remote Script is Ableton-specific. Future versions may support other DAWs.

Q: Is this safe? Can AI delete my project?

A: Safe by default. Destructive operations (track/clip deletion) are disabled until you explicitly enable them in Settings → Safety.

Q: What if AI does something wrong?

A: Hit the red "PAUSE AI" button immediately. Then use "Undo Last AI Action" to revert. You can also use Ableton's native undo (Ctrl+Z).

Q: Does this send my music to the cloud?

A: No. All communication is local (Electron app ↔ Remote Script). AI API calls (OpenAI, etc.) only send text prompts, never audio.

Q: Why not build a VST3 plugin instead?

A: VST3 plugins can't control the full DAW - they only see their own track. Ableton's Remote Script API provides complete access.

Q: Can I use this in live performances?

A: Yes, but test thoroughly first. Use "Dry run" mode to preview AI actions. Consider disabling destructive operations during live sets.

Q: What happens if the connection drops during playback?

A: Playback continues normally. The app will show "Disconnected" and attempt to reconnect every 2 seconds. No data is lost.


Last Updated: 2025-11-10 Status: Simplified architecture - Electron App + Remote Script only Next Step: Week 1-2 - Foundation & Safety implementation Estimated Completion: 6-8 weeks from start date