AI Virtual Painter

Real-time hand-tracking painter that lets you draw on your webcam feed using only finger gestures — no mouse or stylus required.

Built following Murtaza's Workshop tutorial, extended with a virtual mouse module.

---

Result Image

---

Features

• Freehand drawing on live webcam feed
• 6 colors: Purple, Blue, Green, Red, Yellow + Eraser
• Gesture-based control — no keyboard/mouse needed while painting
• AI Virtual Mouse — control your system cursor with your hand (bonus module)

Video Demonstration

screencast.mp4

Gestures

Gesture	Action
Index finger up	Draw mode
Index + Middle fingers up	Selection mode (choose color from header)
C key	Clear canvas
Q key	Quit

Project Structure

.
├── HandTrackingModule.py   # Reusable hand detector (MediaPipe wrapper)
├── AiVirtualPainter.py     # Main painter application
├── AiVirtualMouse.py       # Bonus: control mouse cursor with hand gestures
└── requirements.txt

Setup

1. Create conda environment

conda create -n ai-painter python=3.10 -c conda-forge -y
conda activate ai-painter

2. Install dependencies

pip install mediapipe==0.10.14 opencv-python numpy

Note: Pin mediapipe==0.10.14 — versions ≥ 0.10.20 remove the mp.solutions API used here.

3. Run

python AiVirtualPainter.py

By default uses webcam index 0. Change cv2.VideoCapture(0) to 1 if you have multiple cameras.

Virtual Mouse (bonus)

Requires autopy (Linux may need additional system dependencies):

pip install autopy
python AiVirtualMouse.py

Gesture	Action
Index finger only	Move cursor
Index + Middle (close together)	Left click

How It Works

1. MediaPipe detects 21 hand landmarks per frame at 30 fps
2. Finger state (up/down) is determined by comparing tip vs. pip landmark positions
3. Drawing strokes are painted onto a transparent canvas layer
4. The canvas is blended onto the camera feed via bitwise operations:
   • bitwise_and masks out the drawn region from the camera
   • bitwise_or composites the colored strokes on top

Dependencies

Package	Version	Purpose
mediapipe	==0.10.14	Hand landmark detection
opencv-python	≥4.x	Camera capture & drawing
numpy	≥1.x	Array operations
autopy	latest	Mouse control (virtual mouse only)