What it does

No mouse, no touchpad — just your hand in front of a webcam. Move your index finger and the cursor follows it. Pinch your thumb and index together to click. Pinch thumb and middle for a right click. Pinch thumb and ring, then move up or down, to scroll.

MediaPipe does the heavy lifting of finding 21 landmark points on your hand in real time; this project is really just the logic that turns those points into mouse behavior.

See it in action

index finger drives the cursor, a pinch fires a click — all live off the camera feed

Gestures

Gesture	Action
👆 Index finger extended	Moves the cursor
🤏 Thumb + index pinch	Left click
🤏 Thumb + middle pinch	Right click
🤏 Thumb + ring pinch, then move hand up/down	Scroll

How it works

┌───────────────┐     ┌──────────────────────┐     ┌────────────────────┐
│  Webcam Feed  │────▶│  MediaPipe Hands      │────▶│  Gesture Logic      │
│  (cv2.Video   │     │  (21 landmark points)  │     │  (pinch distances)  │
│   Capture)    │     │                        │     │                    │
└───────────────┘     └──────────────────────┘     └────────────────────┘
                                                              │
                                                              ▼
                                                  ┌────────────────────┐
                                                  │  PyAutoGUI          │
                                                  │  (move / click /    │
                                                  │   scroll)           │
                                                  └────────────────────┘

1. Every frame gets passed to MediaPipe, which returns 21 (x, y) points per detected hand
2. The index fingertip's position gets mapped from camera space to screen space, with smoothing so the cursor doesn't jitter
3. Distances between fingertip pairs are checked each frame — cross a threshold, and that gesture fires
4. A short cooldown after every click stops one pinch from registering as five clicks

Setup

git clone https://github.com/Amirzamani1l/hand-mouse.git
cd hand-mouse
python -m venv venv

Activate the virtual environment:

# Windows (PowerShell)
.\venv\Scripts\activate

# macOS / Linux
source venv/bin/activate

Then install dependencies:

pip install -r requirements.txt
python main.py

Needs a webcam — built-in laptop cam works fine. Press q to quit.

Troubleshooting (Windows)

MediaPipe on Windows has a couple of known rough edges. If you hit any of these, here's the fix:

ImportError: DLL load failed while importing _framework_bindings Install the Microsoft Visual C++ Redistributable, restart your machine, and try again. If it still fails, it's a known unresolved issue in newer MediaPipe builds on Windows — the version pin in requirements.txt (mediapipe==0.10.13) sidesteps it.

AttributeError: module 'mediapipe' has no attribute 'solutions' This means a newer, incompatible MediaPipe version got installed. Recent MediaPipe releases (0.10.21+) restructured their API and broke the legacy solutions module this project uses. Stick to the pinned version:

pip uninstall mediapipe -y
pip install mediapipe==0.10.13

A module that was compiled using NumPy 1.x cannot be run in NumPy 2.x Your NumPy is too new for MediaPipe. Fix:

pip install "numpy==1.26.4"

Using Anaconda and still hitting version conflicts? Anaconda base environments tend to have a lot of packages fighting over shared dependencies (numpy especially). Using a clean venv like above, instead of installing into (base), avoids this entirely.

PowerShell won't let you activate the venv (running scripts is disabled) Run this once per terminal session, then try activating again:

Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass

Configuration

All the tunables live in config.py:

Setting	What it controls
FRAME_MARGIN	how close to the camera's edge you need to reach to hit the screen's edge
SMOOTHING	higher = smoother cursor movement, but more lag
PINCH_THRESHOLD	how close two fingertips need to be to count as a pinch
CLICK_COOLDOWN_SECONDS	minimum time between two registered clicks
SCROLL_SENSITIVITY	how much hand movement it takes to scroll one unit

Project structure

hand-mouse/
├── main.py         camera loop, gesture routing
├── tracker.py       MediaPipe hand tracking wrapper
├── controller.py     cursor movement + click/scroll logic
└── config.py         all the settings

Notes

• Works best in decent, even lighting — MediaPipe's accuracy drops in low light
• If the cursor feels jittery, raise SMOOTHING; if it feels laggy, lower it
• FAILSAFE is disabled on PyAutoGUI on purpose (normally moving the mouse to a screen corner aborts the script) — if you want that safety back, remove the pyautogui.FAILSAFE = False line in controller.py
• Only tracks one hand at a time by default; bump max_hands in tracker.py if you want to experiment with two

License

MIT — do whatever you want with it.