Gesture Controlled Web Automation using Computer Vision

This project is a real-time computer vision application that allows users to open websites using hand gestures captured through a webcam. The system detects finger-count gestures and opens predefined websites securely in Google Chrome Guest Mode.

The application also records both the system screen and webcam feed simultaneously, providing clear proof of gesture-based system interaction.

Features

✔ Real-time hand gesture detection using OpenCV and MediaPipe ✔ Finger count–based gestures (no hand movement required) ✔ Left and Right hand differentiation ✔ Secure website opening in Google Chrome Guest Mode ✔ Each website opens only once per session (no repetition) ✔ Multi-person detection blocked for reliability ✔ Simultaneous recording of:

System screen (opened websites)

Webcam feed (gesture detection) ✔ Both streams merged into a single video file

How It Works

The webcam captures hand gestures in real time

The system counts the number of fingers shown

Each gesture is mapped to a specific website

When a valid gesture is held for a short time:

The website opens in Chrome Guest Mode

The event is recorded in the output video

The recorded video shows gestures and website opening side-by-side

Tech Stack

Python

OpenCV

MediaPipe

MSS (system screen capture)

NumPy

Output

A real-time OpenCV window showing gesture detection

Google Chrome opening websites in guest mode

A recorded video file combining:

System screen

Webcam feed

Use Cases

Touchless system control

Accessibility applications

Computer vision demonstrations

Smart workspace automation

AI and CV academic or industry projects