HandPi Kiosk: Gesture-Controlled Kiosk Player on Raspberry Pi OS Lite

As the Raspberry Pi platform has evolved, it is now capable of real-time computer vision and interactive media applications. When paired with Raspberry Pi OS Lite, background services and GPU overhead are minimized, allowing more resources for MediaPipe hand tracking and low-latency video playback.

HandPi Kiosk is an open-source gesture-controlled video player designed to run headless on Raspberry Pi OS Lite.

It uses MediaPipe for hand tracking and MPV for fullscreen video playback.

The system auto-starts on boot, loops a MAIN video, and allows users to browse and control secondary videos entirely via hand gestures.

The project can be adapted for kiosks, exhibitions, and installations where touchless interaction is required.

Before You Begin:

This project was tested on:

1. Raspberry Pi 5 (8 GB RAM)
2. Raspberry Pi 400 (4 GB RAM)

Both systems were running Raspberry Pi OS Lite (64-bit, Bookworm) and provided reliable performance for MediaPipe.

Hardware

1. Raspberry Pi 5 or Raspberry Pi 4 / 400 ~ €60–€90
2. USB webcam (Full HD) or Raspberry Pi Camera Module V3 ~ €25–€35
3. HDMI compatible display ~ €80–€150
4. Speakers (optional) ~ €10–€30

Software

1. Raspberry Pi OS Lite (64-bit, Bookworm) — Free
2. Python 3 — Free
3. MediaPipe — Free / Open source
4. MPV — Free / Open source
5. OpenCV — Free / Open source

1. MPV starts in fullscreen on boot
2. MAIN video loops continuously
3. Gesture-controlled menu opens on demand
4. Secondary videos play once, then return to MAIN
5. No mouse, keyboard, or touch input required after installation
6. The kiosk runs directly on the Linux console (no desktop environment).

MAIN video

1. Thumb + Index (hold) → Open menu
2. Thumb + Pinky (hold) → Next video

Menu

1. Thumb + Middle (hold) → Scroll
2. Thumb + Ring (hold) → Play selected

Playback

1. Palm (hold) → Pause / Resume
2. Thumb + Index (hold) → Return to MAIN
3. Thumb + Pinky (hold) → Next video
4. Thumb + Middle / Ring (hold) → Volume up / down

Gestures use stability frames, hold duration, and cooldowns to avoid false triggers.

The full project is available on my GitHub repository (https://github.com/Thanos3G/HandPi-Kiosk). To install it, clone the repository on your Raspberry Pi and run the included install script.

Project Structure (Raspberry Pi)

All specific Raspberry Pi files are located in the pi/ folder:

Add Your Videos

1. Place exactly one video in videos/main/
2. → this is the looping MAIN video
3. Place any number of videos in videos/secondary/
4. → these are browsed and played via gestures
5. Secondary videos are loaded alphabetically, so name them 01_, 02_, etc.

Once the install script has completed and videos are in place, reboot the Raspberry Pi.

The kiosk will start automatically.

1. Gesture thresholds are defined in main_pi.py
2. Overall it worked better with Full HD USB camera but Pi Camera V3 wasn't that bad
3. Menu OSD is optimized for 1920×1080 but can be adjusted
4. USB audio devices are preferred, HDMI is fallback

If the kiosk does not start, check the systemd service: