Rock Paper Scissors With Hand Tracking AI (No Coding Required)

Play Rock Paper Scissors against your Raspberry Pi using AI-powered hand gesture recognition! The AI detects your hand in real time, picks a random opponent move, and announces the result with sound effects — all without writing a single line of code.

Hardware:

1. Raspberry Pi 4 (or above)
2. Camera (Raspberry Pi Camera Module or USB Camera)
3. Speaker (3.5mm, USB, or Bluetooth)

Software:

1. Grablo — No-Code IoT Platform (grablo.co)

Want to skip the build and play right away? Copy the project from the Grablo Gallery:

1. Download and install Grablo software on your Raspberry Pi from grablo.co/download
2. Connect a camera and speaker to your Raspberry Pi
3. Get this project from the Grablo Gallery
4. Open the Grablo app at app.grablo.co in your browser
5. Connect to your Raspberry Pi and hit RUN!
6. Press Start on the dashboard and show your hand to the camera!

If you'd like to build it from scratch, follow the steps below.

Create a new project at app.grablo.co, name it "Rock Paper Scissors," and select your Raspberry Pi model. Then create a dashboard and add the following widgets: a Camera widget for the live video feed, two Push Buttons for Start and Stop, two Image Switch widgets to show the opponent's hand and the round result, and two Labels for the score and game status.

This project uses 3 logics with 7 controls total. The first logic simply starts the camera when the project launches. The second logic handles the Start and Stop buttons — pressing Start activates AI hand tracking and sets the game to "Playing," while pressing Stop clears the analysis and resets the scores. It also continuously updates the score display on the dashboard.

The third logic is the main game loop. When a round begins, it plays the "Rock, Paper, Scissors, Shoot!" audio prompt, then waits for the AI to detect your hand gesture — Closed Fist for Rock, Open Palm for Paper, or Victory Sign for Scissors. Once captured, a Blockly script randomly picks the opponent's hand, determines the winner, updates the scores, and plays a win, lose, or tie sound effect. The game then loops back for the next round.

Launch Your Project

1. Open the Grablo app and select your project
2. Connect to your Raspberry Pi
3. Hit RUN!
4. Press Start on the dashboard and show your hand gesture to the camera

Expected Results

1. The camera feed shows your hand in real time
2. After the audio prompt, show your gesture — the AI captures it instantly
3. The opponent's random choice appears as an animated image
4. A sound effect plays for win, lose, or tie
5. Scores update automatically — press Stop to reset and play again

Expand Your Project

Now that you have a working Rock Paper Scissors game, here are some ideas to take it further:

1. Add more gestures (Lizard, Spock) for an extended game
2. Display a win streak counter on the dashboard
3. Add LED indicators that light up on win or lose
4. Connect a servo motor to make a physical hand that shows the AI's choice
5. Log game history to track your performance over time

Troubleshooting

Hand gesture not detecting:

1. Ensure good lighting — avoid backlighting or shadows on your hand
2. Hold each gesture (fist, open palm, peace sign) steadily for 1–2 seconds
3. Make sure only one hand is visible to avoid confusion

No sound playing:

1. Check that your speaker is properly connected
2. On Raspberry Pi, set the correct audio output (3.5mm vs HDMI) in system settings

Camera feed not showing:

1. For CSI cameras, ensure the ribbon cable is securely seated
2. For USB cameras, try a different USB port