Control a Wheelchair With Your Eyes and Jaw

In this project, we will control a Wheelchair in game using the Neuro PlayGround (NPG) Lite. By placing electrodes on your face, we can intercept the tiny electrical signals from your eyes (Electrooculography) and your jaw muscles (Electromyography). The NPG Lite processes these signals and acts as a Bluetooth keyboard, allowing you to "type" or play games hands-free.+

To complete this project, you will need the following hardware and software:

Hardware:

1. Neuro PlayGround Lite
2. 3x Gel Electrodes
3. 3x BioAmp Snap Cables
4. Nuprep skin preparation gel and Alcohol swabs
5. USB-C Cable
6. Laptop/ Desktop

Software:

1. Arduino IDE: With the ESP32 (v3.2.0) board package installed.
2. Libraries:
3. BleCombo (for Bluetooth Keyboard functionality)
4. Adafruit_NeoPixel (for the onboard status LEDs)
5. Firmware: The Wheelchair-Sim.ino file provided in the repository.

This project shows a proof of concept to control a wheelchair using bio-potential signals from your face. Using the Neuro PlayGround Lite, you can detect eye movements (EOG) and jaw clenches (EMG) and convert them into keyboard inputs.

With just three electrodes and one analog input, you can control movement keys in a game using eye movement and jaw clenches.

This is a great project for learning about Human-Computer Interaction (HCI) & Brain-Computer Interfaces (BCI),

In this project:

1. Eye movements → control left/right
2. Jaw clenches → control forward/backward
3. Signals are processed in real-time
4. Commands are sent via Bluetooth BLE keyboard

This allows you to control games and even a real wheelchair using facial muscle signals.

Apply Nuprep Skin Preparation Gel on the skin surface where electrodes would be placed to remove dead skin cells and clean the dirt from the skin surface. After rubbing the skin surface thoroughly, clean it with a wet wipe or an alcohol swab.

About Nuprep Gel:

Nuprep skin preparation gel is a mildly abrasive, highly conductive gel that should be applied before placing the electrodes on the skin to improve measurements. When applied gently, it strips away the top layer of skin and moistens the underlying skin layer, which reduces the skin impedance with minimal skin irritation and discomfort.

Complete guide on Skin Preparation: https://docs.upsidedownlabs.tech/guides/usage-guides/skin-preparation/

Let's understand the electrode placements before moving forward in this project.

1. Positive electrode → Outer corner of the left eye
2. Negative electrode → Outer corner of the right eye
3. Reference electrode → Bony area behind the earlobe

This setup captures both:

1. EOG signals from eye movement
2. EMG signals from jaw muscles

Once your skin is prepared, it’s time to connect BioAmp Snap Cables.

Connect 3 BioAmp Snap Cables to

1. Positive of Channel 1 to A0P
2. Negative of Channel 1 to A0N
3. Reference cable to REF

Install Arduino IDE (https://www.arduino.cc/en/software) and launch it.

Install the following board in Arduino IDE

1. Install board manager: ESP32.

Tools → Board Manager → search for esp32 → Install esp32 by Espressif Systems(version - 3.2.0 compulsary).

Install the following library in Arduino IDE

1. Install library: Adafruit NeoPixel.

Sketch → Include libraries → Manage libraries → Search for Adafruit NeoPixel → Install Adafruit NeoPixel

Go to this link, click on Code and Download Zip.

In Arduino IDE, go to Sketch - Include Library - Add .zip Library and select the downloaded zip

Turn ON the NPG Lite device, connect it to your Laptop with a Type-C cable, and then follow the steps below:

Open the project on Github and copy and paste the code.

A. NPG Lite Device

1. Select the connected ESP32 board

Tools → Board → esp32 → esp32c6 Dev Module.

1. Enable CDC On Boot in the Tools menu.

Tools → USB CDC ON BOOT "ENABLED".

Click on the upload button.

Electrical interference can affect signals.

The firmware includes a notch filter to remove powerline noise.

Set the correct frequency:

Set to 50 or 60 according to your region's powerline noise.

The system converts facial signals into keyboard inputs.

At the top of Wheelchair-Sim.ino, find these four lines:

Change the letter inside the single quotes to whichever key your game requires and re-upload.

Example: If you want to use arrow keys instead of WASD, you would change them to:

For standard letter or number keys, use single quotes like 'q' or '1'. For special keys like arrow keys, spacebar, or Enter, use the BLE HID constants like KEY_LEFT_ARROW, KEY_RETURN, or KEY_SPACE.

For testing and calibration, you can enable debug output.

Set the debug level in the firmware:

Options:

1. 0 → Debug OFF
2. 1 → Jaw signal debug
3. 2 → Eye movement debug

Open the Serial Monitor (115200 baud) to view signal data.

Debugging

Using Debug Mode

At the top of Wheelchair-Sim.ino, find this line:

Set it to 1 or 2, re-upload, then open Serial Monitor at 115200 baud.

DEBUG_LEVEL 1 — Prints JAW_ENV: <value> every 200ms. Clench your jaw and watch the number rise.

​DEBUG_LEVEL 2 — Prints EYE_ABS: <value> every 200ms. Look left or right and watch the number rise.

​

Set it back to 0 when done tuning.

Tuning Thresholds

Jaw (DEBUG_LEVEL 1):

Clench your jaw multiple times the same way you would while controlling the game. Note the max JAW_ENV value you see. Then set:

JAW_THRESHOLD → 80% of your max value

JAW_RELEASE_THRESHOLD → 40% of your max value

For example, if your max clench value is 200, set JAW_THRESHOLD to 160 and JAW_RELEASE_THRESHOLD to 80.

Eye (DEBUG_LEVEL 2):

Look left and right a few times the way you naturally would during gameplay. Note the max EYE_ABS value you see. Then set:

EYE_MOVEMENT_THRESHOLD → 80% of your max value

For example, if your max eye movement value is 180, set EYE_MOVEMENT_THRESHOLD to 144.

Updating Key Mappings

At the top of Wheelchair-Sim.ino, find these four lines:

Change the letter inside the single quotes to whichever key your game requires and re-upload.

Example: If you want to use arrow keys instead of WASD, you would change them to:

For standard letter or number keys, use single quotes like 'q' or '1'. For special keys like arrow keys, spacebar, or Enter, use the BLE HID constants like KEY_LEFT_ARROW, KEY_RETURN, or KEY_SPACE.

For testing and calibration, you can enable debug output.

Set the debug level in the firmware:

Options:

1. 0 → Debug OFF
2. 1 → Jaw signal debug
3. 2 → Eye movement debug

Open the Serial Monitor (115200 baud) to view signal data.

Once powered, the device appears as:

Open your computer’s Bluetooth settings and pair the device.

The board will now act as a BLE keyboard.

The firmware processes signals in multiple stages:

1. Raw Signal Acquisition

1. Analog input A0
2. Sampled at 500 Hz

2. Notch Filtering

Removes powerline interference (50/60 Hz).

3. Eye Movement Detection

Pipeline:

Detects left and right eye movements.

4. Jaw Clench Detection

Detects single and double jaw clenches.

5. Output

Commands are sent as:

1. BLE keyboard inputs
2. LED indicators using NeoPixels