You’ve seen googley eyes or xeyes, but what if the googley eyes could Google what they see (and then announce it); I present googley googley eyes. (Ok, they don’t actually ‘Google’ the image, they actually use something far more cool - OpenAI).

The eyes rotate using the motors with encoders for accurate positioning. This means they can rotate all the way round, but also rotate to a specific position, which is needed for making the eyes look in a direction accurately.

Bill of Materials

3d printed parts

Here are the 3d printed parts for the project:

• pi_holder.stl - Holds the Raspberry Pi, this is from a modular robotics system I’m working on
• eye_holder - Holds the motors and provides a desktop mount for the project
• eye.stl - These are the white eye pieces, you’ll need to print two of them
• pupil.stl - These are the black, pupil pieces, you’ll need to print two of them

Assembley

1. Setup the Raspberry Pi with a fresh install of Raspberry Pi OS 64-bit, Click here for a how to video

2. Push the Pimoroni Inventor HAT Mini onto the top of the Raspberry Pi Zero 2W

3. Connect the motors up to the Inventor Hat Mini

4. Push the motors into the eye_holder

5. If you have access to a vinyl cutting machine such as a Cricut Maker 3, you can cut out a circle 50mm in diameter for the white eye, and 30mm for the black pupil. The White eye has a small 7mm hole in the center. The 3d printed parts may require some filing to ensure a smooth motion

6. Push the black pupil 3d printed part into the white eye piece

7. Glue the eye 3d printed disk onto the eye_holder with some super-glue, be sure not to glue the pupil part ensuring it can still turn friction free

8. Screw the Raspberry Pi Zero 2W into the pi_holder using the M2.5 screws

9. Carefully push the Raspberry Pi camera cable into both the Raspberry Pi camera module and the Raspberry Pi Zero 2W

10. Screw the Raspberry Pi camera module into the eye_holder with some M2 screws

11. Push the small PicoBlade speak cable into the speaker connector on the Inventor HAT Mini

Setting up the image recognition

OpenAI has a nice automatic image captioning service, you can get this up and running by:

1. Download the project code:

    git clone https://www.github.com/kevinmcaleer/googley_eyes
   

2. Sign up for an account on https://platform.openai.com

3. Create a new API key: Click on your profile picture at the top right, and then View API Keys, then click create new secret key

4. Copy the secret API key into a file named .env into the downloaded code:

    OPENAI_API_KEY=<INSERT YOUR OPENAI API KEY HERE>
   

5. Create a virtual environment:

    python3 -m venv venv
   

6. Activate the environment:

    source venv/bin/activate
   

7. Install the dependencies:

    pip install -r requirements.txt
   

8. Running the demo program caption_this.py:

    python3 caption_this.py
   

9. You can upload your own image and update the code that loads the image.

Tweak the code

You can also tweak the query, to ask OpenAI how to describe the image, here is a response generated from the query:

result = index.query('describe what is in the image, be nonchalant and snarky')

Results

“A cat and dog lounging on a blanket, like they own the place.”

Making the eyes move

The eyes can be moved using the Pimoroni Inventor HAT mini python library; for instructions on how to install that, click here.

Once you’ve installed that, headover to the motors example code and run the motor_wave.py program:

cd InventorHATMini/examples/motors
python3 motor_wave.py

Code

Here is the code you will need, please note this is a work in progress so the code will improve over time.