Cover image for Build your own AI Assistant in Python


Watch the associated videos here:

Hey Robot Makers! Today, we’ll explore an exciting topic. We’re going to show you how to build your own AI assistant using Python. This endeavor promises to be an intriguing journey, so let’s get right into it!


Before we dive in, remember this is going to be a series. We won’t achieve everything in one session because there’s quite a bit to cover. In this part, we will go over the basics and other necessary elements to create your AI Assistant. We’ll discuss how AI assistants work, what you’ll need, how to add AI skills and discuss the exciting possibilities of what we can expect to achieve with these skills.

It’s important to note that this is more towards an intermediate level. It’s probably not the best fit for absolute beginners just stepping into the world of Python. This journey will require a certain level of Python familiarity.

Getting Started: Required Hardware and Software

To create our AI assistant, we require a Raspberry Pi computer. I am personally using a Raspberry Pi 4, but it can also work on a Raspberry Pi Zero. Additionally, you will need a USB microphone and speakers to interact with your AI.

On the software side, we’ll use a few necessary tools including: the Raspberry Pi and Raspbian, GitHub for source code control, Python version 3, and Visual Studio Code on the Raspberry Pi for editing our scripts.

The AI Assistant Arbor

Our AI Assistant will have four main components:

  1. Speech Engine: This engine will be responsible for speech recognition and synthesis.
  2. Web User Interface: This will be our visual output built with the Flask python web framework.
  3. Orchestration Engine: This will be the main control loop that checks for commands and delegates them to the appropriate functions.
  4. Skills: These will be additional modules that we can add to our AI Assistant. These will interact with various web services like weather forecasts, stock information, and more.

Our AI assistant will work with a main loop that continuously listens for speech. It will process the interpreting, take necessary actions, and give responses through the web UI.

Speech Engine Building

Let’s talk about the foundation of our AI assistant — the speech engine. There are a few prerequisites for this engine; ensure port audio and eSpeak are installed. Then, we will install the ‘pyaudio’, ‘speech recognition’ and ‘Python text to speech x3’ packages in Python using pip.

The speech engine includes methods for our Assistant to listen and speak. Listening is as simple as capturing a sample from the microphone, and then sending it to Google’s speech recognition service which gives us the interpreted command as a phrase.

Speaking is also straightforward. We use the Python package ‘Python text to speech x3’ to convert a string into spoken text which the Assistant then vocalizes.

Future Steps

That’s all for the first part of the series. Moving forward, we will create a framework for adding skills to our assistant. Each skill will be a class with a specific function, version, success/failure status, and possibly a web UI for interactions.

Next time, we’ll dig deeper into adding skills to our assistant like Home Automation, Weather Inquiry, Stocks & Shares updates and more. The opportunity with this AI assistant is endless. Once the base is built, we can continually add new skills, making our small robot assistant smarter and more helpful.

So there you have it! We’ve built the foundation of an AI assistant with Python, and we’re set to take the next steps. Tune in next time as we equip our assistant with more skills and functionalities.

Happy Coding!

Did you find this content useful?

If you found this high quality content useful please consider supporting my work, so I can continue to create more content for you.

I give away all my content for free: Weekly video content on YouTube, 3d Printable designs, Programs and Code, Reviews and Project write-ups, but 98% of visitors don't give back, they simply read/watch, download and go. If everyone who reads or watches my content, who likes it, helps fund it just a little, my future would be more secure for years to come. A price of a cup of coffee is all I ask.

There are a couple of ways you can support my work financially:

If you can't afford to provide any financial support, you can also help me grow my influence by doing the following:

Thank you again for your support and helping me grow my hobby into a business I can sustain.
- Kevin McAleer