Cover image for Ollama - local ChatGPT on Pi 5


Watch the associated video here:

Greetings Robot Makers! If you’re eager about creating, developing, and programming robots and you love exploring these matters in an enjoyable and hands-on manner, then you’ve landed at the right place.

Today, we’re heading into an adventure of establishing your private GPT server, operating independently and providing you with impressive data security via Raspberry Pi 5, or possibly, a Raspberry Pi 4. If this piques your interest, buckle up and let’s get straight into it!

What is Olama?

Olama is an offline AI that performs similarly to ChatGPT. Olama is designed to function entirely offline and locally, which implies you can engage in text-based conversations, share codes, photos or documents and receive an interactive responses without a grain of your data being compromised.

With the use of Raspberry Pi 5 operating through Docker, we’ll be guiding you through the process of installing and setting up Olama along with its web user interface, which bears a striking resemblance to Chat GPT. Rest assured, though it might seem complicated at first, the process is easy to navigate.

Ollama with WebUI Screenshot

Ollama Screenshot

Advantages of Olama: Why Choose It?

It’s natural to question the benefits of using Olama. Here, we’ll shed light on the distinctive attributes it brings to the table:

Privacy and Security One of the highlights of Olama lies in its promise of privacy. With all your data being processed locally, you maintain the complete confidentiality of your data. There’s zero data sharing with third parties, making it an ideal choice for those who prioritize privacy.
Accessibility and Reliability Another advantage is its ability to perform without relying on internet connectivity. It operates independently of external servers, opening new frontiers of opportunities for its use.
Customization One of the appealing features of Olama is the profound level of customization it extends to users. Instead of being confined to a particular AI model, you have the freedom to select whichever one suits your needs. From coding-specific language models to analytic models for image processing, you have the liberty to choose the perfect model for your requirements.
Cost Efficiency Last but not least, Olama shines for being cost-effective. It carries no ongoing subscription fees, and once you download the models, you save significantly on internet bandwidth costs.

Setting Up Olama: A Step-by-Step Guide

Before proceeding with the setup, it’s pivotal to note that a Raspberry Pi 5 with 8GB RAM is the ideal machine for maximizing the potential of running Olama. Though, Olama can still function on a Raspberry Pi 4 albeit at a (much) slower pace.

Preliminary Set-Up

Our vital first step is creating two separate commands: one dedicated to Olama and another for its web user interface to enhance smooth operation. It’s imperative to have your Raspberry Pi’s operating system and Docker updated to evade any potential issues and enhance the overall performance.

  1. Install Docker (see this free course if you’ve never used Docker before):
  2. Download the latest snapshot of ollama-webui:
      git clone webui
  3. Create a docker-compose file (note you will need to update the volumne path to match your local setup; you may not be using the pi username):
      version: "3.9"
       container_name: ollama
       image: ollama/ollama:latest
       restart: always
         - /home/pi/ollama:/root/.ollama
         context: ./webui/
           OLLAMA_API_BASE_URL: '/ollama/api'
         dockerfile: Dockerfile
       container_name: ollama-webui
         - ollama-webui:/app/backend/data
         - ollama
         - ${OLLAMA_WEBUI_PORT-3000}:8080
         - 'OLLAMA_API_BASE_URL=http://ollama:11434/api'
         - host.docker.internal:host-gateway
       restart: unless-stopped
     ollama-webui: {}
     ollama: {}
  4. Bring the container up:
      docker-compose up -d
  5. Access the webui at http://localhost:3000
  6. Create a free account for the first login
  7. Download the model you want to use (see below), by clicking on the little Cog icon, then selecting Models

    Pull a Model for use with Ollama

    download model

  8. For a list of Models see the ollama models list on the Ollama GitHub page

Running Olama on Raspberry Pi

With the setup finalized, operating Olama is easy sailing. Be it on Raspberry Pi 4 or 5, expect outstanding performance, though keep in mind that the response time tends to be slower on the Raspberry Pi 4.

What’s remarkable is Olama’s autonomous operation, working efficiently without reliance on internet connectivity or connection to an external server.

Interacting with Language Models

With Olama, you can ask the system to tell a joke or engage in more complex tasks. The response speed depends on the complexity of the model being used. The impressive bit is that even without an active internet connection, it operates flawlessly, establishing Olama as a powerful offline AI.

How to use Python with Ollama and LangChain

LangChain is a framework, set of tools and libraries for working with language models. It is designed to be used with Ollama, but can be used with any language model.

Here is a simple example of how to use LangChain with Ollama:

from langchain_community.llms import Ollama

model = "llama2"

llm = Ollama(model=model)

question = "tell me a joke"

response = llm.invoke(question)

print(f"response is {response}")

Note: you will need to install the langchain-community package first, using pip install langchain-community

Conclusion: Exploring the Wonders of Olama and Raspberry Pi

By harnessing Olama’s potential to operate positive and secured language models, we can take our robot-building and programming abilities to an all-time high. While the complexity might seem overwhelming, it’s surprising how approachable, practical, and fruitful this venture can turn out to be.

So, gear up and set foot into the exciting domain of independent local programming with Olama and Raspberry Pi. Let your creativity flow as you build your private chat GPT server and uncover myriads of possibilities that lay within the realm of AI. Happy programming!

Did you find this content useful?

If you found this high quality content useful please consider supporting my work, so I can continue to create more content for you.

I give away all my content for free: Weekly video content on YouTube, 3d Printable designs, Programs and Code, Reviews and Project write-ups, but 98% of visitors don't give back, they simply read/watch, download and go. If everyone who reads or watches my content, who likes it, helps fund it just a little, my future would be more secure for years to come. A price of a cup of coffee is all I ask.

There are a couple of ways you can support my work financially:

If you can't afford to provide any financial support, you can also help me grow my influence by doing the following:

Thank you again for your support and helping me grow my hobby into a business I can sustain.
- Kevin McAleer