Cover image for ClusteredPi

ClusteredPi

Learn how to build a cluster of Raspberry Pi's to learn more about distributed computing and to experiment with Docker Swarm.


 25 February 2024   |     6 minute read   |   By Kevin McAleer   |   Share this article on

Video

For every project I create, I often make a corresponding YouTube video. Sometimes, there might be more than one video for a single project. You can find these videos in this section.

Explore more through this this dedicated video.

In December 2021 I decided to build a cluster of Raspberry Pi’s to learn more about distributed computing and to experiment with Docker Swarm. I had a few Raspberry Pi Zero’s lying around, so I decided to put them to good use.

I started by installing the latest version of Raspberry Pi OS on each of the Pi Zero’s, and then I installed Docker on each of them. Once I had Docker installed, I was able to create a Docker Swarm cluster and deploy a simple web application to it - the clustered-pi website itself https://www.clustered-pi.com.


The Clustered-Pi Logo

Clustered-Pi


I based the 3D printed design of the cluster on the original Cray-1 Super Computer.

The Clustered-Pi Design based on the Cray-1 Super Computer

Clustered-Pi

The Clustered-Pi Design

Clustered-Pi


One of the challeneges with this design is that the Raspberry Pi Zero’s don’t have a network port, so I had to use the built-in WiFi to connect them to the network, this usually fine but with so much radio being broadcast in such close proxmity its not ideal.

The next version of Clustered-Pi was built using Raspberry Pi 4’s, which have a built-in network port, so I was able to connect them to the network using a network cable.

The Raspberry Pi 4 Clustered-Pi went on to host a number of web applications, including a blog, KevsRobots website https://www.kevsrobots.com which includes hosting a python based search engine https://search.kevsrobots.com/search?query=python and a few other web applications, read more about them here.


Rack Mounted Clustered-Pi

I sourced the rack mount from a local supplier (https://www.thepihut.co.uk), and it was a perfect fit for the Raspberry Pi 4’s. The Pi’s were powered through a 5V 20A power supply, and the network switch beneath the rack was powered by a 12V 5A power supply.


Bill of Materials

Item Description Qty Approx Price Total
A Raspberry Pi 4 Raspberry Pi 4 4 £58 £232
An SD Card 64sdGb SD Card 4 £15 £60
A network switch Network Hub 1 £18 £18
some network cables CAT6 Cables 1 £13 £13
a usb cable USB A to USB C Charging Cables 4 £6 £24
usb power supply USB Charger - 3A per port 1 £15 £15
a raspberry pi cluster chassis Raspberry Pi 4 Chassis 1 £42 £42
      Grand Total £404

Raspberry Pi 4

Clustered-Pi consists of 4 Raspberry Pi 4 computers. Each one has slightly different sizes of memory, which is more to do with when they released these than a design choice.

Raspberry Pi 4 Raspberry Pi 4, 8Gb


SD Cards

I recommend getting a branded SD card, from a reputable supplier; in my experience unbranded SD Cards only last a couple of months under reasonable use, whereas the branded ones, particularly those designed for heavy write demands last years.

sd cards 64Gb SanDisk Extreme SD Card

Disk Imaging

I used the official Raspberry Pi Imager


Configuration

Each node has a hostname that is unique:

  • node01
  • node02
  • node03
  • node04

You set the hostname when first booting up the Raspberry Pi, or later using the raspi-config command, from the terminal.


Network hub & Cables

Lets look how to wire this all up.


Network Hub

I purchased a cheap 8 port network hub (a TP Link 8 port Gigabit Desktop Switch). This gives some room for expansion later on, and doens’t take up too much space on the desk.

It also just happens to be the perfect size to sit underneath the Raspberry Pis.


Network Cables

I also bought bunch of colourful patch CAT6 cables to connect the hub to each Raspberry Pi. The cables are only 25cm long, so they fit neatly into the pi’s and hub.


   
TP Link 8 Port Hub Cables
Network Hub Cables

The network hub is connected to my router, so I can connect to them from my laptop, and more importantly, they are connected to the Internet so they can download Docker images.


Chassis & Power

Lets get our Pi’s Racked and stacked.

Cluster Chassis

I originally had the Raspberry Pi’s in their cases, just sat on top of the network hub, however this was shortly improved by purchasing a purpose built chassis. Not only does it make the Raspberry Pi’s more structurally stable, it provides additional cooling via 2 large fans behind the Pis.

   
No Chassis Chassis
No Chassis With Chassis

Power

For Power, I bought some short 25cm USBA to USB C charger cables, and a 4 port USB charger plug that can deliver 3.1Amps per port.


UPS

This Cluster will be running a live website, accessible via the internet, so to ensure its always available I have powered it via an uninterruptable power supply (UPS), I used the Cyberpower BR1200ELCD BRIC

Chassis

Cyberpower UPS: 1200VA/720W, with 6 UK Outlets


Pi 5 upgrades

The Clustered-Pi 5 Upgrades

Clustered-Pi

In October 2023 the Raspberry Pi 5 was released, and this meant I could finally upgrade the Pi 4 cluster with more power, and more reliable and larger storage, via the NVMe SSD’s.

I managed to source another rack enclosure, however these are now discontinued.

Each Pi 5 is powered by the official 27W power supply, each also with a 1TB NVMe SSD, and a 1TB NVMe Base from Pimoroni (if you buy one of these, remember to include the kevin coupon code - I’ll thank you later).

Read more about the upgrades here