ClusteredPi Learn how to build a cluster of Raspberry Pi's to learn more about distributed computing and to experiment with Docker Swarm. 25 February 2024 6 minute read By Kevin McAleer Share this article on Table of Contents Rack Mounted Clustered-PiBill of MaterialsRaspberry Pi 4SD CardsDisk ImagingConfigurationNetwork hub & CablesNetwork HubNetwork CablesChassis & PowerCluster ChassisPowerUPSPi 5 upgrades Tags: cluster raspberry pi docker Difficulty: beginner Category: raspberrypi Home Blog Clusteredpi ClusteredPi Learn how to build a cluster of Raspberry Pi's to learn more about distributed computing and to experiment with Docker Swarm. 25 February 2024 | 6 minute read | By Kevin McAleer | Share this article on Video For every project I create, I often make a corresponding YouTube video. Sometimes, there might be more than one video for a single project. You can find these videos in this section. Explore more through this this dedicated video. In December 2021 I decided to build a cluster of Raspberry Pi’s to learn more about distributed computing and to experiment with Docker Swarm. I had a few Raspberry Pi Zero’s lying around, so I decided to put them to good use. I started by installing the latest version of Raspberry Pi OS on each of the Pi Zero’s, and then I installed Docker on each of them. Once I had Docker installed, I was able to create a Docker Swarm cluster and deploy a simple web application to it - the clustered-pi website itself https://www.clustered-pi.com. The Clustered-Pi Logo I based the 3D printed design of the cluster on the original Cray-1 Super Computer. The Clustered-Pi Design based on the Cray-1 Super Computer The Clustered-Pi Design One of the challeneges with this design is that the Raspberry Pi Zero’s don’t have a network port, so I had to use the built-in WiFi to connect them to the network, this usually fine but with so much radio being broadcast in such close proxmity its not ideal. The next version of Clustered-Pi was built using Raspberry Pi 4’s, which have a built-in network port, so I was able to connect them to the network using a network cable. The Raspberry Pi 4 Clustered-Pi went on to host a number of web applications, including a blog, KevsRobots website https://www.kevsrobots.com which includes hosting a python based search engine https://search.kevsrobots.com/search?query=python and a few other web applications, read more about them here. Rack Mounted Clustered-Pi I sourced the rack mount from a local supplier (https://www.thepihut.co.uk), and it was a perfect fit for the Raspberry Pi 4’s. The Pi’s were powered through a 5V 20A power supply, and the network switch beneath the rack was powered by a 12V 5A power supply. Bill of Materials Item Description Qty Approx Price Total Raspberry Pi 4 4 £58 £232 64sdGb SD Card 4 £15 £60 Network Hub 1 £18 £18 CAT6 Cables 1 £13 £13 USB A to USB C Charging Cables 4 £6 £24 USB Charger - 3A per port 1 £15 £15 Raspberry Pi 4 Chassis 1 £42 £42 Grand Total £404 Raspberry Pi 4 Clustered-Pi consists of 4 Raspberry Pi 4 computers. Each one has slightly different sizes of memory, which is more to do with when they released these than a design choice. Raspberry Pi 4, 8Gb SD Cards I recommend getting a branded SD card, from a reputable supplier; in my experience unbranded SD Cards only last a couple of months under reasonable use, whereas the branded ones, particularly those designed for heavy write demands last years. 64Gb SanDisk Extreme SD Card Disk Imaging I used the official Raspberry Pi Imager Configuration Each node has a hostname that is unique: node01 node02 node03 node04 You set the hostname when first booting up the Raspberry Pi, or later using the raspi-config command, from the terminal. Network hub & Cables Lets look how to wire this all up. Network Hub I purchased a cheap 8 port network hub (a TP Link 8 port Gigabit Desktop Switch). This gives some room for expansion later on, and doens’t take up too much space on the desk. It also just happens to be the perfect size to sit underneath the Raspberry Pis. Network Cables I also bought bunch of colourful patch CAT6 cables to connect the hub to each Raspberry Pi. The cables are only 25cm long, so they fit neatly into the pi’s and hub. Network Hub Cables The network hub is connected to my router, so I can connect to them from my laptop, and more importantly, they are connected to the Internet so they can download Docker images. Chassis & Power Lets get our Pi’s Racked and stacked. Cluster Chassis I originally had the Raspberry Pi’s in their cases, just sat on top of the network hub, however this was shortly improved by purchasing a purpose built chassis. Not only does it make the Raspberry Pi’s more structurally stable, it provides additional cooling via 2 large fans behind the Pis. No Chassis With Chassis Power For Power, I bought some short 25cm USBA to USB C charger cables, and a 4 port USB charger plug that can deliver 3.1Amps per port. UPS This Cluster will be running a live website, accessible via the internet, so to ensure its always available I have powered it via an uninterruptable power supply (UPS), I used the Cyberpower BR1200ELCD BRIC Cyberpower UPS: 1200VA/720W, with 6 UK Outlets Pi 5 upgrades The Clustered-Pi 5 Upgrades In October 2023 the Raspberry Pi 5 was released, and this meant I could finally upgrade the Pi 4 cluster with more power, and more reliable and larger storage, via the NVMe SSD’s. I managed to source another rack enclosure, however these are now discontinued. Each Pi 5 is powered by the official 27W power supply, each also with a 1TB NVMe SSD, and a 1TB NVMe Base from Pimoroni (if you buy one of these, remember to include the kevin coupon code - I’ll thank you later). Read more about the upgrades here Liked this article? You might like these too. Cracking Enigma with Raspberry Pi The Enigma code, used by Germany during World War II for secure communications, was among the most complex ciphers. Discover how we can crack it using Raspberry Pi. Guiding Light Hey Robot Makers! Sync Files on your Pis, with Syncthing NextCloud You can host your own NextCloud server on your Raspberry Pi, and share files online with your friends and family. Buddy Jr. This project is perfect for beginners who want to get started with robotics and programming. You'll learn how to build a small robot arm using 4 servos and control it with a Raspberry Pi using Python. Robot Arms Extend your reach with these robotic arm projects. From simple pick and place robots to complex inverse kinematics, we have a range of projects to suit all skill levels.
ClusteredPi Learn how to build a cluster of Raspberry Pi's to learn more about distributed computing and to experiment with Docker Swarm. 25 February 2024 6 minute read By Kevin McAleer Share this article on Table of Contents Rack Mounted Clustered-PiBill of MaterialsRaspberry Pi 4SD CardsDisk ImagingConfigurationNetwork hub & CablesNetwork HubNetwork CablesChassis & PowerCluster ChassisPowerUPSPi 5 upgrades Tags: cluster raspberry pi docker Difficulty: beginner Category: raspberrypi
In December 2021 I decided to build a cluster of Raspberry Pi’s to learn more about distributed computing and to experiment with Docker Swarm. I had a few Raspberry Pi Zero’s lying around, so I decided to put them to good use. I started by installing the latest version of Raspberry Pi OS on each of the Pi Zero’s, and then I installed Docker on each of them. Once I had Docker installed, I was able to create a Docker Swarm cluster and deploy a simple web application to it - the clustered-pi website itself https://www.clustered-pi.com. The Clustered-Pi Logo I based the 3D printed design of the cluster on the original Cray-1 Super Computer. The Clustered-Pi Design based on the Cray-1 Super Computer The Clustered-Pi Design One of the challeneges with this design is that the Raspberry Pi Zero’s don’t have a network port, so I had to use the built-in WiFi to connect them to the network, this usually fine but with so much radio being broadcast in such close proxmity its not ideal. The next version of Clustered-Pi was built using Raspberry Pi 4’s, which have a built-in network port, so I was able to connect them to the network using a network cable. The Raspberry Pi 4 Clustered-Pi went on to host a number of web applications, including a blog, KevsRobots website https://www.kevsrobots.com which includes hosting a python based search engine https://search.kevsrobots.com/search?query=python and a few other web applications, read more about them here. Rack Mounted Clustered-Pi I sourced the rack mount from a local supplier (https://www.thepihut.co.uk), and it was a perfect fit for the Raspberry Pi 4’s. The Pi’s were powered through a 5V 20A power supply, and the network switch beneath the rack was powered by a 12V 5A power supply. Bill of Materials Item Description Qty Approx Price Total Raspberry Pi 4 4 £58 £232 64sdGb SD Card 4 £15 £60 Network Hub 1 £18 £18 CAT6 Cables 1 £13 £13 USB A to USB C Charging Cables 4 £6 £24 USB Charger - 3A per port 1 £15 £15 Raspberry Pi 4 Chassis 1 £42 £42 Grand Total £404 Raspberry Pi 4 Clustered-Pi consists of 4 Raspberry Pi 4 computers. Each one has slightly different sizes of memory, which is more to do with when they released these than a design choice. Raspberry Pi 4, 8Gb SD Cards I recommend getting a branded SD card, from a reputable supplier; in my experience unbranded SD Cards only last a couple of months under reasonable use, whereas the branded ones, particularly those designed for heavy write demands last years. 64Gb SanDisk Extreme SD Card Disk Imaging I used the official Raspberry Pi Imager Configuration Each node has a hostname that is unique: node01 node02 node03 node04 You set the hostname when first booting up the Raspberry Pi, or later using the raspi-config command, from the terminal. Network hub & Cables Lets look how to wire this all up. Network Hub I purchased a cheap 8 port network hub (a TP Link 8 port Gigabit Desktop Switch). This gives some room for expansion later on, and doens’t take up too much space on the desk. It also just happens to be the perfect size to sit underneath the Raspberry Pis. Network Cables I also bought bunch of colourful patch CAT6 cables to connect the hub to each Raspberry Pi. The cables are only 25cm long, so they fit neatly into the pi’s and hub. Network Hub Cables The network hub is connected to my router, so I can connect to them from my laptop, and more importantly, they are connected to the Internet so they can download Docker images. Chassis & Power Lets get our Pi’s Racked and stacked. Cluster Chassis I originally had the Raspberry Pi’s in their cases, just sat on top of the network hub, however this was shortly improved by purchasing a purpose built chassis. Not only does it make the Raspberry Pi’s more structurally stable, it provides additional cooling via 2 large fans behind the Pis. No Chassis With Chassis Power For Power, I bought some short 25cm USBA to USB C charger cables, and a 4 port USB charger plug that can deliver 3.1Amps per port. UPS This Cluster will be running a live website, accessible via the internet, so to ensure its always available I have powered it via an uninterruptable power supply (UPS), I used the Cyberpower BR1200ELCD BRIC Cyberpower UPS: 1200VA/720W, with 6 UK Outlets Pi 5 upgrades The Clustered-Pi 5 Upgrades In October 2023 the Raspberry Pi 5 was released, and this meant I could finally upgrade the Pi 4 cluster with more power, and more reliable and larger storage, via the NVMe SSD’s. I managed to source another rack enclosure, however these are now discontinued. Each Pi 5 is powered by the official 27W power supply, each also with a 1TB NVMe SSD, and a 1TB NVMe Base from Pimoroni (if you buy one of these, remember to include the kevin coupon code - I’ll thank you later). Read more about the upgrades here