(LiM) Kubernetes Cluster set up using Ansible
18 Dec 2022Inspired by “Network Chuck”’s video tutorial “i built a Raspberry Pi SUPER COMPUTER!! // ft. Kubernetes (k3s cluster w/ Rancher)” on youtube and corresponding course on his website as well as Jeff Geerling’s video channel and blog, I set out on four journeys:
- The first led me down my father’s foot steps of electrical engineering designing and building a custom carrier board for the Raspberry Pi compute module.
- The second, learning and creating kubernetes clusters on Raspberry Pis with kubernetes/k3s - a light weight version of kubernetes and managing them with Rancher.
- The third, learning a new to me way of automating/simplifying administrative tasks, cluster installs and management using Ansible.
- The fourth, was designing a cluster tray that would house my cluster in a unique way, inspred by toilet paper roles and a Pringles tube. But more about that later.
I should add that I am well down the path of my career but I always enjoyd tech and Linux. However, I am doing all of this as a hobby. I am sure there is many different (and possibly better) ways of doing things but this was my way.
Journey 1 - Developing the LiM Cluster CM4 Carrier Board
Watching Jeff Geerling’s video New Raspberry Pi Projects - CM4 NAS, Piunora, and Seaberry! where he talks about mebs_t’s self designed NAS, made me think I am interested in doing somehting like this too. At the time, the use case I was woking on was staking Navcoin headlessly with a Raspberry Pi Compute Module 4..
I was looking for a Raspberry Pi Compute Module 4 (CM4) carrier board that only provides power to a CM4 with emmc storage. Many of the carrier boards out at the time provided a lot more functionality so set out to design and build the Less-is-More (LiM) carrier board.
The RPi CM4 - LiM Carrier Board series
The LiM Carrier Board series is a series of minimalistic Raspberry Pi (RPi) Compute Module 4 (CM4) Carrier Boards. Less-is-More (LiM) refers to the minimalistic design only providing the most rudimentary functionality to the CM4 such as 5V power via USB-C power connector and two status (power/activity) LEDs for the original LiM Carrier Board version. Additional + versions of the LiM Carrier Board will/might feature additional functionality such as flashing capability and SD card slot (see conceptual table below).
Except for the LiM CM4 Cluster Carrier Board, the LiM and LiM+ Carrier boards are meant for CM4 boards with onboard storage (the LiM CM4 Cluster Carrier Board supports both CM4 versions with and w/o eMMC (lite version), micro SD card and 2232 and has a 2241 M.2 M-key socket for NVMe PCIe SSD) and wifi as the LiM and LiM+ carrier boards provides no other functionalty other than power and status LEDs. The LiM CM4 Cluster Carrier Board has PoE ethernet and supports wifi and non-wifi CM4 models.C
Various LiM Carrier Board models
Three versions of the LiM Carrier Board series have been prototyped and more information can be found on their respective pages
The original LiM carrier board took about two months from idea (May 22nd, 2021) to delivery (July 14th, 2021). Special thanks to Anish Verma for working with me on the CAD designs of the carrier board.
I was looking for such a minimalistic board and had a conversation with Jeff Geerling when I asked him if he knew of a more minimalistic design than this board? He pointed me to to uptime.lab’s Upberry.
Neither of these boards are exactly what I was looking for for my use case. So iI was motivated to design my own, customized board to my specs and that’s how the Less-is-More board series came to be.
I forked this board design from Shawn Hymel. He has a two part youtube series where he goes through how to design a CM4 Carrier Board.
- Part 1 - How to Make a Raspberry Pi Compute Module 4 Carrier Board in KiCad
- Part 2 - How to Make a Raspberry Pi Compute Module 4 Carrier Board in KiCad
Feel free to modify this design for your own application.
Here is the current sepc and feature list of designed and planned carrier boards:
| Model | Power (5V USB-C) | LEDs (Power/Activity) | Flashing Capability | SD card slot | Development Stage |
|---|---|---|---|---|---|
| LiM Board | ✔ | ✔ | ❌ | ❌ | prototype |
| LiM+ Board | ✔ | ✔ | ✔ | ❌ | prototype |
| LiM++ Board | ✔ | ✔ | ✔ | ✔ | abandoned |
While working onthe LiM++ Board I, got interested in clustering and decided to switch directions. I wanted to add a ccouple more features that coule be of value for a cluster type carrier board. And so the LiM CM4 Cluster Carrier Board idea was born. The LiM CM4 Cluster Board has the following features:
- RPi Pi Compute Module 4 (CM4) support (both versions with and w/o eMMC - Lite version)
- PoE Gigabit RJ45 port (GbE)
- USB-C (USB 2.0) for CM4 module flashing (power + data)
- toggle switch for boot/flash mode
- microSD card slot
- 2232 and 2241 M.2 M-key socket dedicated for NVMe PCIe SSD drives
- Fan Pins
- SDA, SDC, YCC, GND pins for OLED Displays
This is a summary of my Journey 1 and how I designed and built a custom carrier board for the cluster idea I
Journey 4 - The (LiM) Raspberry Pi cluster (Pringles) tray
The idea for a cluster tray or Pi Tray was formed when I started out designing the Less-is-More (LiM) Raspberry Pi Compute Modle 4 (CM4) cluster carrier board.
All the cluster set ups for Raspberry Pis I found on the Internet have the boards lined up either side by side or one above the other. This type of alignemnt of the Raspeberry Pis requires venting and fans to curculate the heat away and out of the cases from the CPU’s.
| Horizontal Cluster | Vertical Cluster |
|---|---|
 |
 |
This is what made me think of aligning the Raspberry Pis in an offset pattern to allow for better airflow. I was tinking of using a Pringles box and aligning trays in a star shaped way.
The original idea and first model
The original idea was a 2D drawing and a model made from toilet paper rolls with bristol board wings for the trays for the Raspberry Pis:
 |
 |
First Model:
{% include carousel.html height=”75” unit=”%” duration=”7” number=”1” %}
The Pringles Prototype
For the next version of the Pi Tray prototype I replace the toilet paper rolls with a Pringles tube:
{% include carousel.html height=”75” unit=”%” duration=”7” number=”2” %}
First 3D Printed Prototype
As a next step I thought, maybe I could design and 3D print a modular tray that could allow for better air circulation. I found a guy on Fiverr who disigned the first prototype of the Pi Tray.
Second 3D Printed Prototype
The problem with the original prototype was that it was too big to be printed on a standard 3D printer surface. It had to be printed in two pieces and then glued together.
Another improvement for the next iteration of the Pi Tray was that it was only designed with mounting holes for the LiM cluster board. So another guy on Upwork helped me design the next iteration of the Pi Tray. This time I had it designed it to fit on a standard 3D printer surface (220mmx220mmx250mm) as well as it should have mounting options for not only the LiM cluster board but also a Raspberry Pi 4B.
| Pi Tray v2 drawing | Pi Tray v2 3 levels drawing |
|---|---|
 |
 |
Here are some pictures of the the Pi Tray v2 printed, with a LiM cluster board and Rasperry Pi 4B mounted:
{% include carousel.html height=”75” unit=”%” duration=”7” number=”3” %}
So, after I had two Pi Tray modules printed and started to put a cluster together I noticed I forgot to add notches in the bottom of the design to allow for the cables to come out of and allow the tray to be positioned evenly on a surface.
I did take a Dremel to one of the modules and added for notches to allow for the network cables to pass through. The final v3 of the Pi Tray will be updated to include these notches.
I will post pictured of the final v3 of the Pi Tray as well as the complete eight node cluster mounted on the modular trays when complete.