Bonsai Kube: k3s on multipass VMs on your local machine

Latest version: Kubernetes 1.25.10 (v1.25.10+k3s1)

Introduction

This repo provides a lightweight multi-node k3s deployment on multipass VMs on your local machine.

It deploys mainly k3s with MetalLB and the upstream kubernetes nginx ingress controller and provides couple of useful addons for developers and ops folks.

About k3s

k3s is a nice tool for development, fun and profit, with k3s you can spin up a lightweight k8s cluster from scratch in less than 3 minutes.

k3s is packed into a single binary, which already includes all you need to quickly spin up a k8s cluster.

To learn more about k3s, please visit the k3s github repo and the official documentation.

Prerequisites

Install multipass (on MacOS or Linux)

You need Multipass running on your local machine, to learn more about Multipass, please visit:

https://github.com/CanonicalLtd/multipass

https://multipass.run/

This setup was tested on MacOS, but should work on Linux or Windows too.

You need to have about 4GB free RAM and 16GB free storage on your local machine, but it should work with less resources.

You need sudo rights on your machine.

You need kubectl in your path, if not, you can download the v1.17.4 version and put it in your path:

MacOS users:

curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.24.9/bin/darwin/amd64/kubectl

Linux users:

curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.24.9/bin/linux/amd64/kubectl

chmod +x ./kubectl
mv kubectl /usr/local/bin/

Important hint for MacOS Intel and Linux users

MacOS Intel and Linux users should adapt the utlis/create-hosts.sh and adapt the network interface name enp0s1 (which is apparently the default nic name of multipass VMs on M2 Silicon MacOS). You can find the nic name with:

multipass launch --name test
multipass exec test -- bash -c 'echo `ls /sys/class/net | grep en`'

Delete and purge the test VM:

multipass delete test
multipass purge

If the above doesn't work somehow, shell into the node and get the nic name:

multipass shell test
ifconfig

k3s Deployment (step 1)

Clone this repo and deploy a 3-node deployment with a single command:

git clone https://github.com/kubernauts/bonsai
cd bonsai
./deploy-bonsai.sh

You'll be asked to provide the desired number of worker nodes, cpu and memory, the default ones shoul be fine.

After the deplyoment the output provides the total runtime similar to this:

############################################################################

Total runtime in minutes: 03:05

############################################################################

Clean Up

./cleanup.sh
./cleanup-rkes.sh

Troubleshooting

k3s uses containerd as CRI (container runtime interface). If you'd like to see the status of the containers on the nodes for e.g. troubleshooting or fun, you can run:

multipass exec k3s-worker -- /bin/bash -c "sudo crictl ps -a"

or shell into the node and run sudo crictl ps -a:

multipass shell k3s-worker1
sudo crictl ps -a

Gotchas

Running ./cleanup.sh throws an error like:

Stopping requested instances -[2019-09-14T14:49:51.096] [error] [rke1] process error occurred Crashed

which can be ignored.

Credits

Thanks to the awesome folks at Rancher Labs for making k3s the first choice for a lightweight Kubernetes solution.

And thanks to Mattia Peri for his great post on medium, which encouraged me to automate everything with this small implementation for k3s on multipass.