K8s Homelab High Avalibility with MetalLB

Preface

In this article I am not going to be going through all the ports you need to configure on your firewalls, for ease of following this guide I recommend disabling the firewall. That being said, you should never do that for production! I want to focus on the k8s stuff here, so decided to skip this for the sake of brevity.

Flannel Setup

First we are going to set up the network stack. There are lots of different choices, but I went with Flannel due to the ease of setup. You just need to apply the file from the URL. If you want to change the default subnet from 10.244. 0.0/16 you will need to download the file first and edit it. That being said, for most people that shouldn’t be required.

kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

We then want to check if the flannel pods are running. You should see this:

kubectl get pods -A

If all went well, we can move onto the next part.

MetalLB

MetalLB is a locally hostable firewall. When you use cloud services, they often have their own offerings, but if you are running in a home lab, this is a good option.

We need to edit the config map first. Run this command, and you will be dropped into a text editor:

kubectl edit configmap -n kube-system kube-proxy

You are looking for a like with ipvs. Change the option strictARP to true.

apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"
ipvs:
  strictARP: true

We can now install MetalLB by running this command:

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.15.2/config/manifests/metallb-native.yaml

We will then want to check it by running:

kubectl get pods -A

Check that all the metallb-system are showing as running.

We will now want to create a file called ipaddresses.yaml with this as the contents:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool
  namespace: metallb-system
spec:
  addresses:
  - X.X.X.X-X.X.X.X
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: example
  namespace: metallb-system

You will want to replace the X.X.X.X to an IP that won’t be used by another device on your network. I recommend setting up a reserved range that DHCP can’t use. You will have to figure that stuff out by your self, as it is different from device to device. Also, I only allocated a range of 5 IPs, but we also will only be using on IP for this tutorial. We then want to apply it by running:

kubectl apply -f ipaddresses.yaml

Nginx Service

Now we want to create a file called nginx.yaml. This is just a container that runs the latest version of Nginx.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        imagePullPolicy: Always
        ports:
          - containerPort: 80

We are also going to need a service, so create a file called nginx_service.yaml.

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  labels:
    app: nginx-service
spec:
  type: LoadBalancer
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  sessionAffinity: None

We can now run::

kubectl apply -f nginx.yaml
kubectl apply -f nginx_service.yaml

Now when we run kubectl get services -A we should see: And if we navigate to the EXTERNAL-IP listed, we should see:

You now have a supper simple load balanced website. While it isn’t the best setup, it is a starting point for your home lab.

If you are interested on how to deploy a K8s stack with Terraform and Ansible, you might want to have a look at my previous blog article HERE