Development with K8s locally
Different ways to run “lightweight” K8s locally, and a basic measurement of how resource heavy it is.
Kind
kind creates local multi-node Kubernetes clusters using Docker container nodes.
https://github.com/kubernetes-sigs/kind
brew install kind
kind create cluster
kind delete cluster
minikube
https://minikube.sigs.k8s.io/docs/
brew install minikube
minikube start
minikube pause
minikube delete
k3d (k3s)
krd is k3s in docker. k3s is the lightweight Kubernetes distribution by Rancher: rancher/k3s.
https://github.com/rancher/k3d
brew install k3d
k3d cluster create test
k3d cluster delete
Resource usage
Resource usage when using docker and K8s.
Docker settings: CPU 4, mem 4 GB, swap 1 GB
| image + (pod) | CPU % | Memory GB | |
|---|---|---|---|
| docker | - | 4 | 4,7 - 6.8 |
| * | redis | 7 | * |
| * | redis+mongo | 9 | * |
| docker-desktop with K8s | ca 20 k8s_XXX + (1 redis) | 37 | * |
| * | ca 20 k8s_XXX + (10 redis) | 42 | * |
| * | ca 20 k8s_XXX + (100 redis) | 71 | * |
| docker with minikube | gcr.io/k8s-minikube + (1 redis) | 50 (80 after one hour) | * |
| * | gcr.io/k8s-minikube + (10 redis) | 52 | * |
| * | gcr.io/k8s-minikube + (100 redis) | 90 (140 after one hour) | * |
| * | gcr.io/k8s-minikube + (wordsmith) | 100 | * |
| docker with kind | kindest/node + (1 redis) | 55 | * |
| * | kindest/node + (10 redis) | 61 | * |
| * | kindest/node + (100 redis) | 90 | * |
| docker with k3d/k3s | rancher/krd-proxy, rancher/krs + (1 redis) | 36 | * |
| * | rancher/krd-proxy, rancher/krs + (10 redis) | 46 | * |
| * | rancher/krd-proxy, rancher/krs + (100 redis) | 75 | * |
| * | rancher/krd-proxy, rancher/krs + (wordsmith) | 43 | * |
Adding Tekton inside one of these K8s clusters adds about 25% constant CPU usage.
Which one is the best?
- K8s in Docker-Desktop or K3s are the least resource heavy.