There are narratives being built by cloud providers to help 5G telecom operators migrate CNF/VNF/virtualization platforms to public clouds. A number of telecom companies and system integrators have already mandated their staff to learn cloud computing. CNF/VNF are in fact written in favor of the public cloud.
In spite of this, I see numerous challenges, which I cite
- Connecting transport network providers (X-haul, fiber, packet core), integration of heterogeneous environments ORAN/RAN termination for data centers and mobile edge computing
- Availability and footprint of public cloud is limited outside the United States.
- Change management in environment where administrative controls are split
Metal-LB provides load-balancing functionality for local or bare metal Kubernetes clusters that are not deployed in public cloud environments such as AWS, Azure, or Google Cloud Platform.
Recently, I worked on Affirmed networks, private 5G cloud running CNF, and this prompted me to write this blog on how to deploy Metal LB on baremetal Kubernetes clusters and demonstrate all public cloud features are available in private cloud.
I have 3 worker nodes and 1 master node in my setup.
I have uploaded the manifest files to Github, feel free to use them in your environment. The link is mentioned below.
https://github.com/ranjeetbadhe/metallb.git
Here are my manifest files. All you need is to apply these manifest files and your LB is ready.
[root@kubemaster metal]# ll total 96 -rw-r--r-- 1 root root 1190 Oct 4 10:06 commands.txt -rw-r--r-- 1 root root 164 Oct 4 10:19 l2advertisement.yml -rw-r--r-- 1 root root 76201 Oct 3 11:56 metallb-native.yaml -rw-r--r-- 1 root root 159 Oct 4 10:11 pool-1.yml -rw-r--r-- 1 root root 885 Oct 4 10:15 web-app-deployment.yml -rw-r--r-- 1 root root 353 Oct 4 10:17 web-app-ingress.yml
[root@kubemaster metal]# cat l2advertisement.yml apiVersion: metallb.io/v1beta1 kind: L2Advertisement metadata: name: ranjeetbadhe.com-home-lab namespace: metallb-system spec: ipAddressPools: - first-pool
[root@kubemaster metal]# cat pool-1.yml apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: first-pool namespace: metallb-system spec: addresses: - 192.168.0.240-192.168.0.250 [root@kubemaster metal]# cat web-app-deployment.yml apiVersion: apps/v1 kind: Deployment metadata: name: web-app labels: app.kubernetes.io/name: web-app name: web-app spec: replicas: 3 selector: matchLabels: app.kubernetes.io/name: web-app template: metadata: labels: app.kubernetes.io/name: web-app spec: containers: - image: nginx name: web-app command: - /bin/sh - -c - "echo 'Ranjeet Badhe welcome to my web app!' > /usr/share/nginx/html/index.html && nginx -g 'daemon off;'" dnsConfig: options: - name: ndots value: "2" --- apiVersion: v1 kind: Service metadata: name: web-app labels: app.kubernetes.io/name: web-app spec: selector: app.kubernetes.io/name: web-app ports: - name: http port: 80 protocol: TCP targetPort: 80 type: LoadBalancer
[root@kubemaster metal]# kubectl apply -f metallb-native.yaml namespace/metallb-system created customresourcedefinition.apiextensions.k8s.io/addresspools.metallb.io created customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created serviceaccount/controller created serviceaccount/speaker created role.rbac.authorization.k8s.io/controller created role.rbac.authorization.k8s.io/pod-lister created clusterrole.rbac.authorization.k8s.io/metallb-system:controller configured clusterrole.rbac.authorization.k8s.io/metallb-system:speaker configured rolebinding.rbac.authorization.k8s.io/controller created rolebinding.rbac.authorization.k8s.io/pod-lister created clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller unchanged clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker unchanged configmap/metallb-excludel2 created secret/webhook-server-cert created service/webhook-service created deployment.apps/controller created daemonset.apps/speaker created validatingwebhookconfiguration.admissionregistration.k8s.io/metallb-webhook-configuration created
[root@kubemaster metal]# kubectl get ns NAME STATUS AGE default Active 216d kube-flannel Active 216d kube-node-lease Active 216d kube-public Active 216d kube-system Active 216d kubevirt Active 62d metallb-system Active 2m35s monitoring Active 208d vms Active 61d
[root@kubemaster metal]# kubectl get all -n metallb-system Warning: kubevirt.io/v1 VirtualMachineInstancePresets is now deprecated and will be removed in v2. NAME READY STATUS RESTARTS AGE pod/controller-995466675-wfdvs 1/1 Running 1 (110s ago) 2m50s pod/speaker-f85wg 1/1 Running 0 2m50s pod/speaker-ggvbl 1/1 Running 0 2m50s pod/speaker-gz4mm 1/1 Running 0 2m50s pod/speaker-kc59b 1/1 Running 0 2m50s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/webhook-service ClusterIP 10.96.212.184 <none> 443/TCP 2m50s NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/speaker 4 4 4 4 4 kubernetes.io/os=linux 2m50s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/controller 1/1 1 1 2m50s NAME DESIRED CURRENT READY AGE replicaset.apps/controller-995466675 1 1 1 2m50s [root@kubemaster metal]# kubectl get all -n metallb-system Warning: kubevirt.io/v1 VirtualMachineInstancePresets is now deprecated and will be removed in v2. NAME READY STATUS RESTARTS AGE pod/controller-995466675-wfdvs 1/1 Running 1 (3m33s ago) 4m33s pod/speaker-f85wg 1/1 Running 0 4m33s pod/speaker-ggvbl 1/1 Running 0 4m33s pod/speaker-gz4mm 1/1 Running 0 4m33s pod/speaker-kc59b 1/1 Running 0 4m33s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/webhook-service ClusterIP 10.96.212.184 <none> 443/TCP 4m33s NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/speaker 4 4 4 4 4 kubernetes.io/os=linux 4m33s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/controller 1/1 1 1 4m33s NAME DESIRED CURRENT READY AGE replicaset.apps/controller-995466675 1 1 1 4m33s
root@kubemaster metal]# kubectl get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 217d load-balancer-service LoadBalancer 10.98.76.222 192.168.0.242 80:31321/TCP 216d nginx-deployment LoadBalancer 10.111.226.2 192.168.0.244 80:30371/TCP 216d [root@kubemaster metal]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 217d load-balancer-service LoadBalancer 10.98.76.222 192.168.0.242 80:31321/TCP 216d nginx-deployment LoadBalancer 10.111.226.2 192.168.0.244 80:30371/TCP 216d
[root@kubemaster metal]# kubectl get svc nginx-deployment NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx-deployment LoadBalancer 10.111.226.2 192.168.0.244 80:30371/TCP 216d [root@kubemaster metal]# curl http://192.168.0.245 Ranjeet Badhe welcome to my web app!
root@kubemaster ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION kubemaster.ranjeetbadhe.com Ready control-plane,master 216d v1.23.2 node2.ranjeetbadhe.com Ready <none> 216d v1.23.2 node3.ranjeetbadhe.com Ready <none> 216d v1.23.2 node4.ranjeetbadhe.com Ready <none> 216d v1.23.2
root@kubemaster ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION kubemaster.ranjeetbadhe.com Ready control-plane,master 216d v1.23.2 node2.ranjeetbadhe.com Ready <none> 216d v1.23.2 node3.ranjeetbadhe.com Ready <none> 216d v1.23.2 node4.ranjeetbadhe.com Ready <none> 216d v1.23.2