Kubernetes Installation. [updated]

by

For the HyperFlex Container Storage Interface (HX-CSI) you will need a Kubernetes cluster. Here is the start of an image that have the right software on it and which you can clone before configuring Kubernetes.

Start with a normal Ubuntu 18.04 installation.

I just want to have certain versions of software and that is why I disable the automatic updates.

During the installation, it’s nice that OpenSSH will be installed and configured so a SSH session to the image is possible.

After the installation, you can login with the user you’d created.

Default there is no root password, so when you configure it, you can easily use it. It’s not recommended, but sometimes it can be very handy.

sudo passwd root

Now Docker and other things can be installed.

sudo apt-get install docker.io -y
sudo apt-get install apt-transport-https -y

Now you can enable and start the Docker Service so that it will run even after a reboot.

sudo systemctl enable docker
sudo systemctl start docker

Verify if the installed Open-iSCSI is the correct version (2.0-874) for HX-CSI with the command :

iscsiadm -V

Install Kubernetes with the native package management

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"

Now we can install the correct Kubernetes version for HX-CSI. 

sudo  apt-get install -qy kubelet=1.18.0-00 kubectl=1.18.0-00 kubeadm=1.18.0-00

Make sure to hold it and verify the version.

sudo apt-mark hold kubeadm kubelet kubectl
kubeadm version

It’s always nice to have the VMtools running on the image. The easiest way to do this is :

sudo apt-get install open-vm-tools

You will need to disable the swap file. The command swapoff -a will do the trick, but after a reboot, it’s there again. The best thing to do is to disable it at the /etc/fstab file.

If you’re creating a “GoldenImage” you can now Shutdown the image:

sudo shutdown -P now

When it is powered off, clone it, so you will have a Master and Nodes with the correct software installed. This can save you some time.

Make sure every clone has a different Hostname, IP and iSCSI IQN.

In my case I will have 1 K8S Master and 3 Workers with static ip addresses.

Finalize the Kubernetes installation

On the Master type the following command:

sudo kubeadm init --pod-network-cidr=192.168.0.0/16

The command will generate a command with a token (Last line of the output) and some output and there next line should be done on the Master:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

On all the nodes install the Calico-System. This is the L3 Network.

kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml
kubectl create -f https://docs.projectcalico.org/manifests/custom-resources.yaml
watch kubectl get pods -n calico-system

[UPDATED]
When there is POD to POD traffic, make sure you configure IP in IP encapsulation for cross-traffic in Calico:

apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
  name: ippool-ipip-cross-subnet-1
spec:
  cidr: 192.168.0.0/16
  ipipMode: CrossSubnet
  natOutgoing: true

If you want to have IP in IP encapsulation for inter workload traffic, just change the ipipMode to Always.

Let have all the Workers join the cluster. Use the command seen at the last 2 lines of previous kubeadm init command. The token is each time different.

sudo kubeadm join 10.2.12.220:6443 --token ccbecb.3338kgzn79o6jtre --discovery-token-ca-cert-hash sha256:f0e4e421b82d679c0cb0edbc4d8e112009005785334941f7d391e1cedbeb81a1

To see if everything is running fine use the command :

kubectl get nodes -o wide
Kubectl get nodes

More Technical Posts:

Leave a Comment