Specialized applications can need to read/write directly to a block device. The main use case are databases. A filesystem is more “human-readable” and it adds extra overhead. Instead of the filesystem, block devices are chosen more often. More information about Kubernetes can be found HERE.
Storage Class YAML File
I am using the default Storage Class of the HX-CSI example directory. Nothing fancy.
kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: csi-hxcsi-default provisioner: csi-hxcsi parameters: datastore: default-ds datastoreSize: "-1"
Persistent Volume Claim YAML File.
After creating the Storage Class, create a PVC as a Block Device with ReadWriteMany because different containers wants to read and write to the same shared storage.
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: demo-block-rwx spec: storageClassName: csi-hxcsi-default volumeMode: Block accessModes: - ReadWriteMany resources: requests: storage: 10Gi
Deployement YAML File
In this example I’m using the Ubuntu container image. Because it’s a block device, devicePath is now used instead of volumeMount.
apiVersion: apps/v1 kind: Deployment metadata: name: demo-block labels: app: demo-block spec: replicas: 1 selector: matchLabels: app: demo-block template: metadata: labels: app: demo-block name: demo-block spec: volumes: - name: blockvolume persistentVolumeClaim: claimName: demo-block-rwx containers: - name: demo-block image: ubuntu command: [ "/bin/bash", "-c", "--" ] args: [ "while true; do sleep 30; done;" ] ports: - containerPort: 22 volumeDevices: - devicePath: /dev/block name: blockvolume
When the SC, PVC and the container are deployed, verification if everything is working can be done.
To verify if the block device connected in the container:
kubectl exec demo-block-757cfdc75c-59dbz – bash -c "ls -l /dev/"
total 0 brw-rw-- – 1 root disk 8, 32 Mar 11 19:48 block lrwxrwxrwx 1 root root 11 Mar 11 19:21 core -> /proc/kcore lrwxrwxrwx 1 root root 13 Mar 11 19:21 fd -> /proc/self/fd crw-rw-rw- 1 root root 1, 7 Mar 11 19:21 full drwxrwxrwt 2 root root 40 Mar 11 19:21 mqueue crw-rw-rw- 1 root root 1, 3 Mar 11 19:21 null lrwxrwxrwx 1 root root 8 Mar 11 19:21 ptmx -> pts/ptmx drwxr-xr-x 2 root root 0 Mar 11 19:21 pts crw-rw-rw- 1 root root 1, 8 Mar 11 19:21 random drwxrwxrwt 2 root root 40 Mar 11 19:21 shm lrwxrwxrwx 1 root root 15 Mar 11 19:21 stderr -> /proc/self/fd/2 lrwxrwxrwx 1 root root 15 Mar 11 19:21 stdin -> /proc/self/fd/0 lrwxrwxrwx 1 root root 15 Mar 11 19:21 stdout -> /proc/self/fd/1 -rw-rw-rw- 1 root root 0 Mar 11 19:21 termination-log crw-rw-rw- 1 root root 5, 0 Mar 11 19:21 tty crw-rw-rw- 1 root root 1, 9 Mar 11 19:21 urandom crw-rw-rw- 1 root root 1, 5 Mar 11 19:21 zero
There is a /dev/block and you can see it’s a block device, because the first letter on the line is a “b”
Let’s write some data on the block device:
kubectl exec demo-block-757cfdc75c-59dbz – bash -c "echo "I_Am_Joost_._com" | dd conv=unblock of=/dev/block"
Verify what is written on the storage device:
kubectl exec demo-block-757cfdc75c-59dbz – bash -c "od -An -c -N 20 /dev/block"
jvdmade@K8S-M:~$ kubectl exec demo-block-757cfdc75c-59dbz – bash -c "od -An -c -N 20 /dev/block" I _ A m _ J o o s t _ . _ c o m
It is possible to create some Replica’s, but I want to have a complete “new” server connect to this RWX Block device, so I’ve created another deployment. Basically it’s the same YAML file, but with different names.
Here you see the Server2-demo pod is running:
jvdmade@K8S-M:~/block$ kubectl get pods
NAME READY STATUS RESTARTS AGE
csi-attacher-hxcsi-0 2/2 Running 0 3d21h
csi-nodeplugin-hxcsi-2w4cg 2/2 Running 2 3d21h
csi-nodeplugin-hxcsi-h2pfz 2/2 Running 2 3d21h
csi-nodeplugin-hxcsi-vcbjq 2/2 Running 2 3d21h
csi-provisioner-hxcsi-0 2/2 Running 0 3d21h
csi-resizer-hxcsi-7979549b8f-wh58n 2/2 Running 0 3d21h
demo-block-757cfdc75c-59dbz 1/1 Running 0 2d23h
server2-block-fcd48df5-4tctb 1/1 Running 0 15s
test-647b4c8874-l5g5d 1/1 Running 0 3d20h
To see if the block device is really has the same content of the other container:
kubectl exec server2-block-fcd48df5-4tctb – bash -c "od -An -c -N 16 /dev/block"
And the result is what the other container wrote to the device.
I _ A m _ J o o s t _ . _ c o m
Via Server2 we’re going to overwrite this data:
kubectl exec server2-block-fcd48df5-4tctb – bash -c "echo "This_Is_Server_2_Writing_Something_To_The_Block_Device" | dd conv=unblock of=/dev/block"
Lets see if the content can be read from the other server with the command:
kubectl exec demo-block-757cfdc75c-59dbz – bash -c "od -An -c -N 75 /dev/block"
And here is the result:
T h i s _ I s _ S e r v e r _ 2 _ W r i t i n g _ S o m e t h i n g _ T o _ T h e _ B l o c k _ D e v i c e \n \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
Hooray! It’s working.
Here are other HyperFlex / Technical Posts:
- HX-CSI ReadWriteMany Block Device ExampleSpecialized applications can need to read/write directly to a block device. The main use case are databases. A filesystem is more “human-readable” and it adds extra overhead. Instead of the filesystem, block devices are chosen more often. More information about Kubernetes can be found HERE. After the installation of Kubernetes and the HyperFlex Container Storage
- HyperFlex 4.5 Container Storage Interface (HX-CSI) InstallationIn a containerized environment, you need persistent volumes . With HyperFlex 4.5, iSCSI thisis possible. What are the difference between using iSCSI or CSI in a container. Information about Kubernetes Persistent Volumes can be found HERE. iSCSI can only do: ReadWriteOnce and ReadOnlyMany. HX-CSI can do both AND also ReadWriteMany. (Block) There is a very
- Kubernetes Installation.For the HyperFlex Container Storage Interface (HX-CSI) you will need a Kubernetes cluster. Here is the start of an image that have the right software on it and which you can clone before configuring Kubernetes. Start with a normal Ubuntu 18.04 installation. I just want to have certain versions of software and that is why
- Cutting MP4 Into smaller Files.To make some awesome videos, I’m using my GoPro Hero8 with 2K7 (2704×1520) resolution. This is because I’m using a green screen and although the end videos are in FullHD, with this resolution I can easily crop, resize parts of the video without losing any quality. Cutting MP4 into smaller files would make life easier.
- Restricted Shell in HyperFlex 4.5It was already impossible to login with root in HyperFlex Connect, but you still could login with SSH or locally with a root account. In HXDP 4.5 there is the Restricted Shell or Admin Shell. What is the Restricted Shell? In the restricted shell there is a limited number of commands that you can perform.