Hyper-converged systems are hot, but what is it ?

In short : It’s a server where you run VMWare vsphere and you’re using local storage for your NAS/SAN with some better functions than the standard local datastores of VMWare got.

Is it possible to create one for home/SMB purpose ? Well.. I saw the page FreeNas on VMWare ESXi 6.0 and I thought ; Let’s give it a try.

What do you need :

• a x86 server with lots of memory (for deduplication)
• Some harddisks.
• Bootable USD/SDCard
• Two raid controllers ! One for the disks you want to put in FreeNas. One other harddisk (non-usb or sdcard) for a local datastore for the FreeNas image.
• VMWare vsphere software
• FreeNas software.

Howto start :

1 : First prepare yourserver. Configure only your local datastore drive. All other drives you don’t have to configure your Raidcontroller card. FreeNas will take care of the raid!

Now I got a problem ! I got a raid controller for the local datastore drive and a cable, but no power for it.. How I solved this ? I got some cables and was able to pull out the power of the SAS interface. 🙂

2 : Install VMWare vsphere on an USB/SDCard.

3 : At the configuration/advanced configure the raid controller for pass through. Of course you’ll have to reboot the ESXi server after this configuration.

 

4 : Create a new vmware machine and add the PCI device of the raid controller. You will have to install this  image on the local datastore.

Now just mount FreeNas ISO at the vmware dvd drive an install FreeNas.

FreeNas :

With the command lspci at the FreeNas shell you can see the type of your raidcontroller. The easiest part is to have a controller which is supported by FreeNas, otherwise you will have to install drivers for the raidcontroller.

When everything is setup correct, you can see the drives at the FreeNas GUI.

5 : Create a volume and right now I did a NFS share. iSCSI is also possible.

6 : It’s always a good idea to have your MTU setup at 9000. No only at your ESXi server, but on every devices in your network.

7 : Mount the NFS in VMWare.

Right now you’re capable to install images on the NFS share on your local host or maybe on another server.

So, now you got a Hyper-Converged system with FreeNas !

What about Dedup and Compression ?

Although diskspace is not expensive anymore, you want to use it totally and preferable put more data on it then the size of the disks. You can do this by deduplication and compression.

FreeNas got inline lz4 compression and it’s working pretty well. There are some HCI systems with a dedicated asic for compression and compared that system with my system, my CPU is consuming 500 Mhz when it try to compress a file. This is not a lot, because I am working with an Intel E5 v1 processor and the other system got a E5 v3 CPU.

It’s possible to verify the dedup status of your volume with the command :

zdb -S <Volume Name>

Right now my dedup ratio isn’t high, but I didn’t have a copy of the image or a backup. Dedup will then hit it and the number will raise.

RSync

When you got more then one FreeNas it’s possible to sync volumes etc between the FreeNas devices. Very nice feature !

IOMeter Results :

Is my system fast ? Well.. Maybe not really as fast as enterprise HCI, but it’s very fast for a home HCI.

Here are results of  iometer with 100% 4K read. With 75% 4K read the system will have 400 iops. Still nice, because I don’t have a lot of fast drives for testing.

To test the system I create a random (almost non compressible) file on the CentOS Server with the command :

dd if=/dev/urandom bs=1G count=10 of=a.log

This generates a random file a.log of 10 Gb in size. It took me 628 sec to create the file. (About 18 MB/s)

Have fun building your own Hyper-Converged System. For questions just ping me at : info@joostvandermade.com

This is not a step-by-step hyper-converged installation/configuration guide, but the main things you will have to do for this system. Just google some things and you’ll find your answers.

Everything is my own and I am not sponsored by any vendor.