RedHat 6.x / CentOS 6.x: How to Install ZFS native

Here we go again.
After installing ZFS on a Debian based Proxmox Node I now need some bang on a CentOS Server.
To remind you all: ZFS on Linux is considerably stable and matured, but you put it in place on your own risk.
ZFS Native comes from http://zfsonlinux.org/
I use a Centos 6.3 minimal installation and I have a 2GB disk configured to it for demo.
fdisk -l Disk /dev/sdb: 2147 MB, 2147483648 bytes
Updated (26.04.2013)
And a Russian fellow describes his way to install it which I used and improved. http://habrahabr.ru/post/152853/
He build his own repo under http://yum.aclub.net/pub/linux/centos/6/umask-zfsonlinux/x86_64/ which is maintained as per 05.03.2013
So we start with:
cd /etc/yum.repos.d/ wget http://yum.aclub.net/pub/linux/centos/6/umask-zfsonlinux/umask-zfsonlinux.repo rpm --import http://yum.aclub.net/pub/linux/centos/5/umask/RPM-GPG-KEY-umask
There is an original repo from zfs on Linux now which we are going to use since it has the latest version 0.6.1 of ZFS http://zfsonlinux.org/epel.html
yum localinstall --nogpgcheck
If you don’t have yet enabled the EPEL repo please do now cause we need a pew packages from there. You can find details here or else: Obsulete since 08.2013 due to dkms in epel with a incompatible version
rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
We now install DKMS and all necessary packages for building the kernel modules:
yum update yum install dkms gcc make kernel-devel perl
And now we come to the serious stuff … installing zfs ..
yum install spl zfs
That might take a while now depending on the machine performance since its compiling the packages.
You now better turn on ZFS in case you plan to reboot that server again.
chkconfig zfs on
That should do the jobs now. you can see it it works by running
# zpool status no pools available
Now lets configure some ZFS. For a deeper howto please refer for example to http://www.datadisk.co.uk/html_docs/sun/sun_zfs_cs.htm
It is advised to use if possible a full disk and let it hand by zfs rather than a slice.
# zpool create -f zfs-data /dev/sdb
zpool status shows what we just configured
# zpool status pool: zfs-data state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zfs-data ONLINE 0 0 0 sdb ONLINE 0 0 0 # zfs list NAME USED AVAIL REFER MOUNTPOINT zfs-data 104K 1.95G 30K /zfs-data
In general you could now already use /zfs-data as a filesystem.
We can create underneath another filesystem.
We need to check first the permissions of /zfs-data it sometimes is set wrongly.
chmod 755 /zfs-data zfs create zfs-data/compressed
Lets quickly turn on compression.
zfs set compression=on zfs-data/compressed
zfs list
NAME USED AVAIL REFER MOUNTPOINT
zfa-data 141K 1.95G 30K /zfa-data
zfa-data/compressed 30K 1.95G 30K /zfa-data/compressed
ZFS mounts the filesystems automatically when booting the system.
Kernel upgrade
If you run yum update and the kernel gets updated then please follow the following steps to update zfs on Centos:
in some occasions after upgrading the kernel it might happen that zfs is not able to load zfs.ko and returns a
Load the module manually by running 'insmod /zfs.ko' as root.
Try the following to fix this. search first for your kernel version. (in this case you will see that I use a openvz kernel)
uname -a Linux oVirt 2.6.32-042stab077.7 #1 SMP Mon Apr 22 14:59:15 MSK 2013 x86_64 x86_64 x86_64 GNU/Linux
make sure you have the kernel headers installed.
yum whatprovides kernel-headers
search for 042stab077.7
install missing headers with
yum install vzkernel-devel-2.6.32-042stab076.5.i686
In this case I had to take an earlier version because there were no headers for 77 and then get with yum -y upgrade the latest version 078 for the kernel and its headers. Reboot to go to the latest version if this has changed
reboot
reinstall zfs
yum reinstall dkms spl zfs spl-modules-dkms zfs-modules-dkms
That should work now
Any prospects for CentOS root on zfs? (might require a separate physical boot disk; maybe a USB key boot?)
the Problem is that no OS as of now (except of solaris derivatives) does support the installation on zfs. I tried to work away with this https://github.com/zfsonlinux/pkg-zfs/wiki/HOWTO-install-Ubuntu-to-a-Native-ZFS-Root-Filesystem for Centos but so far I was only able to produce some nice crashes and no boot’s. My suspicion is that Grub which is on Ubuntu on a more modern version doesn’t play well.
As soon I am able to get it going I will post it.
legend! thanks for this!!
Can you recommend any little tweaks to squeeze out a little more speed?
generic hints:
bigger CPU, more memory, faster HD, better mainboard / IO controller.
I’m sorry on this. I’ve tried it on an elderly test-system which is not state of the art and on a VM.
I have tried various settings from http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide and run some speed copy test and didnt find any difference. Disabling compression and dedup does make a difference but the rest didn’t at all. At the end I suspect its the underlying hardware that limits.
Comparably to ext it appears to be slower and I saw much higher HD operations when writing a file.
Sorry I cant help much on this. Doe to its compression and dedup, I treat it currently more as an archive FS rather than a performant working FS. I store files, owncloud-data, openvz, etc in it and I have no big issue since performance is not my main concern.
halu…
any information for sharing zfs space based on regular file with iscsi?
not out of the box and never done it before.
I would try a combination of http://www.cyberciti.biz/tips/rhel-centos-fedora-linux-iscsi-howto.html
and put the zfs to legacy and mount it in the fstab manually with the _netdev option. that might work
I’ve got this error =(
[root@chibi ~]# zpool status
Failed to load ZFS module stack.
Load the module manually by running ‘insmod /zfs.ko’ as root.
Your error message means that zfs is not running or is not proper installed. Is it a new install? or did an kernel upgrade break it? Do you have the kernel headers installed ?
Try the above kernel upgrade steps
and re-install spl and zfs and look if that works or if there are error messages.
yum reinstall spl zfs
This works for me on CentOS 7:
https://github.com/zfsonlinux/zfs/issues/1155#issuecomment-11499841
Try this if ur on a 64bit machine ,
ln -s /usr/lib/dkms/ /usr/lib64/dkms/
You can also install exclusively without ignoring pgp check etc, by going here https://github.com/zfsonlinux/zfs/wiki/RHEL-%26-CentOS, then coming back to this article after and beginning at the `chkconfig zfs on` command. I found the linked page to be more accurate, and secure way to setup zfs, as of Jan 2017.
Haven’t used ZFS on CentOS for quite some while, but thanks for that.