Here we go again.

After installing ZFS on a Debian based Proxmox Node I now need some bang on a CentOS Server.

To remind you all: ZFS on Linux is considerably stable and matured, but you put it in place on your own risk.

ZFS Native comes from http://zfsonlinux.org/

I use a Centos 6.3 minimal installation and I have a 2GB disk configured to it for demo.

fdisk -l Disk
/dev/sdb: 2147 MB, 2147483648 bytes

Updated (26.04.2013)
And a Russian fellow describes his way to install it which I used and improved. http://habrahabr.ru/post/152853/

He build his own repo under http://yum.aclub.net/pub/linux/centos/6/umask-zfsonlinux/x86_64/ which is maintained as per 05.03.2013

So we start with:

cd /etc/yum.repos.d/
wget http://yum.aclub.net/pub/linux/centos/6/umask-zfsonlinux/umask-zfsonlinux.repo
rpm --import http://yum.aclub.net/pub/linux/centos/5/umask/RPM-GPG-KEY-umask

There is an original repo from zfs on Linux now which we are going to use since it has the latest version 0.6.1 of ZFS http://zfsonlinux.org/epel.html

yum localinstall --nogpgcheck 

If you don’t have yet enabled the EPEL repo please do now cause we need a pew packages from there. You can find details here or else: Obsulete since 08.2013 due to dkms in epel with a incompatible version

rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

We now install DKMS and all necessary packages for building the kernel modules:

yum update
yum install dkms gcc make kernel-devel perl

And now we come to the serious stuff … installing zfs ..

yum install spl zfs

That might take a while now depending on the machine performance since its compiling the packages.

You now better turn on ZFS in case you plan to reboot that server again.

chkconfig zfs on

That should do the jobs now. you can see it it works by running

# zpool status
no pools available

Now lets configure some ZFS. For a deeper howto please refer for example to http://www.datadisk.co.uk/html_docs/sun/sun_zfs_cs.htm

It is advised to use if possible a full disk and let it hand by zfs rather than a slice.

# zpool create -f zfs-data /dev/sdb

zpool status shows what we just configured

# zpool status
pool: zfs-data
state: ONLINE
scan: none requested
config:
NAME        STATE     READ WRITE CKSUM
zfs-data    ONLINE       0     0     0
sdb       ONLINE       0     0     0
# zfs list
NAME       USED  AVAIL  REFER  MOUNTPOINT
zfs-data   104K  1.95G    30K  /zfs-data

In general you could now already use /zfs-data as a filesystem.

We can create underneath another filesystem.

We need to check first the permissions of /zfs-data it sometimes is set wrongly.

chmod 755 /zfs-data
zfs create zfs-data/compressed

Lets quickly turn on compression.

zfs set compression=on zfs-data/compressed
zfs list
NAME USED AVAIL REFER MOUNTPOINT
zfa-data 141K 1.95G 30K /zfa-data
zfa-data/compressed 30K 1.95G 30K /zfa-data/compressed

ZFS mounts the filesystems automatically when booting the system.

Kernel upgrade

If you run yum update and the kernel gets updated then please follow the following steps to update zfs on Centos:

in some occasions after upgrading the kernel it might happen that zfs is not able to load zfs.ko and returns a

Load the module manually by running 'insmod /zfs.ko' as root.

Try the following to fix this. search first for your kernel version. (in this case you will see that I use a openvz kernel)

uname -a
Linux oVirt 2.6.32-042stab077.7 #1 SMP Mon Apr 22 14:59:15 MSK 2013 x86_64 x86_64 x86_64 GNU/Linux

make sure you have the kernel headers installed.

yum whatprovides kernel-headers

search for 042stab077.7

install missing headers with

 yum install vzkernel-devel-2.6.32-042stab076.5.i686

In this case I had to take an earlier version because there were no headers for 77 and then get with yum -y upgrade the latest version 078 for the kernel and its headers. Reboot to go to the latest version if this has changed

reboot

reinstall zfs

yum reinstall dkms spl zfs spl-modules-dkms zfs-modules-dkms

That should work now

  • Any prospects for CentOS root on zfs? (might require a separate physical boot disk; maybe a USB key boot?)

    • Andreas_Schmidt

      the Problem is that no OS as of now (except of solaris derivatives) does support the installation on zfs. I tried to work away with this https://github.com/zfsonlinux/pkg-zfs/wiki/HOWTO-install-Ubuntu-to-a-Native-ZFS-Root-Filesystem for Centos but so far I was only able to produce some nice crashes and no boot’s. My suspicion is that Grub which is on Ubuntu on a more modern version doesn’t play well.
      As soon I am able to get it going I will post it.

  • Zok-Star

    legend! thanks for this!!

  • Zok-Star

    Can you recommend any little tweaks to squeeze out a little more speed?

    • Andreas_Schmidt

      generic hints:

      bigger CPU, more memory, faster HD, better mainboard / IO controller.

      I’m sorry on this. I’ve tried it on an elderly test-system which is not state of the art and on a VM.
      I have tried various settings from http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide and run some speed copy test and didnt find any difference. Disabling compression and dedup does make a difference but the rest didn’t at all. At the end I suspect its the underlying hardware that limits.

      Comparably to ext it appears to be slower and I saw much higher HD operations when writing a file.

      Sorry I cant help much on this. Doe to its compression and dedup, I treat it currently more as an archive FS rather than a performant working FS. I store files, owncloud-data, openvz, etc in it and I have no big issue since performance is not my main concern.

  • adhitya cn

    halu…

    any information for sharing zfs space based on regular file with iscsi?

  • Guest

    I’ve got this error =(

    [root@chibi ~]# zpool status
    Failed to load ZFS module stack.
    Load the module manually by running ‘insmod /zfs.ko’ as root.

    • Andreas_Schmidt

      Your error message means that zfs is not running or is not proper installed. Is it a new install? or did an kernel upgrade break it? Do you have the kernel headers installed ?

      Try the above kernel upgrade steps

      and re-install spl and zfs and look if that works or if there are error messages.

      yum reinstall spl zfs

    • sine

      Try this if ur on a 64bit machine ,

      ln -s /usr/lib/dkms/ /usr/lib64/dkms/

  • You can also install exclusively without ignoring pgp check etc, by going here https://github.com/zfsonlinux/zfs/wiki/RHEL-%26-CentOS, then coming back to this article after and beginning at the `chkconfig zfs on` command. I found the linked page to be more accurate, and secure way to setup zfs, as of Jan 2017.

    • Haven’t used ZFS on CentOS for quite some while, but thanks for that.

%d bloggers like this: