Tag Archives: deduplication

Home »  Tag: deduplication

It is widely know that ZFS can compress and deduplicate. The deduplication works across the pool level and removes duplicate data blocks as they are written to disk. This results into having only unique blocks stored on the disk while the duplicate blocks are shared among the files. There is a good read about how dedupe works and some tweaking things like changing the checksum hashing function. https://blogs.oracle.com/bonwick/entry/zfs_dedup Note: Compression works fine under zfsonlinux but the current version is not yet supporting deduplication (16.09.2014). ZFS on FreeBSD (for example FreeNAS) and Solaris (and Opensolaris) have a higher pool version and support deduplication. Deduplication has been introduced with pool version 31. Zpool versions and features (blogs.oracle.com) List of operating systems supporting ZFS (wikipedia) Now, how to determine if you would actually benefit from deduplicated and compressed datasets? I run the following under FreeNAS with a testsetup filled with real data. (self recorded camera .mov, ISOs, virtual […]

In the wake of the current Truecrypt FUD. It seems not too widely known that you can encrypt your data with zfs for quite some while. And it also works along with compression and deduplication. However this applies only to ZFS zpool version 30 onwards (introduced with Solaris 11) while zfs on Linux currently still runs on zpool version 28. So its not running there. To read a bit in detail on how it works you will find here a few interesting posts. How to Manage ZFS Data Encryption Introducing ZFS Crypto in Oracle Solaris 11 Express Having my secured cake and Cloning it too (aka Encryption + Dedup with ZFS) The encryption options are: aes-128-ccm (default) aes-192-ccm aes-256-ccm aes-128-gcm aes-192-gcm aes-256-gcm Only CCM supports encryption along with compression and deduplication so I ditch the GCM and go for (putting my weak half torn tinfoil hat on) aes-256-ccm. I’ll create a new Filesystem […]

%d bloggers like this: