Debian / Ubuntu / Centos / Zfs on Linux: ZFS Limit Arc Cache

Talking about ZFS and ARC CACHE

Generally ZFS is designed for servers and as such its default settings are to allocate:
– 75% of memory on systems with less than 4 GB of memory
- physmem minus 1 GB on systems with greater than 4 GB of memory
(Info is from Oracle but I expect the same values for ZFS native on Linux)

That might be too much if you intent to run anything else like virtualisation or applications on the server and while ZFS returns cached memory when a memory intensive application asks for it, there might be a delay to do so causing some waits. We want to limit the memory ZFS can allocate to give some air to breath for the applications.

However this won’t make any significant performance improvement to the filesystem itself. It’s just to free memory for other application which might have started to swap due to insufficient free memory.

So how do we check what is currently used.

# cat /proc/spl/kstat/zfs/arcstats |grep c_
c_min 4 318767104
c_max 4 2550136832
arc_no_grow 4 0
arc_tempreserve 4 0
arc_loaned_bytes 4 0
arc_prune 4 0
arc_meta_used 4 0
arc_meta_limit 4 637534208
arc_meta_max 4 0

The values c_min and c_max are in bytes and you can use the Bit calculator to translate into something more readable.

I’m now going to make some new settings by adding a new file /etc/modprobe.d/zfs.conf and put values into.

vi /etc/modprobe.d/zfs.conf

# Min 512MB / Max 2048 MB Limit
options zfs zfs_arc_min=536870912
options zfs zfs_arc_max=2147483648

(it’s a low memory 4GB only system )

Now I reboot the system and check the new values after the change.

cat /proc/spl/kstat/zfs/arcstats |grep c_

c_min 4 536870912
c_max 4 2147483648
arc_no_grow 4 0
arc_tempreserve 4 0
arc_loaned_bytes 4 0
arc_prune 4 0
arc_meta_used 4 0
arc_meta_limit 4 536870912
arc_meta_max 4 0

cross check

# cat /sys/module/zfs/parameters/zfs_arc_min
536870912
# cat /sys/module/zfs/parameters/zfs_arc_max
2147483648

ZFS Arc really usefull know how to read up

http://www.c0t0d0s0.org/archives/5329-Some-insight-into-the-read-cache-of-ZFS-or-The-ARC.html