an update to my zfs dedupe test.
opensolaris-dev build 129 (first build to enable dedupe)
I create a zfs volume that I export with iSCSI, then did a gigabit crossover
cable to my test backuppc server running ubuntu 9.04. I mount the iscsi
volume to /var/lib/backuppc.
let just say that this has to be seen to be believed. zfs dedupe totally
pegs my cpu and used up the 4GB of ram in my test "SAN" box BUT it
drastically improves write performance for files with the same blocks as
other files(which is a lot). The different between a ubuntu 8.04 and 9.04
disk is only about 30MB with block level dedupe. Because the system does
this dedupe in-stream, it doesnt have to write the 'extra' data and can
complete the process in a hurry. I can copy a disk image of ubuntu 9.04 is
litterally 2 seconds because the dedupe is done online so it almost feels
like creating a hardlink at the expense of pegging the CPU. I have a dual
core 2Ghz opteron and I can see the need for a quad core xeon but the
benefits look to be huge.
If you have some hardware lying around I encourage you to test out zfs
dedupe. nexenta core3 alpha2 and opensolaris b129 both have it and it is
I also tested the opensolaris cifs server joined to AD sharing a deduped
disk and that works very well also, your AD users and ACLs are handle well
From: Chris Bennett <chris@ce...> - 2009-12-23 11:53:11
> If you have some hardware lying around I encourage you to test out zfs
> dedupe. nexenta core3 alpha2 and opensolaris b129 both have it and it is
> very nice.
Hmm, thanks for the report - it's great to hear you were able to get
it working and see some good results.
I've thought about backing backuppc onto a lessfs volume (fuse-based
inline deduping filesystem) - havn't tried it yet, but if anyone has,
I'd like to hear more about it also.