TRIM support on LVM and dm-crypt
Brought to you by:
ideguy
Menu
▾
▴
#18 TRIM support on LVM and dm-crypt
I've added support for TRIM'ing even if the fs is on an LVM or dm-crypt device. This currently only supports linear logical volumes on top of a single device (i.e. no striped, mirrored, or concatenated volumes), but it does support arbitrary LVM and dm-crypt stackings (i.e. LVM on top of dm-crypt or visa-versa).
BACK UP YOUR DATA before using this. I've tested actual TRIM's on a dm-crypt device, as well as set up a dummy stacked LVM and dm-crypt and verified the reported ranges from --verbose make sense, so I'm reasonably confident it works properly, but...
To use this, LVM must be activated and/or the encrypted devices opened. For online TRIM specify the fs mount point as usual; for offline TRIM specify the device the fs is usually mounted on (e.g. /dev/mapper/home) (in a stacked setup make sure you specify the top-level device, not some intermediate device; e.g. if you have LVM on top of dm-crypt then specify the logical-volume's device, such as /dev/mapper/VolGroup0-LogVol1).
I'm attaching both the full script as well as the diff; both are against version 2.5 because there is no 'rdev' command on my distro, but the patch should apply straightforwardly to 2.6 too. The attached full script also has the x25-m patch, btw. The patch consists of 3 parts: the first part detects device-mapper devices and walks them to figure out the underlying scsi/sata device and to calculate the additional lba offset to use; the second part replaces the use of 'fsdev' with 'fspar' as the fs-device may not be the same as the underlying scsi/sata device; the third part adjusts 'fsoffset' to account for the additional LVM/dm-crypt headers and to account for the fact that the fibmap will return lba's that are relative to the LVM/dm-crypt container.
This should fix both http://sourceforge.net/tracker/?func=detail&aid=2927763&group_id=136732&atid=736685 and http://sourceforge.net/tracker/?func=detail&aid=2990354&group_id=136732&atid=736682 - the reason I created a separate issue here is so I can attach files.
Patch to wiper.sh 2.5 to support (linear) LVM and dm-crypt
Sorry, I take it back: http://sourceforge.net/tracker/?func=detail&aid=2990354&group_id=136732&atid=736682 is not fixed by this (though that works for me, i.e. this modified wiper.sh assumes 'hdparm --fibmap' works on dm devices, which it does for me using hdparm v9.27 and kernel 2.6.31).
thanks very much for this! i just tested your new script with a ocz vertex2 (sandforce sf-1200) and it works for my root partition (no dmcrypt) and another partition which is dmcrypt/luks (no lvm). at the moment i'm diffing the partition with my backup but i'm confident :)
i only had to change line 747 ( i>=512) to i>=64, everything greater than 64 doesn't work => results in FAILED: Input/output error.
wiper.sh version2.5 with device-mapper and max-ranges patches applied
Thanks. I've created a patch for limiting the max-ranges at https://sourceforge.net/tracker/?func=detail&aid=3002193&group_id=136732&atid=736684 , and updated the attached wiper.sh with that patch - it should automatically limit the range to 64 now for your disk.
the dmcrypt patch doesn't seem to work for the intel x25-m. my root partition is encrypted, here's the wiper.sh output:
./wiper.sh --verbose /
wiper.sh: Linux SATA SSD TRIM utility, version 2.5, by Mark Lord.
rootdev=/dev/mapper/root rdev=/dev/mapper/root
fsmode1: fsmode=read-write
/: fstype=ext4
freesize = 5658380 KB, reserved = 56583 KB
Preparing for online TRIM of free space on /dev/mapper/root (ext4
mounted read-write at /). This will be a DRY-RUN only. Use --commit to
do it for real. Creating temporary file (5601797 KB)..
Syncing disks..
Simulating TRIM operations..
get_trimlist=/sbin/hdparm --fibmap WIPER_TMPFILE.14619
(dry-run) trimming 0 sectors from 0 ranges
Removing temporary file..
Syncing disks..
Done.
=> see "trimming 0 sectors from 0 ranges"
performing a --commit doesn't seem to do anything (no hard disk load at all):
This operation could silently destroy your data. Are you sure (y/N)? y
Creating temporary file (5594336 KB)..
Syncing disks..
Beginning TRIM operations..
get_trimlist=/sbin/hdparm --fibmap WIPER_TMPFILE.14100
Removing temporary file..
Syncing disks..
Done.
the script _does_ work on my /dev/sda1 ext4 partition which is not encrypted.
It's looking like 'hdparm --fibmap' may not be working properly (returning an error). First of all, what kernel version and version of hdparm are you using? Second, can you add the following around line 751 (just before the $get_trimlist call):
$get_trimlist
echo $?
and show me the output.
sorry, i completely forgot to mention that. it's 2.6.33.2 and hdparm v9.28., 32bit.
/dev/mapper/root = /dev/sda6
it seems that the device can't be found:
./wiper_dmcrypt.sh --verbose /dev/mapper/root
wiper_dmcrypt.sh: Linux SATA SSD TRIM utility, version 2.5, by Mark Lord.
rootdev=/dev/mapper/root rdev=/dev/mapper/root
fsmode2: fsmode=read-write
/: fstype=ext4
freesize = 8186220 KB, reserved = 81862 KB
max-ranges = 512
Preparing for online TRIM of free space on /dev/mapper/root (ext4 mounted read-write at /).
This will be a DRY-RUN only. Use --commit to do it for real.
Creating temporary file (8104358 KB)..
Syncing disks..
Simulating TRIM operations..
get_trimlist=/sbin/hdparm --fibmap WIPER_TMPFILE.10511
254,0: device not found in /dev
2
(dry-run) trimming 0 sectors from 0 ranges
Removing temporary file..
Syncing disks..
Done.
It looks like for some reason you have no /dev/dm-<num> devices. On my system I get both /dev/mapper/root and /dev/dm-0 devices (with same major/minor), and 'hdparm --fibmap' only looks in /dev/ (not subdirectories thereof). The proper fix would probably be for hdparm to be fixed to look for devices anywhere in the /dev/ tree, but in the mean time you could work around it by explicitly creating a /dev/dm-<num> device: 'mknod /dev/dm-0 b 254 0' (this won't survive a reboot, though). It might also be possible to add an ugly hack to wiper.sh to temporarily create such a device entry and delete it again on exit - I'll think about that.
wiper.sh version2.5 with device-mapper and max-ranges patches applied; this variant contains a hack to deal with a limitation of 'hdparm --fibmap'
Please give the new variant I just uploaded a try - that creates a temporary device entry directly under /dev/, if necessary, to work around the fibmap issue.
i'm currently creating a new backup before trying the (old) wiper script. i've upgraded my udev and lvm2 packages and the device nodes /dev/dm-* are now being created automatically. a dry-run seems to be working fine, although the output looks a bit different to what i saw on some vertex2 drive. after the backup is done i'll do a --commit and see what happens :)
Thanks for your patch. It seems very important for huge encrypted partitions on ssds.
I have some questions.
How does it free sectors if they are encrypted and consist often pseudo random data?
And what happens for example if you tell the SSD that sector xy is free and some seconds/ms later this sector gets used through the file system or mapper again? If the Trim logic directly frees the sector and then ignores it, it should be no problem but if it is a long process it might mess things up.
And last but not least will this patch be integrated and if so in which version?
Encryption is on a block-by-block basis, and the block-number mapping
from non-encrypted to encrypted is a simple offset (i.e. fs writes to
logical block N, which gets encrypted and written to disk block N+M,
where M is fixed). Therefore you can ask the fs for the list of free
logical blocks and easily compute the corresponding list of free disk
blocks. The contents of the blocks is irrelevant (blocks are not
determined to be free by, say, looking whether they are 0, but instead
by asking the filesystem for the list of free blocks).
As to timing, that is no different here than with the original trim:
the trim command is synchronous (i.e. the blocks have been trim'd when
the operation returns) so there is no race-condition.
Thanks for your reply.
I have tested it with and without dm-crypt and it seems to work - or at least I got no md5 errors :).
Ext4 has a non standard stripe set and dm-crypt a non standard offset so it seems pretty stable but I am going to test it regularly.
Is it a problem to run it with cron. I mean what happens if the script is started during high disc activity like the creation of many/big files and similar?
Running this while you have a lot of write activity will potentially have three effects:
* reduced performance while trim is being done
* if you're writing large files you may run out of space; this is because online trim
works by essentially creating a huge file that takes up all free space and then
freeing those blocks, so during this operation your filesystem is essentially full.
However, the script does leave some free space, namely at least 1% or 7.5MB
(whichever is larger) - see around line 505 in the script - you can certainly adjust
this if you need to.
* for the same reason as above, files created during this time will possibly be
more fragmented than they would otherwise be; while this is not nearly as big
an issue on SSD's, it's not entirely ignorable either (the "seek" times are fast
but not 0).
I have a crypted Ultradrive GX, with LVM..
So : Luks->PV->VG->LV_root & LV_swap.
This does not work. Is there a chance to get it working? :(
"This does not work" is not very descriptive - please explain in full detail: do you get an error? What is the output from running with --verbose? What is the exact command line you're using?
Also, for swap you need a different script - see https://sourceforge.net/tracker/?func=detail&aid=2997560&group_id=136732&atid=736684
It was my fault. Sorry. Didnt get it that i have to commit it on the LV.
It trimmed obviuosly.
I wonder if it ever will be possible to trim my SSD when its mounted?
Thanks man!!
So again me:
If i try to trim from Ubuntu 10.10 directly. But:
"Simulating TRIM operations..
(dry-run) trimming 0 sectors from 0 ranges
wiper_trim.sh: line 771: [: -gt: unary operator expected
Removing temporary file
.."
What happens here? =)
Thx!
It works fine on a mounted filesystem (it will indicate it's doing an online trim).
As to the error, that happens if there are no ranges to trim (I'll fix it), which usually means there's some other issue - --verbose might give some more info.
Strange. Just reinstalled hdparm again ;)
I think it does work now:
cat wiper.sh.log
wiper.sh: Linux SATA SSD TRIM utility, version 2.5, by Mark Lord.
rootdev=/dev/mapper/vg-root rdev=/dev/mapper/vg-root
fsmode2: fsmode=read-write
/: fstype=ext4
freesize = 26199924 KB, reserved = 261999 KB
max-ranges = 65535
Preparing for online TRIM of free space on /dev/dm-2 (ext4 mounted read-write at /).
This will be a DRY-RUN only. Use --commit to do it for real.
Creating temporary file (25937925 KB)..
Syncing disks..
Simulating TRIM operations..
get_trimlist=/sbin/hdparm --fibmap WIPER_TMPFILE.4706
Trim ranges:
Removing temporary file..
Syncing disks..
Done.
How can i check if it works?
It looks like it isn't finding any ranges to trim - or did you omit those from the output below? Also, there should've been an output line like
(dry-run) trimming 3456824 sectors from 546 ranges
(though the number of sectors and ranges will be much higher in your case)
If you want to be extra careful, have a look at the ranges (each is of the form <s tart-block>:<num-blocks>) and make sure they actually map with the logical-volume (for some info on how to figure out how to calculate this mapping, see e.g. http://smartmontools.sourceforge.net/badblockhowto.html#lvm ).
And of course back up your data.
The run the script with --commit .
Attached scripts and patches are (for) version 2.5 of the wiper script, while version 3.1 is available from http://sourceforge.net/projects/hdparm/files/ - however, version 3.1 doesn't seem to include support for LVM/dm-crypt; why is that, and is there any plan to merge the patch with newer wiper.sh versions?
Thanks a lot!
aiming at the same thing:
The current hdparm package contains a contribution for using TRIM on raid1 arrays.
Would it be possible to combine that with this approach here?
as in make it possible to trim ext4 on encrypted raid1 arrays?
Because essentially that is exactly the setup I'm running...
I've looked at the sources myself and couldn't find a reason preventing this, but then again I'm just a humble coder and don't know all that much about the inner workings of hdparm.
Would something like this be possible?