#63 trim-sector-ranges cannot trim start and end part of drive

[Tom Yan]

I used the command:

for i in cat sandisk_list; do sudo hdparm --please-destroy-my-drive --trim-sector-ranges "$i" /dev/sdc; done

with the attached "sandisk_list" to trim my SanDisk Extreme USB 3.0 (32gb, no uas support). By checking with hexdump (and shred -n 1), I can see that the first and the last range in the list cannot be discarded:

...
001fff0 25ac 5acf 13b3 e8fe 9229 6b33 6696 f1df
0020000 0000 0000 0000 0000 0000 0000 0000 0000
*
74e320000 4a68 8a38 23e1 6845 1014 6623 6884 d5d0
...

(0x20000 / 512 = LBA 256; 0x74e320000 / 512 = LBA 61282560)

[Tom Yan]

There is also another weird thing. During the trim loop of hdparm, for some reason there seems to be consistently disk access on my system drive (internal SATA). The HDD LED keep flashing (not the one on the USB drive), and I saw writing access on the system drive, but reading access on the USB drive in iostat:

Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 1276.00 0.00 22978.00 0 45956
sdc 1745.50 12618.50 0.00 25237 0

Does hdparm write a log to somewhere?

[Mark Lord]

No, hdparm does not write to a log or anything else.
Perhaps the "system drive" (did you mean "root filesystem" instead?) wasn't mounted with -onoatime ?

[Mark Lord]

Oh, just noticed your edit, where you added "trim" in front of "loop".

As part of the over-zealous safety features, hdparm does a "-f" before each TRIM command. "-f" causes this sequence: sync, fsync, fdatasync, sync.

The "sync" commands are global, so they will trigger writes of any dirty buffers systemwide, on all mounted filesystems. Overkill, yes, but safe.

[Tom Yan]

By the way is it very likely to be an issue of the disk firmware instead of hdparm or something else?

It make some sense when I can't trim the last 71 sectors because the minimum allowed is 256 for this drive, but I don't understand about the first 256 sectors.

Also there's something weird about LBA 0. Whenever I use ranges starting with 0 (e.g 0:512, 0:65535), the trim command has no effect at all, but if I use 1:511, it does erase the second 256-sector block (LBA 256 - LBA 511), while the first one is still intact.

[Tom Yan]

I have some questions about "--trim-sector-ranges-stdin" as well. According to the man page of hdparm:

"It also permits batching of many more sector ranges into single commands to the drive, up to the currently configured transfer limit (max_sectors_kb)."

But from the output of verbose (outgoing_data), it seems to batch up to 4kb, and when I grep sysfs:

/sys/block/sdc/queue/max_sectors_kb:120

So is it just the man page outdated or some other reasons?

Also, have you really seen drives that can actually handle "batch trim" like this? Because my drive seems to only support one single range anyway, that's why I loop hdparm in shell instead of using the stdin option.

Edit: Actually it seems that i can batch up to 65536 sectors, and if I do more than that and run the command twice, I can trim up to 131072 sectors, but no more even if I run it the third time. For example:

sudo hdparm --please-destroy-my-drive --verbose --trim-sector-ranges 256:65280 65536:256 65792:65280 131072:256 131328:65280 196608:256 196864:65280 262144:256 /dev/sdc

After the first run LBA 256 - LBA 65791 will be discarded, then if I run the same command for a second time, LBA 65792 - LBA 131327 will also be discarded, but further blocks can only be discarded if I do it with another commands with the discarded ranges removed from the command, like:

sudo hdparm --please-destroy-my-drive --verbose --trim-sector-ranges 131328:65280 196608:256 196864:65280 262144:256 /dev/sdc

then the same behaviour goes on.

[Mark Lord]

I don't know when I'll be able to look at the original issue here, but the "batching" does indeed work with ALL drives I have tried it on -- dozens and more.

The included "wiper.sh" script does just that: it gets free block lists for a filesystem from the kernel, reserves them temporarily, and feeds those lists into hdparm for batched TRIM operations. Very fast compared with one-range-at-a-time, or even with the more widely known "fstrim" ioctls.

Cheers

[Tom Yan]

Actually it seems that batch mode does work for my drive as well. It's just that it has a limit on number of sectors that can be discarded per command inspite of the number ranges batched. As mentioned before, 65536 is the reliable maximum (with ease of calculation considered). So instead of looping hdparm in shell, I can actually reliably TRIM the device with 128-sector ranges (because 512 ranges is the upper limit of the current hdparm) through the stdin way and let hdparm loop through the device with TRIM commands. If hdparm has an upper limit of 64 ranges (512 bytes / 8 bytes), I can then use 1024-sector ranges instead.

[Mark Lord]

The maximum number of ranges which can be "batched" together in a single TRIM request is dependent upon the limits reported by the drive firmware. Many drives support only a single 512-byte block of ranges, whereas others support unlimited numbers of 512-byte blocks of ranges per request.

Cheers

[Tom Yan]

Then is there any specific reason for you have set the limit to 4096-bytes block (i.e. eight 512-byte blocks) of ranges per request?

P.S. or does hdparm actually set the limit according to what the device report: Data Set Management TRIM supported (limit 8 blocks)

[Tom Yan]

EDIT: see my reply above

So I did some more test and I found that actually, even if I loop hdparm instead of using the stdin "batch trim", it cannot be guaranteed that the drive is fully wiped. It only works if I introduce some overhead to each iteration, like a sudo session with each iteration:

for i in cat sandisk_list; do sudo hdparm --please-destroy-my-drive --trim-sector-ranges "$i" /dev/sdc; sleep 1; done

which I think is actually why there's constant writes to my "system drive" -- because sudo writes some logs to the systemd journal for each session/iteration; or:

for i in cat /home/tom/sandisk_list; do hdparm --please-destroy-my-drive --trim-sector-ranges "$i" /dev/sdc; sleep 0.05; done

in one sudo session (sudo -i), while 0.05s is basically arbitrary (I mean, I don't really know how much/little is enough, which I consider an issue). I set it to that because it makes the total time consumed a bit more than looping sudo.

I also tried to do it with sudo -i but without sleep, and wait for 1 min (which is more than the total time consumed by either of the previous method) before checking with hexdump. Still, only pieces of the drive is wiped.

[Tom Yan]

Just found something on the geometry:

[tom@localhost ~]$ sudo hdparm /dev/sdc

/dev/sdc:
multcount = 0 (off)
readonly = 0 (off)
readahead = 256 (on)
geometry = 29923/64/32, sectors = 61282631, start = 0

while 29923 * 64 * 32 = 61282304 and 61282631 - 61282304 - 256 - 71 = 0