Verify only button does not work properly
A Windows tool for writing images to USB sticks or SD/CF cards
Brought to you by:
gruemaster,
tuxinator2009
Menu
▾
▴
#50 Verify only button does not work properly
Hi there,
I'm writing an almost 8GB image to an 8GB SD card. After writing completes, I click on Verify Only - and it fails with: Verify Failure - 1.0 Verification failed at sector: 6144. I've tried this three times. Twice on the same SD and then on another SD and all three failed on 6144. It fails instantly as well - which makes me wonder if it even tries. I tried toggling "Read Only Allocated Partitions" as well. One SD is Transcend and the other one is Sandisk.
I then read the files from both SD cards, and used Total Commander to diff them. The two SD card images differ from each other and differ from the original image.
I've now tried it on a third SD card. Which when I read it only difference right at the end and differs with zeros and nothingness. So actually looks very similar. But this one also fails on sector 6144 immediately.
Thanks for the report. I will try to look at this as soon as I have time. I may not have the resources to reproduce this accurately, and it is (slightly) possible that the issue is a difference in the SD cards themselves and how they write the data internally. It could be as simple as a single bit difference.
All good, let me know if I can help.
I would consider trying a different SD card reader/writer. It sounds like your writer is having problems considering how low of a sector its failing on and the fact that total commander sees differences.
Hi James,
I have actually tried that as well. I have a usb one that I use for my gopro that I tested as well as plugging the SD card directly into my notebook's card reader. Both failed the same way.
I have a few additional thoughts on this for you to try.
Start with 1 image (ex: test.img). Generate a checksum (sha256 preferably) and copy it to notepad with the filename on the same line followed by "Baseline". Write it to the SD card.
Now change the name in the textbox to test1.img and do a Read. Generate a checksum and copy to a new line in notepad, same format (except Baseline). Run a Verify test. Record the results (Read Pass/Fail). Write it, reverify, and record the results (Write Pass/Fail). Try this several times and see if they pass/fail each time and the checksums change or are consistent.
My theory is this:
If it fails the first time only, Windows is doing something to the written filesystem post-write. If however it fails each time, either the SD controller is caching something wrong, or it is injecting some spurious data.
Another test is to write teh same image to a USB drive (not a USB>SD adapter) and see if it behaves the same way. In my limited testing, I only had a handful of USB sticks to work with, and a couple of fairly old SD cards (from when I was on the Arm Enablement team working on Ubuntu 10.04-12.04 timeframe, so 6 years old).
I am also experiencing this problem. Running Windows 10 on a Surface Pro 3, using the built-in micro SD card slot.
I downloaded the latest (April 10, 2017) Raspbian image from
https://downloads.raspberrypi.org/raspbian_latest
and wrote it to an 8 GB micro SD card. I immediately clicked "verify only" and got the error "Verification failed at sector: 8192".
Then I tried the following:
This time, the verify succeeded without any errors.
Finally, I wrote the ORIGINAL .img file downloaded from Raspberry Pi, and it AGAIN failed the verify.
Checksums for the two image files differ, so there is definitely something weird going on. I hope some of this helps to pinpoint the cause of the discrepancy...
This is good information. It suggests that either the SD reader or possibly the driver is making a subtle change (i.e. a signature) to the image in an area that doesn't impact the actual image (changing the drive label maybe?). Unfortunately, my only Windows test systems have USB SD readers, but I will see what I can do to get a system to test this theory out with.
Last edit: Tobin Davis 2017-05-18
I did some more tests a week ago and then put it on hold. I had some more time to look at it this morning and I agree with Kris' conclusion that it is greatly dependent on the file being written.
I have two images that I can write, read, verify etc as much as I want and they don't change. I have one image that fails verification immediately after writing on sector 6144. Thinking about it these images were created locally after reading them from the device in Win32DiskImager.
One obvious issue is that if you write a 8GB image to a 16GB drive and then read it back, the image size increases to 16GB as it includes the empty space (unless you have selected read only allocated). But even with read only allocated the file size read does not match the file size written. Written file = 7.37 GB (7,913,603,072 bytes). Read file = 7.35 GB (7,896,825,856 bytes) or 16GB with read only allocated unticked.
a) Baseline sha256 - 7ad91ca36020b90617edbb8b8fe98f4a57a45a023b341954033f78321d291695
Verify failed on sector 6144.
b) Written and read - dc396505c5f1e3a33d2ddf94394a16fd90e3c49e17739925bf20e4b51368590a
c) Written and read - dc396505c5f1e3a33d2ddf94394a16fd90e3c49e17739925bf20e4b51368590a
Verify ran and passed successfully.
d) Read at full size - ca3078263dda46099dd557e5f430a6de4df9e8d4a06d685b13a978f362f832ab
File size = 14.8 GB (15,987,638,272 bytes)
Verify with read only unchecked - passed successfully
Verify with read only checked - passed successfully
b and c was read with read only allocated on. d was read without it selected.
It seems that files can change when they are generated by a third party and then written / read by Win32DiskImager. But if they are created by reading in Win32DiskImager they remain the consistent.
All tests were performed on the same SD card on the same notebook with the same controller etc. I verified the integrity of the SD card using the h2testw application.
Interesting theory, but not valid with the programming model I have written. The program is written to read a block from the file and compare to the same block on the device. With this method, theoretically, a 1G image written to a 128G device will still verify successfully.
I belive there is some other mechanism in play here that writes an unknown quantity of bits after the write operation has completed (short of physical removal, the program has exclusive control of the device during read/write operations).
What I need to do to help narrow this down is to first reproduce it (need hardware), then manually compare the failing area with a hex editor.
Last edit: Tobin Davis 2017-06-16
Let me know if I can help.
As above, the only difference between repeatedly working verification and repeatedly failing verification was the source image.
Wrote the img file from https://octopi.octoprint.org/download/stable/octopi-jessie-lite-0.14.0.zip to a 32 GB micro SD card (windows 10 Lattepanda system). and immediately clicked "verify only" and got the error "Verification failed at sector: 8192". Funny the SD will boot up and run once in the PI but, fails after reboot.
I have enough information to go on to debug the Verify function. As to your Pi image failing after second boot, this is not something I can help with here. It could be something happening on the image after 1st boot. I will be checking this in my spare time as I recently picked up a couple of PI3-B systems for exactly this configuration (Octoprint for my 3D printers). But this is not the correct forum to fix issues with that install (unless I see a reason Win32DiskImager is breaking the image).
While verify threw the 8192 error the image is working.Thanks for checking
it out..
Note, The PI failure didn't fail when I disabled security. Just saying.
Cheers,
Steve
try my Android Simulator:
https://play.google.com/store/apps/details?id=com.stevokeano.angrycanyon
Nature made time so everything doesn't happen at once. Nature Made space so
everything doesn't happen to me!
On Sep 11, 2017 12:16, "Tobin Davis [Masked]"
FWD-737RLIAYEBAHYCMAMMTFQNDBEMOCOIC3JIRVQOKAIYAEMKNTIUQFWIIGENGDABIKIATSVVAVPEAIMVGSEA3QANQ6FIBEAAKYPQRVRIAAEA======@ abinemail.com wrote:
Related
Bugs: #50
I encountered the same problems as above, with back-ups made from my RuneAudio Pi-image. Me thinks it has to do with the fact that Disk Imager needs a Windows drive letter. Which means that there must be at least one (1) partition on the card that Windows recognizes (FAT or FAT32). In that partition Windows 10 creates a hidden directory "System Volume Information" with 2 files in it: "IndexerVolumeGuid" and "WPSettings.dat". When I stick the card in my PC Windows 10 does this, and the back-up image will contain this dir. When you restore the image on another card (which ALSO needs a drive letter partition to begin with!), after the restore Windows probably overwrites the restored hidden dir with the original hidden dir from the other card. So the VERIFY fails...
After making a new image from the other card, and comparing old and new image files by running FC in a CMD-box: it listed two differences:
C:>fc /B "q:\Radio-1.lan.PART.20170915.img" "q:\Radio-1.lan.PART.20170905.img"
Comparing files Q:\MIJN BACK-UPS RPI\RUNEAUDIO\Radio-1.lan.PART.20170915.img and Q:\MIJN BACK-UPS RPI\RUNEAUDIO\RADIO-1.LAN.PART.20170905.IMG
00136292: 2F 25
001362F2: 2F 25
C:>
When Listing (hex) both images in Total Commander I found the differences to be in that "hidden dir" area.
Hope this helps.
BTW a proposal for Win32DiskImager: is it possible to continu verifying, and listing a (max) number of differences with adresses (like dos 5.0 FC = file compare does)?
Hi,
The same problem is also here.
I see that computers with Windows 10 fails, while the same PCs (HW of the same type/model) works when the installed OS is Windows 7.
Another different that I noticed is that in the file explorer windows 10 shows the SD card as 2 separate drives (one empty and one with data), while for windows 7 it is recognized only as a single drive in the windows file explorer.
Windows 7 can backup and succesfullt veryify SD card (ubuntu for Raspberry PI), while the same SD card failsthe same backup-verify process on windows 10 - at the very beginning of the verification setp.
The md5sum of the images that were backed up using 2 different OS (wind10 vs win7) are different.
Hope that this info helps further.
Any new ideas?
Thanks,
Amit
P.S.
I used win32diskimager version 1.0.0
This is indeed helpful. My dev system is running Windows 7 (which is why I don't see it). Windows 10 is making a filesystem change after we release the drive from writing. I'll bet they are adding their 'Lost & Found' directory or a partition timestamp of some sort.
Great !
So we are hopefully on the way to have a solution.... (?)
I would start by trying to reproduce this issue using 2 virtual machines: one with installations of windows 7 andthe other with windows 10, to see the difference and try to debug it.
Can you update here as soon as you will have any news?
Thanks :)
Hi,
Are there any news about that issue? I have the same problem here. I'm not able to boot any image which I write to SD card. I was trying to write images to SD card using Win32DiskImager 1.0, Etcher, dd on Windows, than I tried dd from Linux in VBox but nothing seems to work. So it looks like it is not an issue only with Win32DiskImager. It seems like Windows 10 is making changes to filesystem independently on program you use to write.
I used Win32DiskImager on Windows 7 to write exactly same image and everything works fine.
I will be waiting for any news.
Thank you
Sorry, no news to report. The information you have provided is very useful, though. I will use it to further debug the problem when I can get some spare cycles.
Thanks.
More data to help determine what is going wrong.
I did the following:
1. wrote a raspian image to a 16G SD card using etcher on a windows 10 laptop with internal card reader
2. installed the card in my raspberry pi and booted the pi and configured some things, wifi, location, tft display
3. moved the SD card back to the windows 10 machine and used Win32DiskImager to copy the contents of the SD card to the hard drive on the lap top
4. did a verify only and got an error "verification failed at sector 74752"
5. left the SD card in the windows machine and did another read with Win32DiskImager this time without the "read only allocated partitions" selected
6. did a verify only and got an error "verification failed at sector 74752"
Images are the same 15,440,896 size if I select "read only allocated partitions" or not.
I can confirm this is still an issue with version 1.0 and win10.
I start with a properly FAT32 formatted 8GB uSD card that appears as F:
When I try and use the 2019-07-10-raspbian-buster-full image I end up with a 256MB partion labelled "boot" (Striclty speaking: 264,289,792 bytes) of which 40,929,280 are used. and a 5.73 GB unformatted primary partition (or at least Win10 thinks it is unformatted.) This fails verification at block 8192 and will not boot in the pi. The files in this 39 MB boot partition appear to be valid for the pi boot, but not nearly enough is there.
If I use a Win7 system I end up with a single useable partion that boots correctly.
For Raspbian images created from scratch, this might be caused by the folder "System Volume Information" as we can see in the forensic tool "FTK Imager".
Win32Disk Imager reported sector 8192:
However, I can neither find a difference in logical sector 8192 nor physical sector 8192 with FTK Imager.