From: AJ O. <coo...@gm...> - 2011-06-29 17:49:06
|
I have a few inter-related issues: - Why would one kernel boot a card that another kernel can't? - Why would a card's disk geometry matter for boot? - Who is a good manufacturer for getting hardware-identical cards in bulk? - How can I probe the actual "disk geometry" of an sd card? I bought 100 Transcend SD cards a little while ago and duplicated them with an OpenEmbedded-based filesystem (linux-2.6.36). There were a few "bad" cards that I threw out, but the success rate was acceptable. In the next round of 40 SD cards I used a Linaro-based filesystem (linux-2.6.39) and had about a 50% failure rate when testing that the cards would boot, which is absurd. There kernel reports: [ 1.003204] mmcblk0: unknown partition table However, the cards would mount and show files just fine. I reduplicated one of the non-booting cards with an OpenEmbedded filesystem and then it booted. Weird! After some investigation I found that using `gparted` (instead of `fdisk`) to create a new partition table and then `rsync`ing the contents of the original filesystem resulted in a booting Linaro card. Rinse and repeat and I ended up with 3 images which only vary by the disk geometry as reported by `fdisk -l`: - 50% -- 255 heads, 63 sectors/track, 974 cylinders - 40% -- 2 heads, 4 sectors/track, 1957632 cylinders - 10% -- 247 heads, 62 sectors/track, 1022 cylinders - 1 card still didn't boot I'm lost. Please advise. AJ ONeal Non-booting kernel message [ 0.923309] Waiting for root device /dev/mmcblk0p2... [ 0.957885] mmc0: host does not support reading read-only switch. assuming write-enable. [ 0.982025] mmc0: new high speed SDHC card at address b368 [ 0.988494] mmcblk0: mmc0:b368 USD 7.46 GiB [ 0.993957] mmcblk0: detected capacity change from 0 to 8018460672 [ 1.003204] mmcblk0: unknown partition table [ 1.036926] VFS: Cannot open root device "mmcblk0p2" or unknown-block(179,2) [ 1.044433] Please append a correct "root=" boot option; here are the available partitions: [ 1.053344] b300 7830528 mmcblk0 driver: mmcblk [ 1.058959] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,2) Booting kernel message [ 1.122070] mmc0: host does not support reading read-only switch. assuming write-enable. [ 1.146087] mmc0: new high speed SDHC card at address b368 [ 1.152557] mmcblk0: mmc0:b368 USD 7.46 GiB [ 1.158020] mmcblk0: detected capacity change from 0 to 8018460672 [ 1.166351] mmcblk0: p1 p2 p3 [ 1.259674] EXT3-fs: barriers not enabled [ 1.265411] kjournald starting. Commit interval 5 seconds [ 1.271331] EXT3-fs (mmcblk0p2): mounted filesystem with ordered data mode [ 1.278686] VFS: Mounted root (ext3 filesystem) readonly on device 179:2. |
From: Paul N. <pa...@id...> - 2011-06-30 00:13:25
|
> I'm lost. Please advise. I hope to be in the same position soon. I mean, duplicating cards, not lost, I`m there already. How big are the partitions? I found it was best to make the partitions not fill the entire card so there was some leeway to allow for cards being slightly different sizes, but I`ve only written to maybe 15 cards to date. Thanks, -- Paul Nolan, CEO Idruna Software Inc. |
From: AJ O. <coo...@gm...> - 2011-06-30 00:24:32
|
The problem isn't with the filesystem duplication or the contents. It's that the kernel doesn't recognize the partition table on boot and fails to continue to boot. One of the weirdest problems I've ever come across. It literally has to do with the *fake* disk geometry used to create the partition table. I think this must be a kernel bug mixed with a very weird variation in the card hardware. The card reads fine on a linux system. AJ ONeal On Wed, Jun 29, 2011 at 5:12 PM, Paul Nolan <pa...@id...> wrote: > > > I'm lost. Please advise. > > I hope to be in the same position soon. I mean, duplicating cards, not > lost, I`m there already. How big are the partitions? I found it was > best to make the partitions not fill the entire card so there was some > leeway to allow for cards being slightly different sizes, but I`ve only > written to maybe 15 cards to date. > > Thanks, > -- > Paul Nolan, CEO Idruna Software Inc. > > > ------------------------------------------------------------------------------ > All of the data generated in your IT infrastructure is seriously valuable. > Why? It contains a definitive record of application performance, security > threats, fraudulent activity, and more. Splunk takes this data and makes > sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-d2d-c2 > _______________________________________________ > gumstix-users mailing list > gum...@li... > https://lists.sourceforge.net/lists/listinfo/gumstix-users > |
From: Ash C. <as...@gu...> - 2011-07-05 20:14:03
|
HI AJ, On Wed, Jun 29, 2011 at 10:48 AM, AJ ONeal <coo...@gm...> wrote: > After some investigation I found that using `gparted` (instead of `fdisk`) > to create a new partition table and then `rsync`ing the contents of the > original filesystem resulted in a booting Linaro card. > Rinse and repeat and I ended up with 3 images which only vary by the disk > geometry as reported by `fdisk -l`: > > 50% -- 255 heads, 63 sectors/track, 974 cylinders > 40% -- 2 heads, 4 sectors/track, 1957632 cylinders > 10% -- 247 heads, 62 sectors/track, 1022 cylinders > 1 card still didn't boot I suspect the partitioning method is the key difference. Two details: - does your version of linaro-image-tools use gparted or sfdisk? - does your version of linaro-image-tools 'dd' the partition table (e.g. the first megabyte) clean before calling any partitioning agent? I'm using the version of this package directly from source control and I've not noticed any problems but I'd be interested to know of any. -Ash |
From: AJ O. <coo...@gm...> - 2011-07-07 22:47:30
|
> > I suspect the partitioning method is the key difference. > Two details: > - does your version of linaro-image-tools use gparted or sfdisk? - does your version of linaro-image-tools 'dd' the partition table > (e.g. the first megabyte) clean before calling any partitioning agent? > > I'm using the version of this package directly from source control and > I've not noticed any problems but I'd be interested to know of any. I've tried partitioning every which way with a variety of tools and tried both letting linaro-image-tools overwrite the partition and leaving it alone. I don't think it's a problem at all with the linaro-image-tools. |
From: AJ O. <coo...@gm...> - 2011-07-07 22:38:02
|
Confirmed: the combination of a linaro-2.6.39 kernel with a transcend 8gb card results in flakey boots. - linaro-2.6.39 kernel is affected - transcend cards are affected - oe-2.6.36 kernel is *not* affected - sandisk 8gb cards are *not* affected (67 megabytes smaller) I zero'd a card, but instead of only zeros, I also wrote the sector number in each 512-byte block. I changed the kernel to print out the 512 bytes it read as the mbr to the screen. When the card didn't boot fully the printk showed all zeros except for a sector number about 100mb into the card. How weird! Hardware? Kernel? Anyway, today my sandisk cards arrived and all 10 of them worked flawlessly. I ordered another 50 and hope to see the same results. AJ ONeal On Wed, Jun 29, 2011 at 11:48 AM, AJ ONeal <coo...@gm...> wrote: > I have a few inter-related issues: > > - Why would one kernel boot a card that another kernel can't? > - Why would a card's disk geometry matter for boot? > - Who is a good manufacturer for getting hardware-identical cards in > bulk? > - How can I probe the actual "disk geometry" of an sd card? > > > I bought 100 Transcend SD cards a little while ago and duplicated them with > an OpenEmbedded-based filesystem (linux-2.6.36). > There were a few "bad" cards that I threw out, but the success rate was > acceptable. > > > In the next round of 40 SD cards I used a Linaro-based filesystem > (linux-2.6.39) and had about a 50% failure rate when testing that the cards > would boot, which is absurd. > There kernel reports: [ 1.003204] mmcblk0: unknown partition table > However, the cards would mount and show files just fine. > > I reduplicated one of the non-booting cards with an OpenEmbedded filesystem > and then it booted. Weird! > > > After some investigation I found that using `gparted` (instead of `fdisk`) > to create a new partition table and then `rsync`ing the contents of the > original filesystem resulted in a booting Linaro card. > Rinse and repeat and I ended up with 3 images which only vary by the disk > geometry as reported by `fdisk -l`: > > - 50% -- 255 heads, 63 sectors/track, 974 cylinders > - 40% -- 2 heads, 4 sectors/track, 1957632 cylinders > - 10% -- 247 heads, 62 sectors/track, 1022 cylinders > - 1 card still didn't boot > > I'm lost. Please advise. > > AJ ONeal > > > > Non-booting kernel message > > [ 0.923309] Waiting for root device /dev/mmcblk0p2... > [ 0.957885] mmc0: host does not support reading read-only switch. > assuming write-enable. > [ 0.982025] mmc0: new high speed SDHC card at address b368 > [ 0.988494] mmcblk0: mmc0:b368 USD 7.46 GiB > [ 0.993957] mmcblk0: detected capacity change from 0 to 8018460672 > [ 1.003204] mmcblk0: unknown partition table > [ 1.036926] VFS: Cannot open root device "mmcblk0p2" or > unknown-block(179,2) > [ 1.044433] Please append a correct "root=" boot option; here are the > available partitions: > [ 1.053344] b300 7830528 mmcblk0 driver: mmcblk > [ 1.058959] Kernel panic - not syncing: VFS: Unable to mount root fs on > unknown-block(179,2) > > > Booting kernel message > > [ 1.122070] mmc0: host does not support reading read-only switch. > assuming write-enable. > [ 1.146087] mmc0: new high speed SDHC card at address b368 > [ 1.152557] mmcblk0: mmc0:b368 USD 7.46 GiB > [ 1.158020] mmcblk0: detected capacity change from 0 to 8018460672 > [ 1.166351] mmcblk0: p1 p2 p3 > [ 1.259674] EXT3-fs: barriers not enabled > [ 1.265411] kjournald starting. Commit interval 5 seconds > [ 1.271331] EXT3-fs (mmcblk0p2): mounted filesystem with ordered data > mode > [ 1.278686] VFS: Mounted root (ext3 filesystem) readonly on device > 179:2. > |
From: Arnd B. <ar...@ar...> - 2011-07-08 07:53:18
|
On Friday 08 July 2011 00:37:35 AJ ONeal wrote: > Confirmed: the combination of a linaro-2.6.39 kernel with a transcend 8gb card results in flakey boots. > linaro-2.6.39 kernel is affected > transcend cards are affected > oe-2.6.36 kernel is not affected > sandisk 8gb cards are not affected (67 megabytes smaller) > I zero'd a card, but instead of only zeros, I also wrote the sector number in each 512-byte block. > > I changed the kernel to print out the 512 bytes it read as the mbr to the screen. > > When the card didn't boot fully the printk showed all zeros except for a sector number about 100mb into the card. > > > How weird! Hardware? Kernel? My guess is that it's the card's fault. There are a lot of cards that are simply not going to work with a Linux file system. In my experience, the Sandisk cards tend to have controllers that cope with proper file systems, while Kingston never do. Transcend doesn't make their own cards, they buy stuff from everybody, so you can be lucky or not. > Anyway, today my sandisk cards arrived and all 10 of them worked flawlessly. > I ordered another 50 and hope to see the same results. Ok. Can you be more specific which cards worked for you and which had problems? Please list the contents of /sys/block/mmcblk0/device/*, in particular the *id and date fields. We are still doing more analysis what the specific requirements are given a particular file system, but I have a good understanding of what the problems with many of the cards are. If you haven't seen my articles, please have a look at https://lwn.net/Articles/428584/ and https://wiki.linaro.org/WorkingGroups/Kernel/Projects/FlashCardSurvey If you send me a specimen of each cards you're interested in, I'll gladly do my analysis, or I can teach you how to do it yourself. It does take a bit of experience because there are so many different ways in which the drives can be screwed up. If you want a very simple test to see if a card is any good before you buy a lot of them, I recommend running (Warning: overwrites data on the card) flashbench --open-au --erasesize=$[4*1024*1024] --blocksize=4096 --open-au-nr=5 --random /dev/mmcblk0 This will print performance ratings for random write accesses to the card on five different erase blocks (assuming 4 MB erase block size). A card that can handle this should look like 4MiB 9.03M/s 2MiB 6.17M/s 1MiB 6.24M/s 512KiB 4.06M/s 256KiB 4.54M/s 128KiB 3.8M/s 64KiB 6.47M/s 32KiB 5.79M/s 16KiB 2.72M/s 8KiB 1.33M/s while a card that cannot handle this look more like 4MiB 8.81M/s 2MiB 5.43M/s 1MiB 3.81M/s 512KiB 2.27M/s 256KiB 1.34M/s 128KiB 860K/s 64KiB 481K/s 32KiB 229K/s 16KiB 117K/s 8KiB 57K/s If the last row is above 1MB/s, the card is fine, if it is below 100K/s, don't even think about using it. The explanation for this behavior is in the lwn.net article. Arnd |
From: Arnd B. <ar...@ar...> - 2011-07-08 08:29:16
|
Some more points: On Friday 08 July 2011 00:37:35 AJ ONeal wrote: > On Wed, Jun 29, 2011 at 11:48 AM, AJ ONeal <coo...@gm...> wrote: > I have a few inter-related issues: > Why would one kernel boot a card that another kernel can't? Possibly the controller is set up in a slightly different way, resulting in bad timings or other problems. > Why would a card's disk geometry matter for boot? The card's geometry is calculated from the partition table, it's not actually a property of the card. If you have identical cards that report different geometry, the reasons may be: * they were partitioned differently * some cards are read incorrectly, if the kernel interprets random data as a partition table, you get random geometry > Who is a good manufacturer for getting hardware-identical cards in bulk? This is hard. I suggest you first start with another equally hard question though: what is a manufacturer that can provide actually working cards? I would recommend using only those that make both the controller chips and the flash chips, which basically leaves * Toshiba * Samsung * Sandisk (cooperating with toshiba) * Lexar (micron/intel) Contact them directly to get a sample, do an extensive test the way I described, document the results and buy a lot from them. There are also companies like swissbit that know what they are doing and charge a lot extra for providing you cards that are actually made for running with Linux. > How can I probe the actual "disk geometry" of an sd card? It's complicated. For 8GB cards and larger, there is only one possible correct geometry: 63/255/1023. fdisk knows that and should do it correctly. On 4 GB cards, there are multiple options. You can either stick with the geometry that was originally used to partition the card, or you also use 63/255/xxx, with xxx being smaller than 1023 in that case. HOWEVER: Do not align partitions to full cylinders, even though that is the default in old versions of fdisk. Doing that will result in low performance and cards breaking faster when writing to them. The only correct way to partition an SD card is to align each partition to 4MB (8192 sectors). Old versions of xloader couldn't deal with that, so you might have to update xloader, or alternatively start the boot partition at sector 63 anyway (and then never write to it). We fixed this in linaro-media-create for the 11.06 release, but the gumstix wiki and many other places list still describe a procedure that *will* *eat* *your* *data*: http://wiki.gumstix.org/index.php?title=Boot_from_MMC#Partition_the_card http://www.gumstix.org/create-a-bootable-microsd-card.html http://fastr.github.com/articles/Partition-MicroSDHC-for-Gumstix-Overo.html http://www.omappedia.org/wiki/Android_Video_Run_From_SD_Tutorial The reason for this is simply that flash pages on the card have a size between 2KB and 32KB dependening on how dumb the controller is. Writing data that is not aligned to the pages requires expensive read-modify-write operations, and a lot of cards just end up doing many extra erase cycles. Someone should really start documenting a correct procedure in all of those places. What you really need to do using fdisk is to disabled compat mode ('c'), set sector addressing mode ('u'), and create the partitions each at a multiple of 8192 sectors. Arnd |
From: JamesAng <ang...@gm...> - 2011-07-11 07:21:14
|
Hi Arnd, Arnd Bergmann wrote: > >> How can I probe the actual "disk geometry" of an sd card? > > It's complicated. For 8GB cards and larger, there is only one possible > correct > geometry: 63/255/1023. fdisk knows that and should do it correctly. > On 4 GB cards, there are multiple options. You can either stick with the > geometry that was originally used to partition the card, or you also use > 63/255/xxx, with xxx being smaller than 1023 in that case. > For my 8GB Sandisk card and using Gumstix's wiki computation equation, my geometry is 63/255/966 instead of 63/255/1023 as you mentioned above. sudo fdisk -l /dev/sdc Disk /dev/sdc: 7948 MB, 7948206080 bytes 245 heads, 62 sectors/track, 1021 cylinders Units = cylinders of 15190 * 512 = 7777280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Is this correct or wrong in your opinion? (^^) Arnd Bergmann wrote: > > HOWEVER: Do not align partitions to full cylinders, even though that is > the > default in old versions of fdisk. Doing that will result in low > performance > and cards breaking faster when writing to them. The only correct way to > partition an SD card is to align each partition to 4MB (8192 sectors). > Old versions of xloader couldn't deal with that, so you might have to > update > xloader, or alternatively start the boot partition at sector 63 anyway > (and > then never write to it). > > We fixed this in linaro-media-create for the 11.06 release, but the > gumstix > wiki and many other places list still describe a procedure that *will* > *eat* > *your* *data*: > > http://wiki.gumstix.org/index.php?title=Boot_from_MMC#Partition_the_card > http://www.gumstix.org/create-a-bootable-microsd-card.html > http://fastr.github.com/articles/Partition-MicroSDHC-for-Gumstix-Overo.html > http://www.omappedia.org/wiki/Android_Video_Run_From_SD_Tutorial > > The reason for this is simply that flash pages on the card have a size > between > 2KB and 32KB dependening on how dumb the controller is. Writing data that > is not aligned to the pages requires expensive read-modify-write > operations, > and a lot of cards just end up doing many extra erase cycles. > > Someone should really start documenting a correct procedure in all of > those > places. What you really need to do using fdisk is to disabled compat mode > ('c'), set sector addressing mode ('u'), and create the partitions each at > a multiple of 8192 sectors. > > Arnd > What should be the correct procedure on your opinion that will prevent the ugly situation of "*will* *eat* *your* *data*" (^^) Thanks in adv. James. -- View this message in context: http://old.nabble.com/trouble-with-should-be-bootable-SD-cards-and-kernel-versions-tp31956950p32035344.html Sent from the Gumstix mailing list archive at Nabble.com. |
From: Steve S. <sa...@gm...> - 2011-07-11 13:59:03
|
On Mon, Jul 11, 2011 at 12:21 AM, JamesAng <ang...@gm...> wrote: > What should be the correct procedure on your opinion that will prevent the > ugly situation of "*will* *eat* > *your* *data*" (^^) I've updated the script for SD card preparation that is available on my site: http://www.sakoman.com/OMAP/a-script-for-partitioningformatting-a-bootable-sdmicrosd-card.html Arnd has reviewed the result and pronounced it acceptable. Based on my testing it seems fine, let me know if you find any issues. Any errors are my fault, not Arnd''s :-) Steve |
From: JamesAng <ang...@gm...> - 2011-07-12 02:15:52
|
sakoman wrote: > > I've updated the script for SD card preparation that is available on my > site: > > http://www.sakoman.com/OMAP/a-script-for-partitioningformatting-a-bootable-sdmicrosd-card.html > > Arnd has reviewed the result and pronounced it acceptable. > > Based on my testing it seems fine, let me know if you find any issues. > Any errors are my fault, not Arnd''s :-) > > Steve > Hi Steve, Arnd mentioned "What you really need to do using fdisk is to disabled compat mode ('c'), set sector addressing mode ('u'), and create the partitions each at a multiple of 8192 sectors." X-reference to your script's output, "Device Boot Start End #sectors Id System /dev/sdc1 * 128 131071 130944 c W95 FAT32 (LBA) /dev/sdc2 131072 7815167 7684096 83 Linux /dev/sdc3 0 - 0 0 Empty /dev/sdc4 0 - 0 0 Empty " is the compatibility mode disabled for the FAT partition? (^^,) Is the size of the MBR is fixed at 127 sectors for all size of SDcard? As example for my 8GB card and to set 1GB (2097152 sectors) for FAT, should it start at 128 and end at 2097025? or some formula to calculate the size of the MBR? Please correct my mistakes. (^^) TIA. James. -- View this message in context: http://old.nabble.com/trouble-with-should-be-bootable-SD-cards-and-kernel-versions-tp31956950p32042833.html Sent from the Gumstix mailing list archive at Nabble.com. |
From: Steve S. <sa...@gm...> - 2011-07-12 03:37:45
|
On Mon, Jul 11, 2011 at 7:15 PM, JamesAng <ang...@gm...> wrote: > Arnd mentioned "What you really need to do using fdisk is to disabled > compat mode > ('c'), set sector addressing mode ('u'), and create the partitions each at > a multiple of 8192 sectors." The script uses sfdisk, so Arnd's instructions for fdisk don't directly apply to my implementation. The basic strategy is of course the same. > X-reference to your script's output, > > "Device Boot Start End #sectors Id System > /dev/sdc1 * 128 131071 130944 c W95 FAT32 (LBA) > /dev/sdc2 131072 7815167 7684096 83 Linux > /dev/sdc3 0 - 0 0 Empty > /dev/sdc4 0 - 0 0 Empty > " > > is the compatibility mode disabled for the FAT partition? (^^,) > > Is the size of the MBR is fixed at 127 sectors for all size of SDcard? See the comments in the script. The MBR is 1 sector, 127 sectors of padding is added to align the start of the FAT partition to the underlying nand structure. > As example for my 8GB card and to set 1GB (2097152 sectors) for FAT, should > it start at 128 and end at 2097025? No, it should start at 128 and end at 2097151, and the ext partition should start at 2097152. If you take a look at the script source you will see that it is quite trivial to make this change. Do you really need a 1GB FAT partition? Steve |
From: JamesAng <ang...@gm...> - 2011-07-12 08:22:49
|
sakoman wrote: > > The script uses sfdisk, so Arnd's instructions for fdisk don't > directly apply to my implementation. The basic strategy is of course > the same. > Noted. (^^) If we intend to use fdisk manually to understand the procedures, what are the changes against the original wiki that we need? sakoman wrote: > > See the comments in the script. The MBR is 1 sector, 127 sectors of > padding is added to align the start of the FAT partition to the > underlying nand structure. > > No, it should start at 128 and end at 2097151, and the ext partition > should start at 2097152. > > If you take a look at the script source you will see that it is quite > trivial to make this change. > Ok, now I understand. The first sector is MBR and the # of sector needed for padding depends on the page size of the NAND structure. How do we determine the actual page size of the underlying NAND then? sakoman wrote: > > Do you really need a 1GB FAT partition? > The extra large FAT is just an educational example. (^^) No particular requirement. -- View this message in context: http://old.nabble.com/trouble-with-should-be-bootable-SD-cards-and-kernel-versions-tp31956950p32044081.html Sent from the Gumstix mailing list archive at Nabble.com. |
From: Steve S. <sa...@gm...> - 2011-07-12 13:18:10
|
On Tue, Jul 12, 2011 at 1:22 AM, JamesAng <ang...@gm...> wrote: > If we intend to use fdisk manually to understand the procedures, what are > the changes against the original wiki that we need? I prefer to use sfdisk. I'll leave fdisk instructions for someone else to figure out :-) > Ok, now I understand. > The first sector is MBR and the # of sector needed for padding depends on > the page size of the NAND structure. > > How do we determine the actual page size of the underlying NAND then? There really isn't an easy way to discover the exact structure of the card internals. Arnd recommends 128 as a safe value for the types of NAND currently in use. Steve |
From: maxbc <bon...@ho...> - 2012-04-05 22:39:41
|
I have been encountering similar issues, and am also quite stunned by the reaction of the cards : I have a 2GB SanDisk (staying at 4 heads, 16 sectors) sfdisk'd and it's booting fine. I have 2x4GB SDHC Emtec (it may be crap! I'm not an expert on flash cards) which also stay at 4 heads 16 sectors, but I get a read/write error while booting, when trying to access the linux partition. My own Ubuntu Desktop (the one that partitioned everything) does not recognize 'rootfs' after a plug-out/plug-in (the FAT partition works fine). What do you think? What's the best brand to buy not-even-duplicable-but-simply-working microSD? PS : the 5-year old 2GB non-SDHC card is faster than the 4GB SDHC... (both sticking around 10MB/s reading speed) -- View this message in context: http://gumstix.8.n6.nabble.com/trouble-with-should-be-bootable-SD-cards-and-kernel-versions-tp572969p4691046.html Sent from the Gumstix mailing list archive at Nabble.com. |