Re: [Aoetools-discuss] Introduction
Brought to you by:
ecashin,
elcapitansam
From: kelsey h. <kh...@dr...> - 2008-07-10 19:56:17
|
Hi Phil, > I'm sorry to sound like a whiner, but this is still not working. You're not. It took me several weeks to figure out what was going on. > Relevant hardware: * 750GB SATAII drive @ 3Gb/s attached to an Adaptec > 31605 RAID controller > * I actually have 15 disks hooked up on this, but I'm using just the one > disk for this experiment. It is currently identified as sdc on the > target machine > > Software: * RHEL5.2_x64 - up to date as of today. > * qaoed v0.81 - one single export (/dev/sdc) Let me see your config file. > * latest (v62) aoe drivers from coraid > * aoe-tools v26 Does the kernel on the initiator report the correct frame size? Should be 8000-some byte frames. If you're not using jumbo frames it'll never be aligned because the write()s issued won't ever be a multiple of the page size :) > 3) ran fdisk exactly as you specified on /dev/etherd/e1.1 Your fdisk output should look something like this. Does it? Command (m for help): p Disk /dev/etherd/e522.0: 38.6 GB, 38654705664 bytes 248 heads, 56 sectors/track, 5436 cylinders Units = cylinders of 13888 * 512 = 7110656 bytes Device Boot Start End Blocks Id System /dev/etherd/e522.0p1 * 1 5436 37747556 8e Linux LVM > 4) ran 'mkfs.ext2 /dev/etherd/e1.1p1' > --> While I was doing this, I used iostat to watch the disk on the > target machine. the mkfs got up to about 415 sectors into the format > fantasticly, but then fell in a screaming heap after that. The same > problem seemed to be occuring. mkfs does a lot of reads. Especially with ext3. It's not the most efficient filesystem out there by any stretch of the imagination. > 6) copied a file to the newly mounted partiton and watched the > reads/writes with iostat on the target machine Try using: dd if=/dev/zero of=/mnt/1/somefile count=4096 bs=1M that will write 4G of zeros to the disk. > As far as I can tell, I'm doing it right... It just doesn't seem to > want to work... Unless of course, you can see something I'm doing > wrong? Admittedly, I WAS partitioning it on the target before, but I've > done it all as you specified this time. Nothing I see glaring at me. This is how my systems are typically set up. target system: disk (be it raid or single-spindle) single partition type 8e LVM PV LVM VG with 4Mbyte extents LVM LVs created for each host LVM LVs exported via qaoed Initiator system: AoE target disk single partition with altered geometry LVM VG with 4Mbyte extents LVM LVs created for all additional partitions So yeah, this could be said I'm running LVM over AoE over LVM over hardware RAID. Mind-boggling, when you really think about it :\ > Is there a way to find out where the partition table actually starts and > work it out from that? After reading your writeup about the alignment > problem, I wondered if there was something I could do to see if this was > some sort of weird, one-off not-like-every-one-else misalignment. Like > maybe the OS is different from the norm and using a larger or smaller > cache size somewhere... What OS are you using with yours? Sure -- bring up /dev/sdc in your favorite hex editor. Look for the partition table end signature (i don't remember this off the top of my head but it's in the PDF) and back up by 510 bytes. The offset there will be the offset at which the partition table starts on the disk. In my experience, however, this has always started at sector 0 irrespective of what media I'm using (whether it be a bare disk or an LVM LV, etc). > I am totally at a loss here for what to do... I could always give you > ssh access to the two machines I'm using and let you see it for > yourself... :) sure. here's a password hash. $1$yMm6RX8M$gpZEJKjQ2x4hngT/PoT0D0 > I have a suspicion it might be the SAS/RAID controller card that is the > problem, so I was thinking of trying it with a drive connected to the > onboard SATA port. What do you think? If it's a matter of software It likely won't make much difference. > versions, I'd be more than happy to install something (anything!) else > to see how it goes. I am currently downloading Plan9 from Bell Labs to > see what it does... I've heard that Coraid uses plan9 on their SR > units, so I wondered if it'd be possible to put that on the hardware and They do -- as a matter of fact sitting on my desk in front of me I have a Disk-On-Module with a copy of Coraid's SR OS from one of the rebranded supermicro boxes with the 15 disks across the front. I never ended up using it, though, because the SR software doesn't allow me to fully virtualize the storage. That's important for my application :) I'll forewarn you though: plan9 has horrible hardware support, largely because nobody uses it. Coraid was the first outside Bell Labs I've seen that use it in production :) Say Hi to Glenda for me. > extract an SR upgrade firmware over it to see what happens... I doubt > it'd be any good, but hey, you never know. :) I'll try anything at his > point! I'd be surprised if your disk array gets detected at all, to be honest. But good luck with that! In Coraid's hardware, they use Supermicro AOC-SAT2-MV8 boards to control the disks. These are, incedentally, poorly supported by Linux, but supported just fine in Coraid's plan9 image. > Any advice greatly appreciated, Let me know how it turns out. Cheers, -Kelsey |