Thread: Re: [Aoetools-discuss] Multiple vblade's and performance?
Brought to you by:
ecashin,
elcapitansam
From: Don H. <Don...@pm...> - 2007-02-22 19:54:39
|
-----Original Message----- >My guess it that the activation of all processes has to do with the AoE >protocol: they probably all have to listen to all incoming traffic just to pick out the traffic for that >specific blade. Yes, if you look at the packet receive routine you will notice that there is a block of code that basically checks to see if the packet is AoE and has a matching shelf/slot otherwise it is discarded. >all up (if that is at all possible). As far as I see, there is no way to have one single > vblade host multiple and separate devices? This is also true, look in main() and you will see that a single block device is opened. Someone has created a fork that can serve up multiple block devices in a single vblade process, check the Aoetools archives. Cheers, don |
From: <de...@we...> - 2007-02-24 11:40:37
|
> On Sat, 24 Feb 2007, Bernard Li wrote: >=20 > > > - Aoeserver > > > http://pi.nxs.se/~wowie/aoeserver/ > > I tried to compile this on CentOS 4.4, but it didn't go - I assume I > > can only build this on a kernel with the AoE headers=3F >=20 > It might be that they have changed something in the kernel since I wrote= > that software. All modern kernels have the aoe-headers, but you dont wan= t > to have the aoe-module loaded at the same time as aoeserver. the kernel development model doesn`t just "fit" on maintaining modules ext= ernally, since the kernel api is changing very often. so - if you get some module for some kernel, it´s very likely that it= will not compile with more recent kernel and the developer needs constant= ly=20 adjust that module to fit to recent kernels. if the module is in mainline,= the kernel developers automatically do that job. what`s the error message you get=3F regards roland > -----Urspr=FCngliche Nachricht----- > Von: Torbj=F6rn Pettersson <wo...@pi...> > Gesendet: 24.02.07 12:03:36 > An: Bernard Li <be...@va...> > CC: Aoe...@li... > Betreff: Re: [Aoetools-discuss] Multiple vblade's and performance=3F >=20 > On Sat, 24 Feb 2007, Bernard Li wrote: >=20 > > > - Aoeserver > > > http://pi.nxs.se/~wowie/aoeserver/ > > I tried to compile this on CentOS 4.4, but it didn't go - I assume I > > can only build this on a kernel with the AoE headers=3F >=20 > It might be that they have changed something in the kernel since I wrote= > that software. All modern kernels have the aoe-headers, but you dont wan= t > to have the aoe-module loaded at the same time as aoeserver. >=20 > > > - Qaoed > > > http://pi.nxs.se/~wowie/qaoed.tgz > > > > This built on CentOS 4.4, however I could not setup any targets. > > Which Linux distributions have you tested this under=3F >=20 > debian >=20 > Kind regards > Torbjorn >=20 >=20 >=20 > ------------------------------------------------------------------------= - > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share = your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php=3Fpage=3Djoin.php&p=3Dsourceforge&CID=3DDEVDE= V > =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F > Aoetools-discuss mailing list > Aoe...@li... > https://lists.sourceforge.net/lists/listinfo/aoetools-discuss >=20 =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F Viren-Scan f=FCr Ihren PC! Jetzt f=FCr jeden. Sofort, online und kostenlos. Gleich testen! http://www.pc-sicherheit.web.de/freescan/=3Fmc=3D022222 |
From: Bernard L. <be...@va...> - 2007-02-24 19:39:09
|
Actually, I was compiling on CentOS 4.4, which is a RHEL4 clone and it is using a severely patched version of 2.6.9 kernel, which does not contain the AoE headers. I can compile (k)vblade fine on this system though. Cheers, Bernard On 2/24/07, de...@we... <de...@we...> wrote: > > > On Sat, 24 Feb 2007, Bernard Li wrote: > > > > > > - Aoeserver > > > > http://pi.nxs.se/~wowie/aoeserver/ > > > I tried to compile this on CentOS 4.4, but it didn't go - I assume I > > > can only build this on a kernel with the AoE headers? > > > > It might be that they have changed something in the kernel since I wrot= e > > that software. All modern kernels have the aoe-headers, but you dont wa= nt > > to have the aoe-module loaded at the same time as aoeserver. > > the kernel development model doesn`t just "fit" on maintaining modules ex= ternally, since the kernel api is changing very often. > so - if you get some module for some kernel, it´s very likely that i= t will not compile with more recent kernel and the developer needs constant= ly > adjust that module to fit to recent kernels. if the module is in mainline= , the kernel developers automatically do that job. > > what`s the error message you get? > > regards > roland > > > > -----Urspr=FCngliche Nachricht----- > > Von: Torbj=F6rn Pettersson <wo...@pi...> > > Gesendet: 24.02.07 12:03:36 > > An: Bernard Li <be...@va...> > > CC: Aoe...@li... > > Betreff: Re: [Aoetools-discuss] Multiple vblade's and performance? > > > > > > On Sat, 24 Feb 2007, Bernard Li wrote: > > > > > > - Aoeserver > > > > http://pi.nxs.se/~wowie/aoeserver/ > > > I tried to compile this on CentOS 4.4, but it didn't go - I assume I > > > can only build this on a kernel with the AoE headers? > > > > It might be that they have changed something in the kernel since I wrot= e > > that software. All modern kernels have the aoe-headers, but you dont wa= nt > > to have the aoe-module loaded at the same time as aoeserver. > > > > > > - Qaoed > > > > http://pi.nxs.se/~wowie/qaoed.tgz > > > > > > This built on CentOS 4.4, however I could not setup any targets. > > > Which Linux distributions have you tested this under? > > > > debian > > > > Kind regards > > Torbjorn > > > > > > > > -----------------------------------------------------------------------= -- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share= your > > opinions on IT & business topics through brief surveys-and earn cash > > http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID= =3DDEVDEV > > _______________________________________________ > > Aoetools-discuss mailing list > > Aoe...@li... > > https://lists.sourceforge.net/lists/listinfo/aoetools-discuss > > > > > _______________________________________________________________________ > Viren-Scan f=FCr Ihren PC! Jetzt f=FCr jeden. Sofort, online und kostenlo= s. > Gleich testen! http://www.pc-sicherheit.web.de/freescan/?mc=3D022222 > > |
From: Ed L. C. <ec...@co...> - 2007-02-22 20:35:15
|
On Thu, Feb 22, 2007 at 11:54:07AM -0800, Don Hiatt wrote: ... > This is also true, look in main() and you will see that a single block device is > opened. Someone has created a fork that can serve up multiple block devices in a > single vblade process, check the Aoetools archives. If I recall correctly, the kvblade also does that. -- Ed L Cashin <ec...@co...> |
From: Arnold H. <a.h...@b-...> - 2007-02-23 09:39:52
|
Ed L. Cashin wrote: > On Thu, Feb 22, 2007 at 11:54:07AM -0800, Don Hiatt wrote: > ... > >> This is also true, look in main() and you will see that a single block device is >> opened. Someone has created a fork that can serve up multiple block devices in a >> single vblade process, check the Aoetools archives. >> > > If I recall correctly, the kvblade also does that. > I think kvblade is still marked as alpha? Althoughg even then, I personally prefer a userspace solution. I looked through the archives, and found a few messages where people referred to solutions they were working on, but I couldn't find anything that appeared to be actually under development. Can you give me a pointer ? Also, is there a specific reason for the 'main' vblade not supporting multiple devices, or is it just a case of 'noone needed it so far' ? TIA, Arnold Hendriks -- Arnold Hendriks <a.h...@b-...> B-Lex Information Technologies <http://www.b-lex.com/> Postbus 545, 7500 AM Enschede, The Netherlands B-Lex: +31 (0)53 4836543 Mobile: +31 (0)6 51710159 MSN: a.h...@b-... ICQ: 86313731 |
From: <wo...@pi...> - 2007-02-23 20:54:12
|
On Fri, 23 Feb 2007, Arnold Hendriks wrote: > I looked through the archives, and found a few messages where people > referred to solutions they were working on, but I couldn't find anything > that appeared to be actually under development. Can you give me a > pointer ? I wrote a multithreaded aoe storage-target that is able to handle multiple storage targets and multiple network interfaces cleanly. However, I gave up on the development due to lack of user interest. If anyone wants to take over any of these two projects, feel free :) - Aoeserver An in kernel storage target for Linux, like kvblade but buffered. Can handle multiple targets. Last update: Jan 2, 2006 http://pi.nxs.se/~wowie/aoeserver/ - Qaoed A portable, multithreaded userland version of aoeserver that runts one thread for each storage target and one thread for each interface. Has a nice configuration file format, supports reloading the configuration in runtime. Last update: Nov 1, 2006 http://pi.nxs.se/~wowie/qaoed.tgz Good luck friends :) |
From: Bernard L. <be...@va...> - 2007-02-24 08:21:00
|
Hi Torbj=F6rn: On 2/23/07, Torbj=F6rn Pettersson <wo...@pi...> wrote: > - Aoeserver > An in kernel storage target for Linux, like kvblade but buffered. Can > handle multiple targets. Last update: Jan 2, 2006 > http://pi.nxs.se/~wowie/aoeserver/ I tried to compile this on CentOS 4.4, but it didn't go - I assume I can only build this on a kernel with the AoE headers? > - Qaoed > A portable, multithreaded userland version of aoeserver that runts one > thread for each storage target and one thread for each interface. > Has a nice configuration file format, supports reloading the > configuration in runtime. Last update: Nov 1, 2006 > http://pi.nxs.se/~wowie/qaoed.tgz This built on CentOS 4.4, however I could not setup any targets. Which Linux distributions have you tested this under? Thanks, Bernard |
From: <wo...@pi...> - 2007-02-24 11:03:07
|
On Sat, 24 Feb 2007, Bernard Li wrote: > > - Aoeserver > > http://pi.nxs.se/~wowie/aoeserver/ > I tried to compile this on CentOS 4.4, but it didn't go - I assume I > can only build this on a kernel with the AoE headers? It might be that they have changed something in the kernel since I wrote that software. All modern kernels have the aoe-headers, but you dont want to have the aoe-module loaded at the same time as aoeserver. > > - Qaoed > > http://pi.nxs.se/~wowie/qaoed.tgz > > This built on CentOS 4.4, however I could not setup any targets. > Which Linux distributions have you tested this under? debian Kind regards Torbjorn |
From: Bernard L. <be...@va...> - 2007-02-27 03:31:03
|
Hi Torbj=F6rn: On 2/24/07, Torbj=F6rn Pettersson <wo...@pi...> wrote: > It might be that they have changed something in the kernel since I wrote > that software. All modern kernels have the aoe-headers, but you dont want > to have the aoe-module loaded at the same time as aoeserver. As you recommended, I added the line: #define ETH_P_AOE (0x88a2) to the file aoeserver/linux/drivers/block/aoeserver/aoe.h and it compiled f= ine. However, if I run the following script to setup aoeserver, I get kernel pan= ic: BLADES=3D10 for i in `seq 1 $BLADES`; do echo add /dev/loop$i $i 0 eth1 > /proc/aoeserver done I believe adding "sleep" in the loop would alleviate the problem. In IRC you mentioned that I can export the file/partition directly. Currently I run dd to create a file and then export it via loop devices, are you saying that I can bypass that? What is the syntax? Thanks, Bernard |
From: <wo...@pi...> - 2007-02-27 06:33:42
|
On Mon, 26 Feb 2007, Bernard Li wrote: > However, if I run the following script to setup aoeserver, I get kernel > panic: > > BLADES=10 > > for i in `seq 1 $BLADES`; do > echo add /dev/loop$i $i 0 eth1 > /proc/aoeserver > done Now that just sucks, doesnt it? :( Despite all those kernel locks, all the books, all the code and everything it turns out that im just a crappy kernel-module programmer :) Ill try to look into it and see if I can figure out whats going on, some kind of race-condition I guess. > In IRC you mentioned that I can export the file/partition directly. > What is the syntax? The same as for a device: "echo add /root/mylargefile.ext2 0 3 eth1 > /proc/aoeserver" Kind regards Torbjorn |
From: Ed L. C. <ec...@co...> - 2007-02-23 22:31:22
|
On Fri, Feb 23, 2007 at 10:39:46AM +0100, Arnold Hendriks wrote: ... > I think kvblade is still marked as alpha? Althoughg even then, I > personally prefer a userspace solution. > > I looked through the archives, and found a few messages where people > referred to solutions they were working on, but I couldn't find anything > that appeared to be actually under development. Can you give me a pointer ? My impression was that kvblade worked pretty well (given certain conditions mentioned in the README) and had developmentally gotten to a good "stopping point" when Sam put it up at sourceforge.net. I believe his motivation was to make what he had available to everyone as an open source project. But you're right that so far nobody has taken up kvblade and run with it. Perhaps it's usable enough as is? I can't say. > Also, is there a specific reason for the 'main' vblade not supporting > multiple devices, or is it just a case of 'noone needed it so far' ? The vblade itself is designed to be a simple but working AoE target, so new features aren't really encouraged. Putting one target in one process keeps the implementation minimal. It's open source, though, so anybody could create a "vblades" or something, and as you've noticed, some folks have done work like that. -- Ed L Cashin <ec...@co...> |