aoetools-discuss

[Aoetools-discuss] Interested in enhancing vblade

[Anthony W. <an...@co...>]

I'm interested in doing some work on vblade, and want to get a feel of 
whether the work would be accepted into the main vblade tree. The reason 
I ask is that it's a fairly substantial piece of work which would 
involve code restructuring, possibly to the point where it's 
substantially different from what it is now, and making such big changes 
has a strong tendancy to upset an existing maintainer.

Let me explain what I'm interested in doing, and why, which I hope will 
better inform your reply.

I spent yesterday doing performance analysis of linux block device 
read/writing, raw ethernet message passing, the aoeclient and the vblade 
server. From this I believe that with 1060 byte ethernet frames I can 
increase the speed with which vblade can read and write by 5 times or 
greater. This is all from within user space, and I doubt an optimised 
kvblade could do much better. I also discovered that again with 1060 
byte ethernet frames there's the potential to increase the network 
performance of the aoeclient by around 25%, though since it's generally 
going to be limited by the disk's performance rather than the network's 
performance, this is probably less important.

Since I believe I can get the disk performance up to a good level it 
means that the user space vblade is worth investing effort in to make it 
usable in a production environment. However there are a number of things 
that need to be fixed to use it in production.....

    Implement direct I/O
            - it buffers at the moment, and this isn't good for RAID
    Proper error handling
            - it's fairly weak on detecting and dealing with run-time errors
    Increase read/write performance
            - I believe I can increase this at least 5 times
    Implement vblade.conf
            - support multiple interfaces & devices in a single server
    Jumbo Frames
            - support the aoeclient's jumbo frames for higher network 
throughput
    Syslog
            - Send messages directly to syslog
    Clean the Code
            - replace magic numbers with #defined ones
            - use meaningful variable names
            - break up the code into smaller logical blocks in functions
    Restructure Code
            - move functions around, create new .c & .h files
            - needed for vblade.conf & performance changes

The most important part of the change is to keep vblade functionally the 
same, and the aoeclient should function as normal without any changes. 
If vblade's implementation seems to be odd or differs from the 
specification I'll go with vblade's implementation and flag up the issue 
on the mailing list.

The most controversial points of my changes will be the last two in the 
list above - I know how I feel about people playing with my code, but I 
can't really make the other changes without doing these at the same time.

As I said I'm interested in making these changes, and believe I can do 
them fairly quickly, but I'm really not at all keen on the idea of 
forking the project. I've seen enough forked projects that haven't been 
maintained for years and really would rather not risk going down that road.

Thanks,

Tony.

Re: [Aoetools-discuss] Interested in enhancing vblade

[Sam H. <sa...@co...>]

> I'm interested in doing some work on vblade, and want to get a feel of 
> whether the work would be accepted into the main vblade tree. The reason 
> I ask is that it's a fairly substantial piece of work which would 
> involve code restructuring, possibly to the point where it's 
> substantially different from what it is now, and making such big changes 
> has a strong tendancy to upset an existing maintainer.

In general, it's unlikely anyone will agree to give you carte blanche
in rewriting code they must maintain.  It's always nice to play in
someone's sandbox for a while before you declare their implementation
mortally limited.

That aside, I'd like to discuss what, specifically, you think you can
accomplish and, specifically, why the existing framework is
insufficient.  Claiming to have run tests and making wide claims about
what can be done if only you can rewrite everything does not instill
much confidence.

>     Implement direct I/O
>             - it buffers at the moment, and this isn't good for RAID

A patch has been submitted to provide this and last I heard, it's
under review.

>     Proper error handling
>             - it's fairly weak on detecting and dealing with run-time errors

specifically?

>     Increase read/write performance
>             - I believe I can increase this at least 5 times

specifically?

>     Implement vblade.conf
>             - support multiple interfaces & devices in a single server

I disdain config files until they're absolutely necessary and I don't
think we're there.  Using a separate process for each vblade is cheap,
and easy.  If you want to maintain what gets started, write a script.

>     Jumbo Frames
>             - support the aoeclient's jumbo frames for higher network 
> throughput

I have a modified vblade-10 that supports jumbo, but for some reason
the kernel is dropping inbound frames.  You're welcome to hack on it
and work out the issue, if you like.

>     Syslog
>             - Send messages directly to syslog

That seems reasonable, if users want that.

>     Clean the Code
>             - replace magic numbers with #defined ones
>             - use meaningful variable names
>             - break up the code into smaller logical blocks in functions

You're stepping into style territory.  Please be specific about
perceived inadequacies.

>     Restructure Code
>             - move functions around, create new .c & .h files
>             - needed for vblade.conf & performance changes

...

> As I said I'm interested in making these changes, and believe I can do 
> them fairly quickly, but I'm really not at all keen on the idea of 
> forking the project. I've seen enough forked projects that haven't been 
> maintained for years and really would rather not risk going down that road.

Forking the project is often the cleanest way to accomplish what you
want if our sandbox isn't the right fit for you.  Then when you've a
finished sandbox you can show why yours is superior.  In reality we're
not talking about much code.  The whole package is only 854 lines and
of that 315 is the freebsd support.

Cheers,

Sam

Re: [Aoetools-discuss] Interested in enhancing vblade

[Ed L. C. <ec...@co...>]

On Tue, Sep 12, 2006 at 12:53:53PM -0400, Sam Hopkins wrote:
...
> I disdain config files until they're absolutely necessary and I don't
> think we're there.  Using a separate process for each vblade is cheap,
> and easy.  If you want to maintain what gets started, write a script.

Yes.  For modularity, there could be a separate vblade manager that
would read a config file and launch vblades using vbladed.  I think
people don't notice vbladed, or they forget about it.  For example,

...
> >     Syslog
> >             - Send messages directly to syslog
> 
> That seems reasonable, if users want that.

The vbladed does that.  Perhaps if there was a manpage for vbladed it
would help it to get noticed.

> >     Clean the Code
> >             - replace magic numbers with #defined ones
> >             - use meaningful variable names
> >             - break up the code into smaller logical blocks in functions
> 
> You're stepping into style territory.  Please be specific about
> perceived inadequacies.

Everyone can agree that a change that fixes a bug is something that
should go into the vblade.  New features are less clear cut, but style
changes are probably just not worth the bother unless they are truly
and obviously beneficial to the maintainability and correctness of the
code.

There's much to be said for the current style.  It's not exactly my
style, but I appreciate the terseness and consistency.  In my opinion,
it is best for contributers (myself included) to make changes in
keeping with the vblade's existing style.

-- 
  Ed L Cashin <ec...@co...>