Re: [Perfparse-devel] libdbi

[Ben C. <Be...@cl...>]

Thanks Yves.

Got to argue with you that the CGI speed is not so important. :)

I cannot get more than about 20000 data samples out of perfparse because 
of speed problems :)  Which is about a months data.  Speed is vitally 
important here, and anything which limits this will be an unwelcome 
addition.  I want to aim to be able to get 1 year time averages :)

I made a comment about 'cursors'.  MySQL does not have a binary cursor, 
which really effects performance.  PostgreSQL, Oracle and others do.  On 
the other hand, MySQL has a 'read-forward' cursor which adds great 
performance, which PostgreSQL does not have.  I wonder whether libdbi 
can return the fastest cursor for the application, or defaults to a 
ASCII non-read-forward (slow) cursor.

I also note that libdbi is not yet version 1.0.0.

PerfParse has it's own very limited API, not always used, to abstract 
away from MySQL.  See 'common.h'

My own feeling would be that this should be developed to cater for all 
DBMSs, using the best cursor available, keeping everything in-house :)

Ben




Yves Mettier wrote:

>>I have never heard of libdbi, I'll have to take a look.  We do want
>>support for PostgreSQL, Fireblade, Interbase and others.  If this lib
>>offers an easy way to do this, then all the better.  But I do echo Yves
>>comments that it may be better for the product to use the basic c API
>>from Oracle.  You will be able to take advantage of the abilities of the
>>API's which a generic API might not have.  Stability, speed, special
>>cursors etc.
> 
> 
> I'd like to explain some of my comments:
> - for perfparse, aka the parser, we need speed, and we need to be sure that our
> dependancies will be maintained forever (at least as longer as perfparse :)
> If we discover a bug in a dependancy and there is nobody to maintain that dependancy, we
> have to drop the dependancy. Here, I wonder if libdbi is a good choice. And perfparse is
> a critical tool because if the user does not run it often, he can loose data in a too
> big log file.
> 
> - for the database tools and graph tools, we do not need that much speed. And if a bug
> appear in a dependancy, the users can stop their tools for a while, before we fix the
> bug or drop the dependancy. Those tools are less critical.
> 
> - we need some abstract layer for databases to port perfparse on other databases,
> whatever the layer is. It is often better not to reinvent the wheel when it exists. That
> is the reason I like the idea to give a try to libdbi. I just regreet there is no
> support for sybase. Chris, if you work with libdbi, maybe I'll try to contribute to
> libdbi for the sybase port, in order to have the perfparse tools also work on sybase !
> 
> Well, for now, just wait 0.0.11. I expect that Ben and me manage to release it today :)
> 
> Yves
>