ruby-dbi-devel

[Ruby-DBI-devel] methods disappearing in ruby-mysql-2.4.2a

[Brad H. <bh...@vp...>]

Hello,

Sorry to cross-post to ruby-dbi-devel, but I wasn't sure which list
might be more appropriate for a problem I'm having.  In one of my
programs I am using ruby-dbi/ruby-dbd-mysql 0.0.13 and ruby-mysql 2.4.2a
on a FreeBSD 4.5 box running ruby 1.6.7 (2002-05-01).

I am inserting thousands of rows on a daily basis, and *sometimes* (I
can't reproduce the problem on demand) the script dies with messages
like these:

usr/local/lib/ruby/site_ruby/1.6/DBD/Mysql/Mysql.rb:321:in `finish':
undefined method `free' for #<Mysql:0x81a9d44> (NameError)
        from /usr/local/lib/ruby/site_ruby/1.6/dbi/dbi.rb:701:in
`finish'

or:

/usr/local/lib/ruby/site_ruby/1.6/DBD/Mysql/Mysql.rb:368:in
`column_info': undefined method `fetch_fields' for #<Mysql:0x81a8d44>
(NameError)
        from /usr/local/lib/ruby/site_ruby/1.6/dbi/dbi.rb:714:in
`column_names'
        from /usr/local/lib/ruby/site_ruby/1.6/dbi/dbi.rb:695:in
`execute'

-----------------------------

The strange thing is, the same exact command succeeds thousands of times
prior to this error.  How could the Mysql object not contain the methods
"fetch_fields" and "free"?  They are compiled in to the mysql.so file
which is necessary to create the Mysql object in the first place.  How
could those methods just disappear?

It is like the Mysql object had all of its methods removed somehow.

Does anyone have any suggestions?

Thanks,
-Brad

Re: [Ruby-DBI-devel] methods disappearing in ruby-mysql-2.4.2a

[Brad H. <bh...@vp...>]

On Fri, 2002-05-10 at 14:44, Brad Hilton wrote:
> usr/local/lib/ruby/site_ruby/1.6/DBD/Mysql/Mysql.rb:321:in `finish':
> undefined method `free' for #<Mysql:0x81a9d44> (NameError)
>         from /usr/local/lib/ruby/site_ruby/1.6/dbi/dbi.rb:701:in
> `finish'

Well, to follow up to myself:

It turns out the error is expected.  Mysql objects don't have the
methods 'free' and 'fetch_fields' defined.  Those methods are actually
in the class MysqlRes, which I found out after looking at the c code.

The problem is that ruby-dbd-mysql assumes that Mysql#query always
MysqlRes objects, when that isn't true.  If query_with_result is false,
Mysql#query returns self (a Mysql object).

Could the maintainer of ruby-dbd-mysql look into this please? 
Basically, every time a @res_handle is grabbed from Mysql#query there is
a false assumption that it is a MysqlRes object, and not a reference to
the original Mysql object.

Thanks,
-Brad

Re: [Ruby-DBI-devel] methods disappearing in ruby-mysql-2.4.2a

[Sean C. <se...@ch...>]

[moving discussion to rub...@li..., reply there]

> On Fri, 2002-05-10 at 14:44, Brad Hilton wrote:
> > usr/local/lib/ruby/site_ruby/1.6/DBD/Mysql/Mysql.rb:321:in `finish':
> > undefined method `free' for #<Mysql:0x81a9d44> (NameError)
> >         from /usr/local/lib/ruby/site_ruby/1.6/dbi/dbi.rb:701:in
> > `finish'
> 
> Well, to follow up to myself:
> 
> It turns out the error is expected.  Mysql objects don't have the
> methods 'free' and 'fetch_fields' defined.  Those methods are
> actually in the class MysqlRes, which I found out after looking at
> the c code.
> 
> The problem is that ruby-dbd-mysql assumes that Mysql#query always
> MysqlRes objects, when that isn't true.  If query_with_result is
> false, Mysql#query returns self (a Mysql object).

Hmmm....  alright.  What's the value of query_with_result being false?
Is it's point to save on the number of objects created?  I don't see
any direct benefit of toggling this option.

> Could the maintainer of ruby-dbd-mysql look into this please?
> Basically, every time a @res_handle is grabbed from Mysql#query
> there is a false assumption that it is a MysqlRes object, and not a
> reference to the original Mysql object.

I think it might be a fair assumption, but I'm neither the dbd_mysql
maintainer, nor am I familair with what that option does.  Could you
ellaborate?  -sc

-- 
Sean Chittenden

Re: [Ruby-DBI-devel] methods disappearing in ruby-mysql-2.4.2a

[Brad H. <bh...@vp...>]

On Sat, 2002-05-11 at 11:33, Sean Chittenden wrote:
> > The problem is that ruby-dbd-mysql assumes that Mysql#query always
> > MysqlRes objects, when that isn't true.  If query_with_result is
> > false, Mysql#query returns self (a Mysql object).
> 
> Hmmm....  alright.  What's the value of query_with_result being false?
> Is it's point to save on the number of objects created?  I don't see
> any direct benefit of toggling this option.

I don't know what the benefit of toggling query_with_result is either. 
I never directly manipulate the Mysql object, since I always use
ruby-dbi as a higher level access point.  I don't know what could
possibly be setting query_with_result to false, but by the mere fact
that it can happen, and thus break ruby-dbd-mysql, it seems that
something needs to be fixed.

> 
> > Could the maintainer of ruby-dbd-mysql look into this please?
> > Basically, every time a @res_handle is grabbed from Mysql#query
> > there is a false assumption that it is a MysqlRes object, and not a
> > reference to the original Mysql object.
> 
> I think it might be a fair assumption, but I'm neither the dbd_mysql
> maintainer, nor am I familair with what that option does.

Unfortunately, neither do I.  I just stumbled on the problem and thought
I'd report the invalid assumption that dbd-mysql is making about
Mysql#query.  Perhaps the code could be fixed by adding an addition
check.  Right now the could looks like:

@res_handle.free if @res_handle

Perhaps that needs to be changed to:

@res_handle.free if (@res_handle and @res_handle.is_a? MysqlRes)

I don't know...

-Brad

Re: [Ruby-DBI-devel] methods disappearing in ruby-mysql-2.4.2a

[Brad H. <bh...@vp...>]

On Sat, 2002-05-11 at 15:45, Brad Hilton wrote:
> I don't know what could possibly be setting query_with_result to false

Check that.  Mysql.rb sets query_with_result to false in line 210.  So
it needs to be prepared for Mysql#query returning a Mysql object in that
case.

-Brad

Re: [Ruby-DBI-devel] methods disappearing in ruby-mysql-2.4.2a

[Sean C. <se...@ch...>]

> > > Could the maintainer of ruby-dbd-mysql look into this please?
> > > Basically, every time a @res_handle is grabbed from Mysql#query
> > > there is a false assumption that it is a MysqlRes object, and not a
> > > reference to the original Mysql object.
> > 
> > I think it might be a fair assumption, but I'm neither the dbd_mysql
> > maintainer, nor am I familair with what that option does.
> 
> Unfortunately, neither do I.  I just stumbled on the problem and thought
> I'd report the invalid assumption that dbd-mysql is making about
> Mysql#query.  Perhaps the code could be fixed by adding an addition
> check.  Right now the could looks like:
> 
> @res_handle.free if @res_handle
> 
> Perhaps that needs to be changed to:
> 
> @res_handle.free if (@res_handle and @res_handle.is_a? MysqlRes)
> 
> I don't know...

I'm a PostgreSQL guy myself.  Could you test the above change and see
if it works for you?  You seem to have done the diagnosis correctly
and I think it should work.  -sc

-- 
Sean Chittenden

Re: [Ruby-DBI-devel] methods disappearing in ruby-mysql-2.4.2a

[Brad H. <bh...@vp...>]

On Sun, 2002-05-12 at 21:20, Sean Chittenden wrote:
> > Right now the could looks like:
> > 
> > @res_handle.free if @res_handle
> > 
> > Perhaps that needs to be changed to:
> > 
> > @res_handle.free if (@res_handle and @res_handle.is_a? MysqlRes)
> > 
> > I don't know...
> 
> I'm a PostgreSQL guy myself.  Could you test the above change and see
> if it works for you?  You seem to have done the diagnosis correctly
> and I think it should work.  -sc

Actually, I think a better solution than my previous one is to set
query_with_result = true in Mysql.rb line 313 in execute().  It is
obvious that the code expects query_with_result to be set to true in
that instance, and there are many places in the code that assume
@res_handle is a MysqlRes object, so I believe this would be the best
solution.

Attached is a sample script which exposes the problem, as well as a
patch to fix the bug.

Thanks,
-Brad

-- patch --

--- Mysql.rb.orig       Mon May 13 11:32:33 2002
+++ Mysql.rb    Mon May 13 09:48:52 2002
@@ -310,6 +310,7 @@ class Statement < DBI::BaseStatement
   end
 
   def execute
+         @handle.query_with_result = true
     @res_handle = @handle.query(@prep_stmt.bind(@params))
     @current_row = 0
     @rows = @handle.affected_rows

Re: [Ruby-DBI-devel] methods disappearing in ruby-mysql-2.4.2a

[Michael N. <uu...@rz...>]

On Mon, May 13, 2002 at 11:35:54AM -0700, Brad Hilton wrote:
> On Sun, 2002-05-12 at 21:20, Sean Chittenden wrote:
> > > Right now the could looks like:
> > > 
> > > @res_handle.free if @res_handle
> > > 
> > > Perhaps that needs to be changed to:
> > > 
> > > @res_handle.free if (@res_handle and @res_handle.is_a? MysqlRes)
> > > 
> > > I don't know...
> > 
> > I'm a PostgreSQL guy myself.  Could you test the above change and see
> > if it works for you?  You seem to have done the diagnosis correctly
> > and I think it should work.  -sc
> 
> Actually, I think a better solution than my previous one is to set
> query_with_result = true in Mysql.rb line 313 in execute().  It is
> obvious that the code expects query_with_result to be set to true in
> that instance, and there are many places in the code that assume
> @res_handle is a MysqlRes object, so I believe this would be the best
> solution.
> 
> Attached is a sample script which exposes the problem, as well as a
> patch to fix the bug.

This will not completely fix the bug, I fear. 

When executing statements concurrently, it could happen that method #do
sets query_with_result=false, then #execute interrupts and sets 
query_with_result=true ....

The solution is to use a Mutex, which prevents parallel execution of 
methods #do and #execute.

The following patch fixes it (checked into -current):

----- patch ------


5c5
< # Version : 0.3.1
---
> # Version : 0.3.2
24a25
> require "thread"   # for Mutex
30c31
< VERSION          = "0.3.1"
---
> VERSION          = "0.3.2"
148a150,154
>   def initialize(handle, attr)
>     super
>     @mutex = Mutex.new 
>   end
> 
208d213
< 
210d214
<     @handle.query_with_result = false
212,213c216,220
<     @handle.query(sql)
<     @handle.affected_rows
---
>     @mutex.synchronize { 
>       @handle.query_with_result = false
>       @handle.query(sql)
>       @handle.affected_rows     # return value
>     }
220c227
<     Statement.new(self, @handle, statement)
---
>     Statement.new(self, @handle, statement, @mutex)
297c304
<   def initialize(parent, handle, statement)
---
>   def initialize(parent, handle, statement, mutex)
300c307
<     @parent, @handle = parent, handle
---
>     @parent, @handle, @mutex = parent, handle, mutex
303d309
<     @handle.query_with_result = true # automatically switches store_result on 
313,315c319,325
<     @res_handle = @handle.query(@prep_stmt.bind(@params))
<     @current_row = 0
<     @rows = @handle.affected_rows
---
>     sql = @prep_stmt.bind(@params)
>     @mutex.synchronize {
>       @handle.query_with_result = true
>       @res_handle = @handle.query(sql)
>       @current_row = 0
>       @rows = @handle.affected_rows
>     }








-- 
Michael Neumann  ***  eMail uu...@rz...