sqlobject-discuss — All things SQLObject-related

Re: [SQLObject] per-instance connection

[John B. <bre...@mc...>]

> On Wed, Feb 03, 2010 at 01:37:08PM -1000, John Bresnahan wrote:
>> 2) at what point is an object associated with a DB connection?
>> 3) is it at all possible to change the DB connection associated with a given object?
> 
>    Objects (SQLObject instances that correspond to rows in a table) are
> seldom associated with a connection (though it is certainly possible).
>    SQLObject classes (that correspond to tables) are usually associated with
> a connection; .setConnection() is a class method.
>    To associate an object with a connection you have to SELECT it via that
> connection:
> 
>> trans = sqlhub.processConnection.transaction()
> 
> and now do
> 
>    xfer = XferTable.get(id, connection=trans)
> or XferTable.select(condition, connection=trans).
>    Most SQLObject methods accept 'connection' parameter.

Thanks much!  Is there a way to change the connection associated with an object you have already 
acquired via select?

Re: [SQLObject] per-instance connection (was: setConnection)

[Oleg B. <ph...@ph...>]

On Wed, Feb 03, 2010 at 01:37:08PM -1000, John Bresnahan wrote:
> 2) at what point is an object associated with a DB connection?
> 3) is it at all possible to change the DB connection associated with a given object?

   Objects (SQLObject instances that correspond to rows in a table) are
seldom associated with a connection (though it is certainly possible).
   SQLObject classes (that correspond to tables) are usually associated with
a connection; .setConnection() is a class method.
   To associate an object with a connection you have to SELECT it via that
connection:

> trans = sqlhub.processConnection.transaction()

and now do

   xfer = XferTable.get(id, connection=trans)
or XferTable.select(condition, connection=trans).
   Most SQLObject methods accept 'connection' parameter.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

[SQLObject] setConnection()

[John B. <bre...@mc...>]

I am trying to change the connection with which a given object is associate for some needed 
transactional behavior.  In an API function a user hands me an SQLObject called xfer.  I want to 
update a value in that object and automically update the value of a row in another table.

To achieve this I created a transaction with

trans = sqlhub.processConnection.transaction()

and then i call xfer.setConnection(trans).

This does not seem to have the effects I assumed so I have a few questions:

1) Is there a better way to achieve the results I am looking for?

2) at what point is an object associated with a DB connection?

3) is it at all possible to change the DB connection associated with a given object?

Any help is greatly appreciated.

Thanks,

John

Re: [SQLObject] ForeignKey and MultipleJoin on legacy (existing) database

[Petr J. <pet...@tp...>]

> > I am trying to connect via SQLObject to the existing database
>
>    You didn't mention fromDatabase but I am sure that what you meant,
> right? Because it's certainly possible to connect to an existing database
> without it.
>
> > What I really can not figure out is how to get ForeignKey and
> MultipleJoin
> > working.
>
>    By declaring them yourself:
>
> class MyTable(SQLObject):
>    class sqlmeta:
>        fromDatabase = True
>    fk = ForeignKey('AnotherTable')
>    mj = MultipleJoin('AnotherTable', joinColumn='at_id')


Thanks for the syntax, it works great.

>
>   Explicit declarations are compatible with fromDatabase. And of course
> you are not obliged to use fromDatabase at all - just declare all you
> columns and joins yourself.
>

I have used fromDatabase  because my lazybones only.

  Of course IWBN if your patch fromDatabase machinery (methods
> columnsFromSchema and guessClass in Connection class) to recognize at least
> ForeignKeys.
>
> IWBN, I would like to. But, to be honest, I think my Pythonic skills are to
low for this task (it is REALLY not about the time or laziness).

Thanks and Best Regards

Petr

Re: [SQLObject] memcache

[Oleg B. <ph...@ph...>]

On Mon, Feb 01, 2010 at 01:40:51PM -0500, Stef Telford wrote:
>      *) subclass the SQLObject into another class (which I called 
> CompassObject, since our reservation system is called Compass ;) and 
> overloaded _SO_setValue to do a delete inside memcache on change and 

   To delete the instance from the cache to make the cache coherent?

> overloaded 'get' such that it looks up memcache. I have to del/create 

   I am going to do it in steps. The first step - to make SQLObject
instances pickleable. MemcachedCache would be the next. But I am going to
develop a separate cache class, not hack into .get machinery.

> _SO_writeLock, sqlmeta and _connection on each object when it's returned.

   _connection? Do you mean per-instance connection? In general, the
connection is stored in the SQLObject table class and it doesn't need to be
pickled or restored. But how do you restore per-instance connection?

>      Also, in a totally random thought, you will probably hit the 1mb 
> memcache limit when storing objects.

   What's the limit?

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

Re: [SQLObject] ForeignKey and MultipleJoin on legacy (existing) database

[Oleg B. <ph...@ph...>]

On Mon, Feb 01, 2010 at 08:40:48PM +0100, Petr Jake?? wrote:
> I am trying to connect via SQLObject to the existing database

   You didn't mention fromDatabase but I am sure that what you meant,
right? Because it's certainly possible to connect to an existing database
without it.

> What I really can not figure out is how to get ForeignKey and MultipleJoin
> working.

   By declaring them yourself:

class MyTable(SQLObject):
    class sqlmeta:
        fromDatabase = True
    fk = ForeignKey('AnotherTable')
    mj = MultipleJoin('AnotherTable', joinColumn='at_id')

   Explicit declarations are compatible with fromDatabase. And of course
you are not obliged to use fromDatabase at all - just declare all you
columns and joins yourself.
   Of course IWBN if your patch fromDatabase machinery (methods
columnsFromSchema and guessClass in Connection class) to recognize at least
ForeignKeys.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

[SQLObject] ForeignKey and MultipleJoin on legacy (existing) database

[Petr J. <pet...@tp...>]

Hi,
I am trying to connect via SQLObject to the existing database (MySQL -
Prestashop database)

The database has different naming style than SQLObject expect.

I know it is possible to use

class sqlmeta:
        idName = 'id_order'

to get primary key working properly.

What I really can not figure out is how to get ForeignKey and MultipleJoin
working.

Thanks for your comments

Petr Jakes

Re: [SQLObject] memcache

[Stef T. <st...@um...>]

Morning Oleg, Everyone,
     I found that to get memcache working, I had do two things;

     *) get rid of the cache writelocks inside main.py (eg; remark out 
any val._SO_writeLock.acquire()) mostly due to explosions whilst pickling
     *) subclass the SQLObject into another class (which I called 
CompassObject, since our reservation system is called Compass ;) and 
overloaded _SO_setValue to do a delete inside memcache on change and 
overloaded 'get' such that it looks up memcache. I have to del/create 
_SO_writeLock, sqlmeta and _connection on each object when it's returned.

     Note that if someone was calling a series of 'setValue' instead of 
one large 'set', this would cause a lot of cache deletes and then misses 
when it tries to set. Thankfully, we have already audited the codebase 
for that (it was about a 3-4% "win" for us to change it over).

     Also, in a totally random thought, you will probably hit the 1mb 
memcache limit when storing objects. Since we are storing objects 
without blobs, it works. The only real workaround if people start to 
complain about memcache 'not storing their objects' is to either go 
Tokyo Tyrant or Redis. If you go Redis, you also end up with more speed 
and m->s replication for free ;).

     I am more than willing to help and/or bounce ideas off if you want. 
The solution I have may not be 'perfect' but it's definitely performant 
enough for our needs :D

     Regards
     Stef


On 02/01/2010 12:05 PM, Oleg Broytman wrote:
> Hello. I have returned to investigate the issue. Are you still interested?
>
> On Fri, Jul 31, 2009 at 01:10:34PM -0400, Stef Telford wrote:
>    
>>     Okay.. so.. assuming I can get a writeLock again after del'ing it..
>> we should be good there. If you can let me know exactly -what- in the
>> sqlmeta blows up, that may also help matters. The connection surely
>> explodes (And rightly so I think).
>>      
>     The problem with sqlmeta is that it's a class generated on the fly. It
> doesn't even have a module - and pickle protocol prefers stable classes,
> with proper modules and names.
>     But I think it's easy to deal with it anyway. We don't need to pickle
> sqlmeta *class* - the class is in the corresponding SQLObject class. We
> probably even don't need to pickle sqlmeta instance - we can just recreate
> it on unpickling 'cause its only instance data is a reference to SQLObject
> instance and we know the instance - at the time of unpickling it's 'self'
> so we can reconstruct a sqlmeta instance using the known class and known
> instance data. Just for the (rare) case a user would have put some data
> into the instance I can save sqlmeta.__dict__ and restore it on unpickling.
>     I am going to add the corresponding __getstate__ and __setstate__ to
> SQLObject and test them a bit.
>
> Oleg.
>    

Re: [SQLObject] memcache

[Oleg B. <ph...@ph...>]

Hello. I have returned to investigate the issue. Are you still interested?

On Fri, Jul 31, 2009 at 01:10:34PM -0400, Stef Telford wrote:
>    Okay.. so.. assuming I can get a writeLock again after del'ing it..  
> we should be good there. If you can let me know exactly -what- in the  
> sqlmeta blows up, that may also help matters. The connection surely  
> explodes (And rightly so I think).

   The problem with sqlmeta is that it's a class generated on the fly. It
doesn't even have a module - and pickle protocol prefers stable classes,
with proper modules and names.
   But I think it's easy to deal with it anyway. We don't need to pickle
sqlmeta *class* - the class is in the corresponding SQLObject class. We
probably even don't need to pickle sqlmeta instance - we can just recreate
it on unpickling 'cause its only instance data is a reference to SQLObject
instance and we know the instance - at the time of unpickling it's 'self'
so we can reconstruct a sqlmeta instance using the known class and known
instance data. Just for the (rare) case a user would have put some data
into the instance I can save sqlmeta.__dict__ and restore it on unpickling.
   I am going to add the corresponding __getstate__ and __setstate__ to
SQLObject and test them a bit.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

Re: [SQLObject] Problem with InheritableSQLObject and ForeignKeys/MultipleJoin

[Oleg B. <ph...@ph...>]

On Fri, Jan 29, 2010 at 02:53:26PM +0300, Oleg Broytman wrote:
> class File(SQLObject):
>     name = StringCol()
>     root = ForeignKey("Directory", default=None)
> 
> class Directory(File):
>     dirs = MultipleJoin("Directory", joinColumn="root_id")
>     files = MultipleJoin("File", joinColumn="root_id")

   On the other hand, this means two separate calls to .select() to draw
Files and Directories while with InheritableSQLObject it could be one.
   And this means the original design was wrong, perhaps. I think you don't
need two different MultipleJoins - you need one (all directory entries) and
two filters. Let me show this with the code:

class DirEntry(InheritableSQLObject):
    name = StringCol()
    parent = ForeignKey("DirEntry")

class File(DirEntry):
    pass

class Directory(DirEntry):
    entries = MultipleJoin("DirEntry", joinColumn="parent_id")

    def _filter_entries(self, klass):
        return [entry for entry in self.entries if isinstance(entry, klass)]

    def _get_files(self):
        return self._filter_entries(File)

    def _get_directories(self):
        return self._filter_entries(Directory)

DirEntry.createTable()
File.createTable()
Directory.createTable()

root = Directory(name='/', parent=None)
etc = Directory(name='/etc', parent=root)
X11 = Directory(name='/etc/X11', parent=etc)
passwd = File(name='/etc/passwd', parent=etc)

print etc.directories
print etc.files

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

Re: [SQLObject] Problem with InheritableSQLObject and ForeignKeys/MultipleJoin

[Juan M. S. <vic...@gm...>]

On Friday 29 January 2010 08:53:26 Oleg Broytman wrote:
> On Thu, Jan 28, 2010 at 09:35:50PM -0300, Juan Manuel Santos wrote:
> > class File(InheritableSQLObject):
> >     name = StringCol()
> >     root = ForeignKey("Directory", default=None)
> >
> > class Directory(File):
> >     dirs = MultipleJoin("Directory", joinColumn="root_id")
> >     files = MultipleJoin("File", joinColumn="root_id")
> >
> > sqlobject.dberrors.OperationalError: no such column: root_id
> >
> >  1/QueryR  :  SELECT id FROM directory WHERE root_id = (1)
> 
>    Table 'directory' doesn't have root_id, only 'file' as it.
> 
>    The problem is that InheritableSQLObject is rather limited thing. These
> limitations are documented:
> http://sqlobject.org/Inheritance.html#limitations-and-notes
>    The problem you are stumbled upon is "Inheritance works in two stages -
> first it draws the IDs from the parent table and then it draws the rows
> from the children tables. The first stage could fail if you try to do
> complex things. For example, Children.select(orderBy=Children.q.column,
> distinct=True) could fail because at the first stage inheritance generates
> a SELECT query for the parent table with ORDER BY the column from the
> children table."
>    Your code choked on the first stage while trying to SELECT IDs from the
> child table using a column that only exists in the parent.
> 
>    You can try simple Python inheritance instead:
> 
> class File(SQLObject):
>     name = StringCol()
>     root = ForeignKey("Directory", default=None)
> 
> class Directory(File):
>     dirs = MultipleJoin("Directory", joinColumn="root_id")
>     files = MultipleJoin("File", joinColumn="root_id")
> 
>    This works by copying all parent columns to the child table.
> 
> Oleg.
> 

Yes, that's what I ended up doing, after I thought it might be a limitation. 
But well, it works fine for the rest of the classes.

Thanks :)

Juan Manuel

Re: [SQLObject] Problem with InheritableSQLObject and ForeignKeys/MultipleJoin

[Oleg B. <ph...@ph...>]

On Thu, Jan 28, 2010 at 09:35:50PM -0300, Juan Manuel Santos wrote:
> class File(InheritableSQLObject):
>     name = StringCol()
>     root = ForeignKey("Directory", default=None)
> 
> class Directory(File):
>     dirs = MultipleJoin("Directory", joinColumn="root_id")
>     files = MultipleJoin("File", joinColumn="root_id")
> 
> sqlobject.dberrors.OperationalError: no such column: root_id
> 
>  1/QueryR  :  SELECT id FROM directory WHERE root_id = (1)

   Table 'directory' doesn't have root_id, only 'file' as it.

   The problem is that InheritableSQLObject is rather limited thing. These
limitations are documented:
http://sqlobject.org/Inheritance.html#limitations-and-notes
   The problem you are stumbled upon is "Inheritance works in two stages -
first it draws the IDs from the parent table and then it draws the rows
from the children tables. The first stage could fail if you try to do
complex things. For example, Children.select(orderBy=Children.q.column,
distinct=True) could fail because at the first stage inheritance generates
a SELECT query for the parent table with ORDER BY the column from the
children table."
   Your code choked on the first stage while trying to SELECT IDs from the
child table using a column that only exists in the parent.

   You can try simple Python inheritance instead:

class File(SQLObject):
    name = StringCol()
    root = ForeignKey("Directory", default=None)

class Directory(File):
    dirs = MultipleJoin("Directory", joinColumn="root_id")
    files = MultipleJoin("File", joinColumn="root_id")

   This works by copying all parent columns to the child table.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

[SQLObject] Problem with InheritableSQLObject and ForeignKeys/MultipleJoin

[Juan M. S. <vic...@gm...>]

Hi everybody. I've recently started using SQLObject for a Python program I'm 
writing to replace the previous object structure (was using Pickle but it got 
too big :P).
Anyway, the relevant part is this: I have a File object (that I defined) and a 
Directory object, like so:

class File(InheritableSQLObject):
    name = StringCol()
    root = ForeignKey("Directory", default=None)

class Directory(File):
    dirs = MultipleJoin("Directory", joinColumn="root_id")
    files = MultipleJoin("File", joinColumn="root_id")

File has more attributes, but I only put the relevant ones. As can be seen, 
each file has a root FK which points to the parent directory. Since 
directories are files, they inherit from File and the only extra attributes 
they provide are two lists, one for the files it contains and one for the 
subdirectories it contains. While I could do without this last part, it is 
there for compatibility's sake (since the old structure worked more or less 
like this). Changing this would require another further rewrite, which I'm 
trying to avoid.

So, the problem: wether I retrieve a Directory through Directory.select or 
Directory.selectBy, I can access directory.files fine, but whenever I try to 
access directory.dirs, I get this:

.select:

  File "/home/godlike/workspace/python/indexor/controller/tvhandler.py", line 
133, in append_directories
    print _dir.dirs
  File "<string>", line 1, in <lambda>
  File "/usr/lib64/python2.6/site-packages/sqlobject/joins.py", line 144, in 
performJoin
    inst.id)
  File "/usr/lib64/python2.6/site-packages/sqlobject/dbconnection.py", line 
549, in _SO_selectJoin
    self.sqlrepr(value)))
  File "/usr/lib64/python2.6/site-packages/sqlobject/dbconnection.py", line 
362, in queryAll
    return self._runWithConnection(self._queryAll, s)
  File "/usr/lib64/python2.6/site-packages/sqlobject/dbconnection.py", line 
262, in _runWithConnection
    val = meth(conn, *args)
  File "/usr/lib64/python2.6/site-packages/sqlobject/dbconnection.py", line 
355, in _queryAll
    self._executeRetry(conn, c, s)
  File "/usr/lib64/python2.6/site-
packages/sqlobject/sqlite/sqliteconnection.py", line 183, in _executeRetry
    raise OperationalError(ErrorMessage(e))
sqlobject.dberrors.OperationalError: no such column: root_id

Select performed:
 1/Select  :  SELECT file.id, file.parent, file.name, file.relpath, 
file.mimetype, file.atime, file.mtime, file.size, file.strsize, file.isdir, 
file.root_id, file.child_name FROM file WHERE (((file.name) = ('Desktop')) AND 
((file.child_name) = ('Directory'))) LIMIT 1
 1/QueryR  :  SELECT file.id, file.parent, file.name, file.relpath, 
file.mimetype, file.atime, file.mtime, file.size, file.strsize, file.isdir, 
file.root_id, file.child_name FROM file WHERE (((file.name) = ('Desktop')) AND 
((file.child_name) = ('Directory'))) LIMIT 1
 1/Select children of the class Directory:  SELECT directory.id, 
directory.child_name FROM directory WHERE ((directory.id) = (1))
 1/QueryR  :  SELECT directory.id, directory.child_name FROM directory WHERE 
((directory.id) = (1))
 1/COMMIT  :  auto
 1/QueryAll:  SELECT id FROM directory WHERE root_id = (1)
 1/QueryR  :  SELECT id FROM directory WHERE root_id = (1)
 1/COMMIT  :  auto

========================

.selectBy:
  File "/home/godlike/workspace/python/indexor/controller/tvhandler.py", line 
133, in append_directories
    print _dir.dirs
  File "<string>", line 1, in <lambda>
  File "/usr/lib64/python2.6/site-packages/sqlobject/joins.py", line 144, in 
performJoin
    inst.id)
  File "/usr/lib64/python2.6/site-packages/sqlobject/dbconnection.py", line 
549, in _SO_selectJoin
    self.sqlrepr(value)))
  File "/usr/lib64/python2.6/site-packages/sqlobject/dbconnection.py", line 
362, in queryAll
    return self._runWithConnection(self._queryAll, s)
  File "/usr/lib64/python2.6/site-packages/sqlobject/dbconnection.py", line 
262, in _runWithConnection
    val = meth(conn, *args)
  File "/usr/lib64/python2.6/site-packages/sqlobject/dbconnection.py", line 
355, in _queryAll
    self._executeRetry(conn, c, s)
  File "/usr/lib64/python2.6/site-
packages/sqlobject/sqlite/sqliteconnection.py", line 183, in _executeRetry
    raise OperationalError(ErrorMessage(e))
sqlobject.dberrors.OperationalError: no such column: root_id

Select performed:
 1/Select  :  SELECT directory.id, directory.child_name FROM directory, file 
WHERE ((((file.name) = ('Desktop')) AND ((directory.id) = (file.id))) AND 
((directory.id) = (file.id))) LIMIT 1
 1/QueryR  :  SELECT directory.id, directory.child_name FROM directory, file 
WHERE ((((file.name) = ('Desktop')) AND ((directory.id) = (file.id))) AND 
((directory.id) = (file.id))) LIMIT 1
 1/COMMIT  :  auto
 1/QueryAll:  SELECT id FROM directory WHERE root_id = (1)
 1/QueryR  :  SELECT id FROM directory WHERE root_id = (1)
 1/COMMIT  :  auto


Any ideas?

Thanks
Juan Manuel

Re: [SQLObject] I wish bool evaluation of selectResults objects ran .count()

[Matthew W. <ma...@tp...>]

On Wed 27 Jan 2010 11:47:27 AM EST, Oleg Broytman wrote:
> On Wed, Jan 27, 2010 at 03:30:18PM +0000, Matthew Wilson wrote:
>>     def __nonzero__(self):
>>         return self.count()
>
>    Let's do it in smaller steps. Patch your copy of SQLObject and run your
> programs for some time. Then report, please, how well it goes.
>
> Oleg.

Sounds good.  I'll keep you informed.

Re: [SQLObject] I wish bool evaluation of selectResults objects ran .count()

[Oleg B. <ph...@ph...>]

On Wed, Jan 27, 2010 at 03:30:18PM +0000, Matthew Wilson wrote:
>     def __nonzero__(self):
>         return self.count()

   Let's do it in smaller steps. Patch your copy of SQLObject and run your
programs for some time. Then report, please, how well it goes.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

Re: [SQLObject] I wish bool evaluation of selectResults objects ran .count()

[Oleg B. <ph...@ph...>]

On Wed, Jan 27, 2010 at 04:27:03PM +0000, Matthew Wilson wrote:
> Here's another solution -- instead of defining nonzero, what about
> defining __len__, so I could use this approach on both lists AND
> selectResults objects:

   http://sqlobject.org/FAQ.html#why-there-is-no-len

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

Re: [SQLObject] I wish bool evaluation of selectResults objects ran .count()

[Matthew W. <ma...@tp...>]

On Wed 27 Jan 2010 11:13:53 AM EST, Oleg Broytman wrote:
> On Wed, Jan 27, 2010 at 03:30:18PM +0000, Matthew Wilson wrote:
>>     def __nonzero__(self):
>>         return self.count()
>
>    This could led to a potential problem in code like this:
>
> if some_condition:
>     query = MyTable.select(...)
> else:
>     query = None
>
> if query:
>     ...
>
>    There is a subtle bug if self.count() returns 0. Currently people are
> accustomed that SelectResults instances are always evaluated to True in a
> boolean context.
>    I don't know, though, how often people write code like that.

Yeah, there's certainly a downside of what I'm suggesting.  The scenario
you describe doesn't worry me personally because I don't usually do
stuff like that, but I wouldn't want to break other people's code.

Here's another solution -- instead of defining nonzero, what about
defining __len__, so I could use this approach on both lists AND
selectResults objects:

    if len(query):
        ...

The goal for me is to not need to remember if my collection is just a
query or whether it is a list of objects retrieved from a query.

Matt

Re: [SQLObject] I wish bool evaluation of selectResults objects ran .count()

[Oleg B. <ph...@ph...>]

On Wed, Jan 27, 2010 at 03:30:18PM +0000, Matthew Wilson wrote:
>     def __nonzero__(self):
>         return self.count()

   This could led to a potential problem in code like this:

if some_condition:
    query = MyTable.select(...)
else:
    query = None

if query:
    ...

   There is a subtle bug if self.count() returns 0. Currently people are
accustomed that SelectResults instances are always evaluated to True in a
boolean context.
   I don't know, though, how often people write code like that.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

[SQLObject] I wish bool evaluation of selectResults objects ran .count()

[Matthew W. <ma...@tp...>]

I have a lot of code that looks vaguely like this:

    if employees.count():
        for emp in employees:
            print emp

That code requires that the employees object is a selectResults object,
not a list.  When employees is a list, I write the same code this way:

    if employees:
        for emp in employees:
            print emp

More than a few times, I forget if I'm working on a selectResults
instance or on a list.

If we add a method to the selectResults class like this:

    def __nonzero__(self):
        return self.count()

Then I would never need to keep track of the nature of my employees
object.

This would REALLY help me out.

If anyone else likes this idea, let me know, and I'll write a patch.

Matt

Re: [SQLObject] This transaction has already gone through ROLLBACK

[Oleg B. <ph...@ph...>]

On Tue, Jan 26, 2010 at 12:26:51PM +0100, Sophana K wrote:
> Do you have an idea what could explain the different tracebacks I got?
> Could it be related to multithreading?

   They could be related to multithreading; actually, I am almost
completely sure they are related. On the other hand they could be related
to a "wrong" sequence of calls (attribute access) - a code accessed an
attribute of the connection, the access gets redirected to the transaction,
but 'self' is wrong and there are problems with further attribute access;
similar to a bug I fixed by adding .close() method to Transaction class
because without it .close gets redirected to DBConnection.close but that's
wrong for a transaction.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

Re: [SQLObject] This transaction has already gone through ROLLBACK

[Sophana K <sop...@gm...>]

Ok thanks
Do you have an idea what could explain the different tracebacks I got?
Could it be related to multithreading?

Re: [SQLObject] This transaction has already gone through ROLLBACK

[Oleg B. <ph...@ph...>]

On Mon, Jan 25, 2010 at 06:34:06PM +0100, Sophana K wrote:
> In all cases, it looks like Transaction.__getattr__ gets called at
> places I don't understand.
> I don't understand very well what is the function of the
> Transaction.__getattr__ method.

   Transaction.__getattr__ redirects attribute access to the underlying
connection (self._dbConnection); non-method attributes are wrapped into
ConnWrapper instances; method access are wrapped as instancemethod with
fake class - the class is self.__class__ (Transaction), not the class of
the connection (self._dbConnection); the last one is neccessary to
redirect attribute access on self._dbConnection back to the transaction
instance.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

Re: [SQLObject] FW: longId=True handling...

[Oleg B. <ph...@ph...>]

On Fri, Jan 22, 2010 at 09:57:13PM +0300, Oleg Broytman wrote:
>    In essence, the problem is in class sqlmeta, method setClass:
> 
>         if cls.idName is None:
>             cls.idName = cls.style.idForTable(cls.table)
> 
>    The method poisons the class so in any class inherited from sqlmeta idName
> will be already set and the test fails.
>    Adding idName to the list of unshared attributes resets idName to None
> but that prevents proper inheritance of idName.
>    To explain all this with code - with current SQLObject the following
> code
> 
> class sqlmeta(sqlmeta):
>     idName = 'ddd'
> 
> class Test1(SQLObject):
>     class sqlmeta(sqlmeta):
>         style = MixedCaseStyle(longID=True)
> 
> print Test1.sqlmeta.idName
> 
> class Test2(SQLObject):
>     class sqlmeta:
>         style = MixedCaseStyle(longID=True)
> 
> print Test2.sqlmeta.idName
> 
>    prints 'ddd' and 'id'; the first idName is right and the second one is
> wrong (it must be 'Test2ID').
>    The same code running with idName added to the list of unshared attributes
> prints 'Test1ID' and 'Test2ID'; the first one is wrong (it must be 'ddd')
> and the second one is right.
>    Probably the correct way of fixing it is to add idName (and 'style'
> because setClass tests and sets it too) to the list of unshared attributes
> but make sqlmeta.__classinit__ to recognize inherited attributes and do not
> reset them. I will think of it...

   Well, instead of trying to find what was inherited and what was set in
parent's setClass() I just prevented setClass() from poisoning the base
sqlmeta. The patch is just

Index: sqlobject/main.py
===================================================================
--- sqlobject/main.py   (revision 4089)
+++ sqlobject/main.py   (working copy)
@@ -853,7 +853,8 @@
                     del values[key]
             cls.sqlmeta = type('sqlmeta', (superclass,), values)
 
-        cls.sqlmeta.setClass(cls)
+        if not is_base: # Do not pollute the base sqlmeta class
+            cls.sqlmeta.setClass(cls)
 
     _SO_setupSqlmeta = classmethod(_SO_setupSqlmeta)

   With the patch the code above prints 'ddd' and 'Test2ID' which is
exactly right.
   The test suite passed so if nobody reports any problem with the patch I
will apply it to the trunk.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

Re: [SQLObject] This transaction has already gone through ROLLBACK

[Sophana K <sop...@gm...>]

Hi

I still haven't resolved my rollback assert problem which happens
about once per month.
But I have gathered some tracebacks.
In all cases, it looks like Transaction.__getattr__ gets called at
places I don't understand.
Maybe tracebacks are corrupted?
If someone has a clue where to look at, it would be nice.
I don't understand very well what is the function of the
Transaction.__getattr__ method.

Thanks for your help.


File "/usr/lib/python2.5/site-packages/SQLObject-0.10.3-py2.5.egg/sqlobject/dbconnection.py",
line 866, in doInTransaction
   conn = old_conn.transaction()
 File "/usr/lib/python2.5/site-packages/SQLObject-0.10.3-py2.5.egg/sqlobject/dbconnection.py",
line 398, in transaction
   return Transaction(self)
 File "/usr/lib/python2.5/site-packages/SQLObject-0.10.3-py2.5.egg/sqlobject/dbconnection.py",
line 681, in __init__
   self._connection = dbConnection.getConnection()
 File "/usr/lib/python2.5/site-packages/SQLObject-0.10.3-py2.5.egg/sqlobject/dbconnection.py",
line 281, in getConnection
   self._poolLock.release()
 File "/usr/lib/python2.5/site-packages/SQLObject-0.10.3-py2.5.egg/sqlobject/dbconnection.py",
line 764, in __getattr__
   self.assertActive()
 File "/usr/lib/python2.5/site-packages/SQLObject-0.10.3-py2.5.egg/sqlobject/dbconnection.py",
line 688, in assertActive
   assert not self._obsolete, "This transaction has already gone
through ROLLBACK; begin another transaction"
AssertionError: This transaction has already gone through ROLLBACK;
begin another transaction

Here getattr is called in self._poolLock.release() inside a DBAPI
method: getConnection
self is a DBAPI instance.
I don't see why the Transaction.__getattr__ gets called.


File "/usr/lib/python2.5/site-packages/SQLObject-0.10.3-py2.5.egg/sqlobject/dbconnection.py",
line 878, in doInTransaction
   conn.commit(close=True)
 File "/usr/lib/python2.5/site-packages/SQLObject-0.10.3-py2.5.egg/sqlobject/dbconnection.py",
line 729, in commit
   if self._dbConnection.debug:
 File "/usr/lib/python2.5/site-packages/SQLObject-0.10.3-py2.5.egg/sqlobject/dbconnection.py",
line 764, in __getattr__
   self.assertActive()
 File "/usr/lib/python2.5/site-packages/SQLObject-0.10.3-py2.5.egg/sqlobject/dbconnection.py",
line 688, in assertActive
   assert not self._obsolete, "This transaction has already gone
through ROLLBACK; begin another transaction"
AssertionError: This transaction has already gone through ROLLBACK;
begin another transaction

here getattr is called in self._dbConnection.debug
self is a Transaction. Maybe _dbConnection doesn't exist anymore?

File "/usr/lib/python2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/dbconnection.py",
line 856, in doInTransaction
   conn = old_conn.transaction()
 File "/usr/lib/python2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/dbconnection.py",
line 754, in __getattr__
   self.assertActive()
 File "/usr/lib/python2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/dbconnection.py",
line 678, in assertActive
   assert not self._obsolete, "This transaction has already gone
through ROLLBACK; begin another transaction"
AssertionError: This transaction has already gone through ROLLBACK;
begin another transaction

Re: [SQLObject] FW: longId=True handling...

[Oleg B. <ph...@ph...>]

Hello! I returned to investigate the issue and found it's more complex than
I thought...

On Thu, Jun 11, 2009 at 10:11:27AM -0400, Joe Lanese wrote:
> This issue seems to be caused by a change in SQLObject.
> The change in behaviour seems to be due to a change made in SQLObject in
> main.py, where the ?idName? attribute was taken out of a list of unshared
> attributes.  This caused the sqlmeta class to share the idName value with
> its containing class, so they both end up with ?id? even when longID is set
> to true in your own class.  If you add it back in:
> _unshared_attributes = ['table', 'columns', 'childName', 'idName']
> then it should work as before...however, I'm not sure if this will cause
> other problems (so far it seems to work).
> 
> By the way, here is a link to a change that seems to have triggered this
> change in behaviour:
> http://www.mail-archive.com/sql...@li.../msg02159.html

   In essence, the problem is in class sqlmeta, method setClass:

        if cls.idName is None:
            cls.idName = cls.style.idForTable(cls.table)

   The method poises the class so in any class inherited from sqlmeta idName
will be already set and the test fails.
   Adding idName to the list of unshared attributes resets idName to None
but that prevents proper inheritance of idName.
   To explain all this with code - with current SQLObject the following
code

class sqlmeta(sqlmeta):
    idName = 'ddd'

class Test1(SQLObject):
    class sqlmeta(sqlmeta):
        style = MixedCaseStyle(longID=True)

Test1.createTable()

print Test1.sqlmeta.idName

class Test2(SQLObject):
    class sqlmeta:
        style = MixedCaseStyle(longID=True)

Test2.createTable()

print Test2.sqlmeta.idName

   prints 'ddd' and 'id'; the first idName is right and the second one is
wrong (it must be 'Test2ID').
   The same code running with idName added to the list of unshared attributes
prints 'Test1ID' and 'Test2ID'; the first one is wrong (it must be 'ddd')
and the second one is right.
   Probably the correct way of fixing it is to add idName (and 'style'
because setClass tests and sets it too) to the list of unshared attributes
but make sqlmeta.__classinit__ to recognize inherited attributes and do not
reset them. I will think of it...

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.