SQLObject

Hi,

I have used SQLObject with twisted+woven without problems.


om

J-P Lee wrote:

> Has anyone used SQLObject with woven+twisted?  Do they work well 
> together?  I'm currently using cherrypy and considering switching.  
> What do other people use?
>
> -- 
> J-P
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: VM Ware
> With VMware you can run multiple operating systems on a single machine.
> WITHOUT REBOOTING! Mix Linux / Windows / Novell virtual machines
> at the same time. Free trial click 
> here:http://www.vmware.com/wl/offer/358/0
> _______________________________________________
> sqlobject-discuss mailing list
> sqlobject-discuss@...
> https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss

On Tuesday, August 26, 2003, at 12:00 PM, J-P Lee wrote:
> Has anyone used SQLObject with woven+twisted?  Do they work well 
> together?  I'm currently using cherrypy and considering switching.  
> What do other people use?
>
I don't think SQLObject can work in Twisted.  Twisted uses an async 
model, and SQLObject blocks all the time (like when transparently doing 
database queries), which would bring Twisted to a halt until the query 
is finished (which you might not even notice until you have concurrent 
requests).  I suppose you could wrap every SQLObject attribute or 
method access in Twisted's threading stuff to avoid the blocking, but I 
don't know if SQLObject would feel very convenient after doing that.  
(I think the same blocking problem applies to CherryPy, but I believe 
there's also a way to configure it to use a thread for each request)

Because of async, when you use Twisted you can't use most normal Python 
libraries.  They are in a world of their own, at least if you want to 
use Twisted style.  They do have an ORM of their own.

I suppose you could use it if you moved to a different thread right 
away in your request, then you wouldn't have to guard every SQLObject 
attribute access.

   Ian

Ian Bicking wrote:

> I don't think SQLObject can work in Twisted.  Twisted uses an async 
> model, and SQLObject blocks all the time (like when transparently 
> doing database queries), which would bring Twisted to a halt until the 
> query is finished (which you might not even notice until you have 
> concurrent requests).  I suppose you could wrap every SQLObject 
> attribute or method access in Twisted's threading stuff to avoid the 
> blocking, but I don't know if SQLObject would feel very convenient 
> after doing that.  (I think the same blocking problem applies to 
> CherryPy, but I believe there's also a way to configure it to use a 
> thread for each request)

Hmm... sorry to hear that.  I can't get CherryPy's threading or forking 
to work with SQLObject.  Threading hangs sometimes, and forking crashes 
the server.

> Because of async, when you use Twisted you can't use most normal 
> Python libraries.  They are in a world of their own, at least if you 
> want to use Twisted style.  They do have an ORM of their own.
>
> I suppose you could use it if you moved to a different thread right 
> away in your request, then you wouldn't have to guard every SQLObject 
> attribute access.


I see.  Sounds like too much trouble.   Are there other suggestions for 
a web app framework that would work well?

Thanks.

--
J-P

On Tuesday, August 26, 2003, at 12:34 PM, J-P Lee wrote:
> I see.  Sounds like too much trouble.   Are there other suggestions 
> for a web app framework that would work well?

I might recommend Webware, SkunkWeb, Quixote, or Spyce.  Besides 
Twisted, CherryPy and Medusa, most frameworks should work fine.

   Ian

Ian Bicking wrote:

> On Tuesday, August 26, 2003, at 12:00 PM, J-P Lee wrote:
>
>> Has anyone used SQLObject with woven+twisted?  Do they work well 
>> together?  I'm currently using cherrypy and considering switching.  
>> What do other people use?
>>
> I don't think SQLObject can work in Twisted.  Twisted uses an async 
> model, and SQLObject blocks all the time (like when transparently 
> doing database queries), which would bring Twisted to a halt until the 
> query is finished (which you might not even notice until you have 
> concurrent requests).  I suppose you could wrap every SQLObject 
> attribute or method access in Twisted's threading stuff to avoid the 
> blocking, but I don't know if SQLObject would feel very convenient 
> after doing that.  (I think the same blocking problem applies to 
> CherryPy, but I believe there's also a way to configure it to use a 
> thread for each request)
>
> Because of async, when you use Twisted you can't use most normal 
> Python libraries.  They are in a world of their own, at least if you 
> want to use Twisted style.  They do have an ORM of their own.
>
> I suppose you could use it if you moved to a different thread right 
> away in your request, then you wouldn't have to guard every SQLObject 
> attribute access.
>
>   Ian
>
Hi,

I said this already but I've used twisted and sqlobject successfully. I use
twisted and sqlobject together in a call card top up system running on
a linux server, which has been running without problems for the past few
months.

I have never had to do anything special to get sqlobject and twisted to 
work.
I simply used sqlobject and If something took I long time to return, I 
use twisted's
deferred mechanism. This allows callback and errorbacks to be set which
get called when the database returns. I 've never used CherryPy so I can't
compare to it. I've only recently starting using woven in my current kiosk
project. But again I have had no problems. I guess it all comes down to
how you design and want your system to work. I'm happy for the calls
to block since I was using mysql on localhost, and it never take long to
excute any query.

om

On Tue, Aug 26, 2003 at 07:26:19PM +0100, Oisin Mulvihill wrote:
[snip Ian's comments]
> Hi,
> 
> I said this already but I've used twisted and sqlobject successfully. I use

Eh, it may *look* like you have. :) SQLObject is synchronous, Twisted
is asynchronous, there's no compatibility here.

e.g.

Person.select("hair_colour = 'blonde'")

In Twisted, every method call must return as fast as possible
(basically, instantly). In the above .select() is at the mercy of how
many blondes you have in your system. By extension, so is the rest of
your code; nothing else will get executed until that .select() comes
back.

A deferred (in Twisted) doesn't solve this, because .select() doesn't
know how to be asynchronous anyway so the code will block no matter
what you try to do.

> twisted and sqlobject together in a call card top up system running on
> a linux server, which has been running without problems for the past few
> months.

Interesting, but dangerous. Rest assurred if your system encounters
non-trivial load you'll need to rewrite it (by replacing SQLObject
with adbapi in Twisted.)

--
Brad Bollenbach
BBnet.ca

Hi,

Brad Bollenbach wrote:

>On Tue, Aug 26, 2003 at 07:26:19PM +0100, Oisin Mulvihill wrote:
>[snip Ian's comments]
>  
>
>>Hi,
>>
>>I said this already but I've used twisted and sqlobject successfully. I use
>>    
>>
>
>Eh, it may *look* like you have. :) SQLObject is synchronous, Twisted
>is asynchronous, there's no compatibility here.
>
>e.g.
>
>Person.select("hair_colour = 'blonde'")
>  
>
Ho-ho very funny.

>In Twisted, every method call must return as fast as possible
>(basically, instantly). In the above .select() is at the mercy of how
>many blondes you have in your system. By extension, so is the rest of
>your code; nothing else will get executed until that .select() comes
>back.
>
>A deferred (in Twisted) doesn't solve this, because .select() doesn't
>know how to be asynchronous anyway so the code will block no matter
>what you try to do.
>  
>
Surely if it doesn't matter how long it takes to return in your system 
this isn't
going to be a problem.  In my situation the caller needs to know and 
will wait
for an answer. I don't see the problem but I've only been dealing with 
database
for about 6months or so, mainly small scale stuff which doesn't take long to
return.

>>twisted and sqlobject together in a call card top up system running on
>>a linux server, which has been running without problems for the past few
>>months.
>>    
>>
>
>Interesting, but dangerous. Rest assurred if your system encounters
>non-trivial load you'll need to rewrite it (by replacing SQLObject
>with adbapi in Twisted.)
>
I presume this mean loosing the ORM and going to a low level of 
generating the
sql queries. Are there other ORMs in which this wouldn't be a problem?

Would it be possible to do an sqlobject reactor for twisted. I mean at a 
low level the
database connections are socket connections. You could poll and read 
socket descriptors.
This would then allow you to have the asynchronous behaviour.
 
All the best,

om

> Brad Bollenbach wrote:
>
>>> twisted and sqlobject together in a call card top up system running on
>>> a linux server, which has been running without problems for the past 
>>> few
>>> months.
>>>   
>>
>>
>> Interesting, but dangerous. Rest assurred if your system encounters
>> non-trivial load you'll need to rewrite it (by replacing SQLObject
>> with adbapi in Twisted.)
>>
I forgot to mention that the top up system did undergo stress testing 
before
going into the field. I had two seperate clients doing 4 topups per second
continuously for 3 days. Each topup consisted of roughly 5 sql queries
(via sqlobject). So thats 20 queries a second for 3 days. In this time I
encountered no problems at all. This stress test pushed the system far 
harder
then it was every going to be in the field, where the gui could only 
allow staff
to create a top up every two seconds or so. I don't know about you but I
consider that a "non-trival" load.

All the best,

om

On Tuesday, August 26, 2003, at 03:53 PM, Brad Bollenbach wrote:
> On Tue, Aug 26, 2003 at 07:26:19PM +0100, Oisin Mulvihill wrote:
> [snip Ian's comments]
>> Hi,
>>
>> I said this already but I've used twisted and sqlobject successfully. 
>> I use
>
> Eh, it may *look* like you have. :) SQLObject is synchronous, Twisted
> is asynchronous, there's no compatibility here.
>
> e.g.
>
> Person.select("hair_colour = 'blonde'")
>
> In Twisted, every method call must return as fast as possible
> (basically, instantly). In the above .select() is at the mercy of how
> many blondes you have in your system. By extension, so is the rest of
> your code; nothing else will get executed until that .select() comes
> back.
>
> A deferred (in Twisted) doesn't solve this, because .select() doesn't
> know how to be asynchronous anyway so the code will block no matter
> what you try to do.

You can still use a deferred.  I don't remember the specifics, but you 
can ask for a deferred on a blocking call, and that call is moved to 
another thread.  Once the call is finished in that other thread the 
deferred in the main thread gets invoked.  That's how *all* database 
access in Twisted works, because as far as I know none of the 
underlying drivers have asynchronous interfaces (at least not exported 
to Python).

Twisted's native ORM (adbapi?) makes the asynchronous stuff easier, 
generating deferreds on its own and such, but I'm pretty sure it works 
the same underneath, putting database calls in a separate thread.

   Ian

Is there a better way to handle multiple connections of a single table 
than this:

> class mytable(SQLObject):
> 	_connection = localConnection
> 	name = StrCol(length=255)
> 	data = StrCol()
>
> localdata = mytable.select()
> # change connection
> mytable._connection = remoteConnection
> remotedata = mytable.select()
> # change it back...
> mytable._connection = localConnection

The object returned by the two different connections are actually 
different objects and update properly through the right database 
connection, but any related objects returned through a RelatedJoin are 
the same, using the current connection.  Is there a better way to do 
this?

I thought I might be able to do this:

> class mytableRemote(mytable):
> 	_connection = remoteConnection

...but it doesn't seem to work.  Or, I thought I could just sub-class 
SQLObject...

> class mySQLObject(SQLObject):
> 	_connection = localConnection
> 	def connectRemotely(self):
> 		self.__class__._connection = remoteConnection
> 	def connectLocally(self):
> 		self.__class__._connection = localConnection

I know that I can duplicate my table definition modules and just give 
each one a different connection object, but I want to be able to re-use 
the class definitions...

Thanks,
Tracy

Was there a suggestion on how to deal with multiple databases with the 
same tables?  I have an admin database that has content-related tables 
that periodically get "published" to the live database.  I was 
wondering if anyone had a suggestion about how to re-use an SQLObject 
class definitions with multiple _connection objects.

--T


On Wednesday, August 27, 2003, at 01:43 PM, Tracy S. Ruggles wrote:

> Is there a better way to handle multiple connections of a single table 
> than this:
>
>> class mytable(SQLObject):
>> 	_connection = localConnection
>> 	name = StrCol(length=255)
>> 	data = StrCol()
>>
>> localdata = mytable.select()
>> # change connection
>> mytable._connection = remoteConnection
>> remotedata = mytable.select()
>> # change it back...
>> mytable._connection = localConnection
>
> The object returned by the two different connections are actually 
> different objects and update properly through the right database 
> connection, but any related objects returned through a RelatedJoin are 
> the same, using the current connection.  Is there a better way to do 
> this?
>
> I thought I might be able to do this:
>
>> class mytableRemote(mytable):
>> 	_connection = remoteConnection
>
> ...but it doesn't seem to work.  Or, I thought I could just sub-class 
> SQLObject...
>
>> class mySQLObject(SQLObject):
>> 	_connection = localConnection
>> 	def connectRemotely(self):
>> 		self.__class__._connection = remoteConnection
>> 	def connectLocally(self):
>> 		self.__class__._connection = localConnection
>
> I know that I can duplicate my table definition modules and just give 
> each one a different connection object, but I want to be able to 
> re-use the class definitions...
>
> Thanks,
> Tracy
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> sqlobject-discuss mailing list
> sqlobject-discuss@...
> https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
>

Tracy S. Ruggles wrote:

> Was there a suggestion on how to deal with multiple databases with the 
> same tables?  I have an admin database that has content-related tables 
> that periodically get "published" to the live database.  I was 
> wondering if anyone had a suggestion about how to re-use an SQLObject 
> class definitions with multiple _connection objects.


I don't think this is possible with SQLObject. The "published" system 
you mention seems
like a nightmare waiting to happen, having separate databases getting 
their information from one source at any time.

Surely another safer way of doing this would be to have a client-server. 
The server manages the main database and the clients go through it. The 
clients at certain
times then ask the server for the latest changes via some control logic. 
They would
get the changes and add them to their own databases.

Well thats my two cents,

all the best,

om





>
> --T
>
>
> On Wednesday, August 27, 2003, at 01:43 PM, Tracy S. Ruggles wrote:
>
>> Is there a better way to handle multiple connections of a single 
>> table than this:
>>
>>> class mytable(SQLObject):
>>>     _connection = localConnection
>>>     name = StrCol(length=255)
>>>     data = StrCol()
>>>
>>> localdata = mytable.select()
>>> # change connection
>>> mytable._connection = remoteConnection
>>> remotedata = mytable.select()
>>> # change it back...
>>> mytable._connection = localConnection
>>
>>
>> The object returned by the two different connections are actually 
>> different objects and update properly through the right database 
>> connection, but any related objects returned through a RelatedJoin 
>> are the same, using the current connection.  Is there a better way to 
>> do this?
>>
>> I thought I might be able to do this:
>>
>>> class mytableRemote(mytable):
>>>     _connection = remoteConnection
>>
>>
>> ...but it doesn't seem to work.  Or, I thought I could just sub-class 
>> SQLObject...
>>
>>> class mySQLObject(SQLObject):
>>>     _connection = localConnection
>>>     def connectRemotely(self):
>>>         self.__class__._connection = remoteConnection
>>>     def connectLocally(self):
>>>         self.__class__._connection = localConnection
>>
>>
>> I know that I can duplicate my table definition modules and just give 
>> each one a different connection object, but I want to be able to 
>> re-use the class definitions...
>>
>> Thanks,
>> Tracy
>>
>>
>>
>> -------------------------------------------------------
>> This sf.net email is sponsored by:ThinkGeek
>> Welcome to geek heaven.
>> http://thinkgeek.com/sf
>> _______________________________________________
>> sqlobject-discuss mailing list
>> sqlobject-discuss@...
>> https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
>>
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> sqlobject-discuss mailing list
> sqlobject-discuss@...
> https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss



-- 
Oisin Mulvihill
Engines Of Creation

Email: oisin@...
Work: +353 1 6791602
Mobile: +353 868191540

On Friday, September 5, 2003, at 10:49 AM, Tracy S. Ruggles wrote:
> Was there a suggestion on how to deal with multiple databases with the 
> same tables?  I have an admin database that has content-related tables 
> that periodically get "published" to the live database.  I was 
> wondering if anyone had a suggestion about how to re-use an SQLObject 
> class definitions with multiple _connection objects.

Shouldn't be a big problem.  You can pass the connection during 
instantiation, if you want to use the exact same class.  Otherwise 
subclassing, like:

develConn = PostgresConnection('devel...')
liveConn = PostgresConnection('live...')

class Whatever(SQLObject):
     _connection = develConn
     ...

class LiveWhatever(Whatever):
     _connection = liveConn

   Ian