From: Max I. <ma...@uc...> - 2003-11-26 09:22:53
|
Hi, I have a question regarding usage of db connection in WebKit and (related to it), how thread-safe servlets are. Currently I have a db conn. as a module global variable in a sitepage and it is used by several servlets. But I'm not sure this is a good way to deal with db. Anyway it is probably buggy as most DB-API interfaces are not thread-safe and WebKit is multithreaded and hence there is a possiblilty for crash due to simulationous access to the same conn from multiple threads. I think db conn should be created when needed and released when unneeded, as to conserve system resources. But was is the best way to achieve this in WebKit? I think about using a conn pool and storing a conn in a session variable or in a servlet instance variable (via awake/sleep). Only not sure whether it would be guaranteed that it can't be accessed from mulitiple threads. If session variable or servlet instance variable can be accessed from mulitiple threads then I see no other route as to request conn only where it needed (in a method body) and release it immediately. What do you think? |
From: Warren S. <wa...@wa...> - 2003-11-26 16:25:00
|
Max Ischenko said: > > Hi, > > I have a question regarding usage of db connection in WebKit and > (related to it), how thread-safe servlets are. > > Currently I have a db conn. as a module global variable in a sitepage > and it is used by several servlets. But I'm not sure this is a good way > to deal with db. Anyway it is probably buggy as most DB-API interfaces > are not thread-safe and WebKit is multithreaded and hence there is a > possiblilty for crash due to simulationous access to the same conn from > multiple threads. > > I think db conn should be created when needed and released when > unneeded, as to conserve system resources. But was is the best way to > achieve this in WebKit? > > I think about using a conn pool and storing a conn in a session variabl= e > or in a servlet instance variable (via awake/sleep). Only not sure > whether it would be guaranteed that it can't be accessed from mulitiple > threads. > > If session variable or servlet instance variable can be accessed from > mulitiple threads then I see no other route as to request conn only > where it needed (in a method body) and release it immediately. > > What do you think? > How you can safely use a database connection in a multi-threaded environment depends on the threadsafety of the DB API module that you are using. Some modules are only thread safe if each thread has exclusive us= e of its own connection (threadsafety of 1). Other modules allow multiple threads to use the same connection simultaneously (threadsafety >=3D 2).=20 The module you are using should have a "threadsafety" or "threadlevel" attribute that contains the level of thread safety it supports. The modules that I have experience with (MySQLdb and mx.ODBC.DB2) have only had a threadsafety of 1. In order to guarantee that each thread has exclusive use of a connection, it must either create and close the connection itself, or get a connection from a pool that guarantees that the connection will not be "handed out" to another thread while the first thread is using it. The DBPool.py module in the MiscUtils package of Webware provides a simpl= e thread-safe pooling implementation that works for DB API modules with a threadsafety of 1 or greater. However, several people (including myself) have found it lacking in features. I recently added an alternative to DBPool.py to the Webware Sandbox: http://cvs.sf.net/viewcvs.py/webware-sandbox/Sandbox/wsmith323/DbConnecti= onPool.py It provides the same basic functionality for threadsafety =3D 1 modules a= s DBPool.py, but has additional features that require it to be completely different in the way it implements that functionality internally. Have a look at the doc string in the module for more details. --=20 Warren Smith wa...@wa... |
From: Ian B. <ia...@co...> - 2003-11-26 16:34:56
|
On Nov 26, 2003, at 3:23 AM, Max Ischenko wrote: > Hi, > > I have a question regarding usage of db connection in WebKit and > (related to it), how thread-safe servlets are. > > Currently I have a db conn. as a module global variable in a sitepage > and it is used by several servlets. But I'm not sure this is a good > way to deal with db. Anyway it is probably buggy as most DB-API > interfaces are not thread-safe and WebKit is multithreaded and hence > there is a possiblilty for crash due to simulationous access to the > same conn from multiple threads. > > I think db conn should be created when needed and released when > unneeded, as to conserve system resources. But was is the best way to > achieve this in WebKit? > > I think about using a conn pool and storing a conn in a session > variable or in a servlet instance variable (via awake/sleep). Only not > sure whether it would be guaranteed that it can't be accessed from > mulitiple threads. > > If session variable or servlet instance variable can be accessed from > mulitiple threads then I see no other route as to request conn only > where it needed (in a method body) and release it immediately. Sessions can be accessed from multiple threads, though usually that would only happen when a browser was downloading dependent files (e.g., images), that were also served off WebKit. But you still don't want to put these in the session, because they aren't pickleable, and the session persists over restarts by using pickle. You should either be able to use DBPool (or something like it), or do some other simple resource sharing. Lists work better than you might think: _storage = [] def getResource(): try: return _storage.pop() except IndexError: return makeNewResource() def returnResource(rsrc): _storage.append(rsrc) .append and .pop are threadsafe (enough) so it's pretty simple to implement. -- Ian Bicking | ia...@co... | http://blog.ianbicking.org |
From: Warren S. <wa...@wa...> - 2003-11-26 19:21:49
|
Ian Bicking said: > Sessions can be accessed from multiple threads, though usually that > would only happen when a browser was downloading dependent files (e.g., > images), that were also served off WebKit. But you still don't want to > put these in the session, because they aren't pickleable, and the > session persists over restarts by using pickle. > > You should either be able to use DBPool (or something like it), or do > some other simple resource sharing. Lists work better than you might > think: > > _storage =3D [] > def getResource(): > try: > return _storage.pop() > except IndexError: > return makeNewResource() > def returnResource(rsrc): > _storage.append(rsrc) > > .append and .pop are threadsafe (enough) so it's pretty simple to > implement. > I'm not sure if we have made it clear that regardless of what "pooling" method you use, the variable that stores the reference to the pool should be defined at either the module level or class level, not at the instance level. This insures that only a single instance of the pool exists for the entire python process. The above example is of a module level pool referenced by the "_storage" variable. Here is an example of the pool reference at the class level: class SomeClass: # Do this, since it creates a class variable. _storage =3D [] def __init__(self, ...): # Don't do this, since it creates an instance variable. #self._storage =3D [] def getResource(self): try: return self._storage.pop() except IndexError: return self.makeNewResource() def returnResource(self, rsrc): self._storage.append(rsrc) The choice of pooling method and where you put the reference to the pool depends on your application. --=20 Warren Smith wa...@wa... |
From: Aaron H. <aa...@me...> - 2003-11-26 19:57:18
|
I usually create a dbpool and get /return a connection just when it is needed. The connections always exist, but they are shared between servlets. def runExecute(user,sql): conn=clientpool[user.company].getConnection() c=conn.cursor() c.execute(sql) c.close() conn.close() Lately I have been thinking that I should just create a new db connection in the __init__ and check it in awake, since pretty much every servlet uses the database. -Aaron Max Ischenko wrote: > > Hi, > > I have a question regarding usage of db connection in WebKit and > (related to it), how thread-safe servlets are. > > Currently I have a db conn. as a module global variable in a sitepage > and it is used by several servlets. But I'm not sure this is a good > way to deal with db. Anyway it is probably buggy as most DB-API > interfaces are not thread-safe and WebKit is multithreaded and hence > there is a possiblilty for crash due to simulationous access to the > same conn from multiple threads. > > I think db conn should be created when needed and released when > unneeded, as to conserve system resources. But was is the best way to > achieve this in WebKit? > > I think about using a conn pool and storing a conn in a session > variable or in a servlet instance variable (via awake/sleep). Only not > sure whether it would be guaranteed that it can't be accessed from > mulitiple threads. > > If session variable or servlet instance variable can be accessed from > mulitiple threads then I see no other route as to request conn only > where it needed (in a method body) and release it immediately. > > What do you think? > > > > ------------------------------------------------------- > This SF.net email is sponsored by: SF.net Giveback Program. > Does SourceForge.net help you be more productive? Does it > help you create better code? SHARE THE LOVE, and help us help > YOU! Click Here: http://sourceforge.net/donate/ > _______________________________________________ > Webware-discuss mailing list > Web...@li... > https://lists.sourceforge.net/lists/listinfo/webware-discuss |
From: Max I. <ma...@uc...> - 2003-11-27 09:09:20
|
Aaron Held wrote: > I usually create a dbpool and get /return a connection just when it is > needed. The connections always exist, but they are shared between > servlets. > Lately I have been thinking that I should just create a new db > connection in the __init__ and check it in awake, since pretty much > every servlet uses the database. But this is dangerous because then db coon can be shared across threads (servlets are shared, right?). |