From: Michael E. <men...@ka...> - 2002-12-02 18:08:32
|
Hi all - I wrote about this a month or two ago and didn't really get a response but it's starting to bite me again. I'm trying to store LDAP connection objects in the user session and because the underlying LDAP objects (from the python-ldap.sf.net modules) aren't "pickleable", on shutdown of the AppServer, an error is thrown and on top of that the AppServer I guess writes the Session to disk anyway (which it probably shouldn't) and then when I try and restart the AppServer, it exits with the following error: File "./WebKit/SessionMemoryStore.py", line 23, in __init__ File "./WebKit/SessionFileStore.py", line 55, in __getitem__ KeyError: 20021202101928-72a11220b9961fe879564b63447a8d6a My problem that I don't get is why when I tell the AppServer to use a MemoryStore model, it still writes to disk. I don't want my sessions to persist after their timeout. There are a million reasons not to want this persistence one of which is that if your objects aren't pickleable, it crashes. I think this should be addressed in the next release as I find it hard to understand the reason behind wanting to store ALL sessions to disk on shutdown or timeout. BTW, in case people aren't familiar with LDAP, I can't just pool the objects like with DBPool because LDAP access control is based upon the unique username/password. Thanks for any help. Mike |
From: Ian B. <ia...@co...> - 2002-12-03 04:15:57
|
On Mon, 2002-12-02 at 12:07, Michael Engelhart wrote: > Hi all - > > I wrote about this a month or two ago and didn't really get a response > but it's starting to bite me again. > I'm trying to store LDAP connection objects in the user session and > because the underlying LDAP objects (from the python-ldap.sf.net > modules) aren't "pickleable", on shutdown of the AppServer, an error is > thrown and on top of that the AppServer I guess writes the Session to > disk anyway (which it probably shouldn't) and then when I try and > restart the AppServer, it exits with the following error: > File "./WebKit/SessionMemoryStore.py", line 23, in __init__ > File "./WebKit/SessionFileStore.py", line 55, in __getitem__ > KeyError: 20021202101928-72a11220b9961fe879564b63447a8d6a Yes, all types of session stores are persistent -- the only configuration is when they store stuff to disk (immediately, lazily, or on AppServer shutdown). This configuration is really for performance tweaking, it does not introduce any real change in the behavior of the session. I think this is good. > My problem that I don't get is why when I tell the AppServer to use a > MemoryStore model, it still writes to disk. I don't want my sessions > to persist after their timeout. There are a million reasons not to > want this persistence one of which is that if your objects aren't > pickleable, it crashes. > > I think this should be addressed in the next release as I find it hard > to understand the reason behind wanting to store ALL sessions to disk > on shutdown or timeout. I think Webware encourages a more robust application through this behavior, but I can also appreciate that this seems somewhat paternalistic -- it's your app, after all. But I really don't think it should be so hard. You could use some sort of pool, paramaterized on the username/password. If your LDAP access is threadsafe, you could use: http://colorstudy.com/software/python/ParamFactory.py Like: ## AppLDAP.py: from ParamFactory import ParamFactory def createLDAPConnection(username, password): # yada yada LDAPConnection = ParamFactory(createLDAPConnection) ## SitePage.py: import AppLDAP class SitePage(Page): # yada yada def ldapConnection(self): ses = self.session() return AppLDAP.LDAPConnection(ses.value('username'), ses.value('password')) ## end SitePage If your LDAP connections aren't threadsafe, you'd need some sort of pool that was paramaterized, something like DBPool except slightly more general. Off the top of my head (hence untested): class Pool: """ ``creator`` must be callable, and require only positional (not keyword) arguments. The arguments must be hashable, i.e., non-mutable (e.g., strings, numbers, tuples) """ def __init__(self, creator, threadsafe=0): self._creator = creator self._pool = {} if threadsafe: self.__call__ = self._threadsafe__call__ else: self.__call__ = self._unthreadsafe__call__ self.returnConnection = self._unthreadsafe_returnConnection self._lock = threading.Lock() def _threadsafe__call__(self, *args): try: return self._pool[args] except KeyError: self._pool[args] = self._creator(*args) return self._pool[args] def _unthreadsafe__call__(self, *args): self._lock.acquire() try: if not self._pool.has_key(args): self._pool[args] = [] if not self._pool[args]: self._pool[args].append(PooledObject(self._unthreadsafe_returnObject, args, self._creator(*args)) return self._pool[args].pop() finally: self._lock.release() def _unthreadsafe_returnObject(self, args, obj): self._pool[args].append(obj) class PooledObject: def __init__(self, callback, args, obj): self._callback, self._args = callback, args self._obj = obj def close(self): if self._pool and self._obj is not None: self._callback(args, self._obj) self._obj = None def __del__(self): self.close() ## End There are a multitude of improvements that could be made, but this should be enough for most pooling needs. You can use it in your SitePage similarly to ParamFactory. -- Ian Bicking Colorstudy Web Development ia...@co... http://www.colorstudy.com PGP: gpg --keyserver pgp.mit.edu --recv-keys 0x9B9E28B7 4869 N Talman Ave, Chicago, IL 60625 / (773) 275-7241 |
From: Michael E. <men...@ka...> - 2002-12-03 04:46:34
|
Hi Ian, Thanks for the code snippets and advice. But I'm afraid my issue with this is more than just my pickling problem with my LDAP connection objects. It's also the problem I have with Webware having 3 types of session storage that all write to disk at some point. By forcing MemorySessionStores to write to disk (even if it is just on shutdown), Webware has effectively blocked out any way to allow developers to "not" keep session's persistent between app server restarts. I guess I just don't see the need for 3 session storage patterns that effectively all do the same thing at some point or another. Why not just have DynamicSessionStore's and be done with it? There's also a serious security issue with storing a session that may have sensitive information in it onto a pickled file that may reside on a server for a long time. What if I need to store a password or credit card number in a session and it's gets written to disk - that's a BAD thing. I realize that you can code carefully around this but I'm convinced we should have to as Webware developers? My workaround to suit my needs was to comment out the code in the storeAllSessions(self) method in MemorySessionStore.py. Now things work exactly as I think they should (which by the way mimics the default functionality of Tomcat, Jetty, JBoss and all the other open source Java Servlet containers). So now I have "choice" in how I store my sessions - I can write to disk or not write to disk just by setting a flag in my Application.config file. Anyway, that's my case.. sorry for being so long-winded. :-) Thanks again Mike On Monday, December 2, 2002, at 11:17 PM, Ian Bicking wrote: > On Mon, 2002-12-02 at 12:07, Michael Engelhart wrote: >> Hi all - >> >> I wrote about this a month or two ago and didn't really get a response >> but it's starting to bite me again. >> I'm trying to store LDAP connection objects in the user session and >> because the underlying LDAP objects (from the python-ldap.sf.net >> modules) aren't "pickleable", on shutdown of the AppServer, an error >> is >> thrown and on top of that the AppServer I guess writes the Session to >> disk anyway (which it probably shouldn't) and then when I try and >> restart the AppServer, it exits with the following error: >> File "./WebKit/SessionMemoryStore.py", line 23, in __init__ >> File "./WebKit/SessionFileStore.py", line 55, in __getitem__ >> KeyError: 20021202101928-72a11220b9961fe879564b63447a8d6a > > Yes, all types of session stores are persistent -- the only > configuration is when they store stuff to disk (immediately, lazily, or > on AppServer shutdown). This configuration is really for performance > tweaking, it does not introduce any real change in the behavior of the > session. I think this is good. > >> My problem that I don't get is why when I tell the AppServer to use a >> MemoryStore model, it still writes to disk. I don't want my sessions >> to persist after their timeout. There are a million reasons not to >> want this persistence one of which is that if your objects aren't >> pickleable, it crashes. >> >> I think this should be addressed in the next release as I find it hard >> to understand the reason behind wanting to store ALL sessions to disk >> on shutdown or timeout. > > I think Webware encourages a more robust application through this > behavior, but I can also appreciate that this seems somewhat > paternalistic -- it's your app, after all. > > But I really don't think it should be so hard. You could use some sort > of pool, paramaterized on the username/password. > > If your LDAP access is threadsafe, you could use: > http://colorstudy.com/software/python/ParamFactory.py > > Like: > > ## AppLDAP.py: > > from ParamFactory import ParamFactory > > def createLDAPConnection(username, password): > # yada yada > > LDAPConnection = ParamFactory(createLDAPConnection) > > ## SitePage.py: > > import AppLDAP > > class SitePage(Page): > # yada yada > > def ldapConnection(self): > ses = self.session() > return AppLDAP.LDAPConnection(ses.value('username'), > ses.value('password')) > > ## end SitePage > > > If your LDAP connections aren't threadsafe, you'd need some sort of > pool > that was paramaterized, something like DBPool except slightly more > general. Off the top of my head (hence untested): > > class Pool: > """ > ``creator`` must be callable, and require only positional (not > keyword) arguments. The arguments must be hashable, i.e., > non-mutable (e.g., strings, numbers, tuples) > """ > def __init__(self, creator, threadsafe=0): > self._creator = creator > self._pool = {} > if threadsafe: > self.__call__ = self._threadsafe__call__ > else: > self.__call__ = self._unthreadsafe__call__ > self.returnConnection = self._unthreadsafe_returnConnection > self._lock = threading.Lock() > > def _threadsafe__call__(self, *args): > try: > return self._pool[args] > except KeyError: > self._pool[args] = self._creator(*args) > return self._pool[args] > > def _unthreadsafe__call__(self, *args): > self._lock.acquire() > try: > if not self._pool.has_key(args): > self._pool[args] = [] > if not self._pool[args]: > > self._pool[args].append(PooledObject(self._unthreadsafe_returnObject, > args, > self._creator(*args)) > return self._pool[args].pop() > finally: > self._lock.release() > > def _unthreadsafe_returnObject(self, args, obj): > self._pool[args].append(obj) > > class PooledObject: > def __init__(self, callback, args, obj): > self._callback, self._args = callback, args > self._obj = obj > def close(self): > if self._pool and self._obj is not None: > self._callback(args, self._obj) > self._obj = None > def __del__(self): > self.close() > > ## End > > > There are a multitude of improvements that could be made, but this > should be enough for most pooling needs. You can use it in your > SitePage similarly to ParamFactory. > > -- > Ian Bicking Colorstudy Web Development > ia...@co... http://www.colorstudy.com > PGP: gpg --keyserver pgp.mit.edu --recv-keys 0x9B9E28B7 > 4869 N Talman Ave, Chicago, IL 60625 / (773) 275-7241 > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Get the new Palm Tungsten T > handheld. Power & Color in a compact size! > http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en > _______________________________________________ > Webware-discuss mailing list > Web...@li... > https://lists.sourceforge.net/lists/listinfo/webware-discuss > |