From: Steve S. <st...@ag...> - 2010-03-15 19:07:44
|
Hi, I don't recall if I've already submitted this enhancement... I've written a memcached based Session that others might find useful. Feel free to use as you wish. Usage: 1. Name the file SessionMemcachedStore.py in WebKit directory 2. Configure your Application.config: SessionStore = 'WebKit.SessionMemcachedStore' SessionMemcachedServers = ['127.0.0.1:11211'] SessionTimeout = 180 # minutes Best Regards, Steve import threading from memcache import memcache from WebKit.SessionStore import SessionStore debug = False class SessionMemcachedStore(SessionStore, threading.local): """A session memached store. Stores the sessions in a single memcached using 'last write wins' semantics. This is useful for various situations: 1. Fault tolerance 2. Clustering In clustering configurations with concurrent writes for the same session(s) the last writer will always overwrite the session. Cleaning/timing out of sessions is performed by the memcached(s) themselves since no one appserver can know about the existence of all sessions or the last access for a given session. Besides it is built in memcached functionality. Consequently, correct sizing of memcached(s) is necessary to hold all user's session data. """ ## Init ## def __init__(self, app): threading.local.__init__(self) SessionStore.__init__(self, app) self._memcachedServers = app.setting( 'MemcachedServers', ['127.0.0.1:11211']) # timeout in seconds self._sessionTimeout = app.setting( 'SessionTimeout', 180) * 60 ## try: ## import cPickle as pickle ## except ImportError: ## import pickle import pickle self._client = memcache.Client(self._memcachedServers, pickleProtocol=pickle.HIGHEST_PROTOCOL) ## Access ## def __len__(self): if debug: print '>> len', len(self.keys()) return len(self.keys()) def __getitem__(self, key): if debug: print '>> getitem(%s)' % (key) # returns None if key non-existent or no server to contact value = self._client.get(key) if value is None: # SessionStore expects KeyError when no result raise KeyError return value def __setitem__(self, key, item): if debug: print '>> setitem(%s, %s)' % (key, item) try: val = self._client.set(key, item, time=self._sessionTimeout) if not val: raise Exception except Exception, e: # Not able to store the session is a failure print "Error saving session '%s' to memcached: %s" % (key, e) self.application().handleException() def __delitem__(self, key): """In contracts with SessionFileStore not finding a key to delete isn't a KeyError""" if debug: print '>> delitem(%s)' % (key) self._client.delete(key) def removeKey(self, key): pass def has_key(self, key): if debug: print ">> has_key(%s) " % (key) return self._client.get(key) is not None def keys(self): if debug: print ">> keys() " return [] def clear(self): """memcache handles clearing keys itself""" if debug: print ">> clear() " def setdefault(self, key, default): """return value associated with key otherwise set key to default and return default""" if debug: print '>> setdefault (%s, %s)' % (key, default) try: return self[key] except KeyError: self[key] = default return default ## Application support ## def storeSession(self, session): if debug: print ">> storeSession(%s) " % (session) key = session.identifier() self[key] = session def storeAllSessions(self): pass |
From: Christoph Z. <ci...@on...> - 2010-03-15 21:04:20
|
Am 15.03.2010 19:38 schrieb Steve Schwarz: > I don't recall if I've already submitted this enhancement... > I've written a memcached based Session that others might find useful. Thanks, Steve. I'll have a look and probably include it with the next Webware release (yes it got postponed but is not abandoned...) -- Christoph |
From: Christoph Z. <ci...@on...> - 2010-04-11 21:23:58
|
Am 15.03.2010 19:38 schrieb Steve Schwarz: > I've written a memcached based Session that others might find useful. > Feel free to use as you wish. I had some time today to try it out. Works nicely. I have added some improvements (namespaces, clear() and some more methods) and tests. Committed to the 1.1. branch now. Only one thing was not quite clear: Why do you inherit the store from threading.local? As far as I see, this should not be necessary. -- Christoph |
From: Steve S. <st...@ag...> - 2010-04-11 21:56:56
|
On Sun, Apr 11, 2010 at 4:23 PM, Christoph Zwerschke <ci...@on...> wrote: > Am 15.03.2010 19:38 schrieb Steve Schwarz: > > I've written a memcached based Session that others might find useful. > > Feel free to use as you wish. > > I had some time today to try it out. Works nicely. I have added some > improvements (namespaces, clear() and some more methods) and tests. > Committed to the 1.1. branch now. > > Only one thing was not quite clear: Why do you inherit the store from > threading.local? As far as I see, this should not be necessary. > > Christoph, Must have been a cut/paste error from the Session subclass on which I based this implementation; there is no need for threading.local. I'll take a look at your version. Steve |
From: Steve S. <st...@ag...> - 2010-04-11 22:59:48
|
On Sun, Apr 11, 2010 at 4:32 PM, Steve Schwarz <st...@ag...>wrote: > On Sun, Apr 11, 2010 at 4:23 PM, Christoph Zwerschke <ci...@on...>wrote: > >> Am 15.03.2010 19:38 schrieb Steve Schwarz: >> > I've written a memcached based Session that others might find useful. >> > Feel free to use as you wish. >> >> I had some time today to try it out. Works nicely. I have added some >> improvements (namespaces, clear() and some more methods) and tests. >> Committed to the 1.1. branch now. >> > Christoph, I took a look at the commit and I think the namespace idea is a good enhancement. We use separate memcached servers only for session data so it hasn't been an issue for us. However I'm concerned by the added code/complexity required to support clear(). In our usage of the memcached session store users are moved between servers based on load. So if the sessions are cleared on one appserver what mechanism is used to trigger the other servers to call getCounter() and reset their notion of the session counter? Similarly if an appserver restarts it will get the latest session namespace key but appservers that didn't call clear() won't have the latest session namespace key. I'd much rather the session store fail early and raise an exception if you try to clear(). The less code in the session implementation the safer I'd feel. Best Regards, Steve |
From: Christoph Z. <ci...@on...> - 2010-04-12 07:27:21
|
Am 12.04.2010 00:59 schrieb Steve Schwarz: > However I'm concerned by the added code/complexity required to support > clear(). In our usage of the memcached session store users are moved > between servers based on load. So if the sessions are cleared on one > appserver what mechanism is used to trigger the other servers to call > getCounter() and reset their notion of the session counter? Ok, I see the problem. It happens if you are using more than one Webware app server connecting to the same memcache, right? In that case we need to fetch the counter every time from memcache (which is currently not done) because the other appservers could have updated it, and that would be indeed a lot of overhead. I'll see if I can come up with a solution, but probably I will just remove the counter mechanism again. -- Christoph |
From: Christoph Z. <ci...@on...> - 2010-04-15 14:49:20
|
Am 12.04.2010 00:59 schrieb Steve Schwarz: > I'd much rather the session store fail early and raise an exception if > you try to clear(). The less code in the session implementation the > safer I'd feel. Ok, I have changed that now. You can configure whether you want an error, warning or ignore things unsupported by Memcached. The default is printing a warning. -- Christoph |