Thread: [SQLObject] Expiring the cache
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
From: Frank B. <fb...@fo...> - 2003-04-15 17:44:19
|
Hallo, I'm haveing some problems with changed SQLObjects in Webware. Too often the database itself is changed when I change an object, but the cache is not expired. In Webware, being a threaded system, I then get invalid data in other threads. As I only change objects in one place, I would love to be able to let the cache expire by hand there, with something like "mySqlobject.cache.expire()" Is this possible - or even a good idea? ciao -- Frank Barknecht _ ______footils.org__ |
From: Ian B. <ia...@co...> - 2003-04-16 21:06:13
|
On Tue, 2003-04-15 at 12:43, Frank Barknecht wrote: > Hallo, > > I'm haveing some problems with changed SQLObjects in Webware. Too > often the database itself is changed when I change an object, but the > cache is not expired. In Webware, being a threaded system, I then get > invalid data in other threads. > > As I only change objects in one place, I would love to be able to let > the cache expire by hand there, with something like > "mySqlobject.cache.expire()" I was thinking of something like that, only calling it .refresh(). What bothered me was if you do this at, say, the beginning of a request you may be confused when some other request does the same thing, and the object gets refreshed in the middle of your request. I'm thinking there needs to be some consolidation of the different ways instances act with respect to threads. There's a few use cases I can see: * You want a fresh instance at the beginning of your "transaction" (not necessarily a database transaction). You don't want that instance changing underneath you. * You want a live copy of the object, but only live for the process you're in. So all threads share a copy, but multiple processes could go out of sync. Works well with Webware when its the only user of a database, but I'm not sure outside of that. * You want a instance that's associated with a database transaction, with whatever semantics that implies. Obviously you may have multiple copies of the object, since multiple transactions could be occurring. * You want a live instance of the object, attached directly to the database. No caching should occur. There's no reason not to share this instance between threads, as far as I can see. More cases? Is there a way all these cases can be expressed more succinctly (like a couple options that can be combined in different ways)? Ian |
From: Frank B. <fb...@fo...> - 2003-04-16 22:18:36
|
Hallo, Ian Bicking hat gesagt: // Ian Bicking wrote: > I was thinking of something like that, only calling it .refresh(). What > bothered me was if you do this at, say, the beginning of a request you > may be confused when some other request does the same thing, and the > object gets refreshed in the middle of your request. > > I'm thinking there needs to be some consolidation of the different ways > instances act with respect to threads. There's a few use cases I can > see: [...] > * You want a live instance of the object, attached directly to the > database. No caching should occur. There's no reason not to share this > instance between threads, as far as I can see. Maybe the last one might be what I currently need. Caching is okay (for performance reasons probably needed), but I need changes to be visible on all Pages. But then maybe the problem _I_ have has nothing to do with the caching, I'm not sure. I just checked in the application I'm working on into the webware-sandbox under Sandbox/fbar/survey, so if you'd like to take a look? It's a survey we want to do with the visitors of our website. Edit-pages will let our editors add and change the questions and their answers. Now I can change the questions without problems, changes are immediatly visible. But not so the answers, which get pulled from the DB per joins over the Questions. That is, a Question has multiple answers: Question "q" Since when do you know us? Answers "q.answers" [ ] 1996 [ ] 1997 ... [ ] 20022 [ ] 2003 The "Answers" tree is build via "for a in q.answers: ..." Now if I change the answers in a web form, to correct the "20022" mistake, this is seen immediatly in the database, but even with strong browser reloads (Ctl-Shift-Reload,... ;) it is not seen in the Pages. ciao -- Frank Barknecht _ ______footils.org__ |
From: Ian B. <ia...@co...> - 2003-05-05 19:33:56
|
I think I fixed the caching problem you've been having -- there was some old code based on the lazy fetching of columns that was causing a problem. I fixed it just after cvs.sf.net went down... I'll give you an update when I get it put in. Ian |
From: Frank B. <fb...@fo...> - 2003-05-05 20:27:42
|
Hallo, Ian Bicking hat gesagt: // Ian Bicking wrote: > I think I fixed the caching problem you've been having -- there was some > old code based on the lazy fetching of columns that was causing a > problem. Thanks, Ian. I'll give it a try when it's online. ciao -- Frank Barknecht _ ______footils.org__ |
From: Frank B. <fb...@fo...> - 2003-05-06 09:58:13
|
Hallo Ian, Ian Bicking hat gesagt: // Ian Bicking wrote: > I think I fixed the caching problem you've been having -- there was some > old code based on the lazy fetching of columns that was causing a > problem. I did an update but I still get the same wrong behaviour, when I turn caching on. To clarify my problem: It only (AFAIK) affects Joins. To give an example using the people.py: A Page should make editing the PhoneNumbers possible: pid = getPersonIDSomehow() p = Person(pid) for number in p.phoneNumbers: self.writeln(""" <form> <input type="hidden" name="NumberID" value="%d"> <input type=input name="PhoneNumber" value="%s"> <input type=input name="Type" value="%s" <input type=submit name="_action_Change" value="Change"> </form> """ % (number.id, number.PhoneNumber, number.PhoneType) ) def Change(self): req = self.request() id = int(req.value('NumberID', None)) num = req.value('PhoneNumber', None) type = req.value('Type', None) p = PhoneNumber(id) p.phoneNumber = num p.phoneType = type If Change() is called, the database actually gets updated, but if the edit form is showed in the response, it doesn't reflect the changes in the list of PhoneNumbers. As I said, only Joins like p.phoneNumbers are affected by this. I can, and always could edit "Person"s directly without the caching errors, only their Phones are messed up. ciao -- Frank Barknecht _ ______footils.org__ |
From: Ian B. <ia...@co...> - 2003-05-06 21:03:47
|
On Tue, 2003-05-06 at 04:57, Frank Barknecht wrote: > I did an update but I still get the same wrong behaviour, when I turn > caching on. To clarify my problem: > > It only (AFAIK) affects Joins. To give an example using the people.py: > A Page should make editing the PhoneNumbers possible: Hmm... can you create a unit test that has this bug in it? I'd added a test (JoinTest2) to test.py in which I try to do this, but there's no bug happening. There's nothing cached in joins, so I'm not sure why this would be happening. Ian |
From: Frank B. <fb...@fo...> - 2003-05-07 08:54:46
Attachments:
JoinCreate.py
JoinTest.py
|
Hallo, Ian Bicking hat gesagt: // Ian Bicking wrote: > Hmm... can you create a unit test that has this bug in it? I'd added a > test (JoinTest2) to test.py in which I try to do this, but there's no > bug happening. Yes, this test looks okay, but is it running multithreaded or in Webware? Attached is my simple Context for Webware. Here the bug happens. First run JoinCreate.py, then call JoinTest as a Webware page and try to change some entries. ciao -- Frank Barknecht _ ______footils.org__ |
From: Frank B. <fb...@fo...> - 2003-05-07 09:23:30
|
Hallo, Frank Barknecht hat gesagt: // Frank Barknecht wrote: > Attached is my simple Context for Webware. Here the bug happens. Ok, major bugs in my scripts: "Change" wasn't called. I now fixed this and, well, now it works correct even with joins. Duh. I won't sent the fixed version, instead I'll first try my older code to see if I can reproduce the bug in the first place. Give me a day... ciao -- Frank Barknecht _ ______footils.org__ |
From: Ian B. <ia...@co...> - 2003-05-07 13:29:24
|
On Wed, 2003-05-07 at 04:22, Frank Barknecht wrote: > Ok, major bugs in my scripts: "Change" wasn't called. I now fixed this > and, well, now it works correct even with joins. Duh. I won't sent the > fixed version, instead I'll first try my older code to see if I can > reproduce the bug in the first place. Give me a day... Okay... hence the request for a unit test :) I don't think the multi-threaded nature should really cause a problem, or at least not this problem, because everything's actually happening serially. Ian |
From: Frank B. <fb...@fo...> - 2003-05-07 15:16:57
|
Hallo, Ian Bicking hat gesagt: // Ian Bicking wrote: > Okay... hence the request for a unit test :) I don't know, how do write real unit tests ;( > I don't think the multi-threaded nature should really cause a > problem, or at least not this problem, because everything's actually > happening serially. Well, the code that still fails is an application that I have in the Webware-Sandbox "survey". It's not the current version, but I'll check that in if I'm back home again... ciao -- Frank Barknecht _ ______footils.org__ |
From: Ian B. <ia...@co...> - 2003-05-07 17:49:44
|
On Wed, 2003-05-07 at 10:16, Frank Barknecht wrote: > Hallo, > Ian Bicking hat gesagt: // Ian Bicking wrote: > > > Okay... hence the request for a unit test :) > > I don't know, how do write real unit tests ;( Look at the tests in tests/test.py, JoinTest2 being the obvious starting point for your problem (maybe adding another method to it). Unit tests remove other dependencies and the possibility of the bug lying elsewhere in the system. All it comes down to is isolating the failure case, at which time I expect the problem will be easy to identify. Short of a proper unit test, a self-contained script that exemplifies the problem will do (and is probably easy enough to convert to a unit test). Ian |