[SQLObject] Slow/Large cache

[Gert B. <ger...@gm...>]

Hi

We have a RPC server which CPU usage increases over time, after a week
it will max out one core.

>From my testing(Using pyrasite's dump stack) it seems that all the time
is spend in:

file "/usr/lib/pymodules/python2.6/turbogears/database.py", line 275, in
commit
    self.threadingLocal.connection.commit()
  File
"/usr/local/lib/python2.6/dist-packages/SQLObject-1.3.1-py2.6.egg/sqlobject/dbconnection.py",
line 802, in commit
    subCaches = [(sub[0], sub[1].allIDs()) for sub in
self.cache.allSubCachesByClassNames().items()]
  File
"/usr/local/lib/python2.6/dist-packages/SQLObject-1.3.1-py2.6.egg/sqlobject/cache.py",
line 263, in allIDs
    for id, value in self.expiredCache.items():

Currently the overhead for each commit and rollback is around 2-3 seconds.

Almost all of our tables have caching disabled via 'sqlmeta: cacheValues
= False'.

No state is kept in memory between requests and we typically write
significantly more rows to the DB than what is read, ie. A lot of writes
which are not read often and a few hundred rows which are read often and
that can be changed by other processes.

Does anyone have a suggestion on how to mitigate or further debug the
problem?

Regards
Gert Burger