Hello:

[ I usually just lurk here, but this are 2 seemingly a small features to add...  ]

Question:  Can we explicitly minimize memory footprints by not caching objects we create?
Question:  Can we explicitly control the amount of cache used?

In a previous post replying to Peter Kropf, Simon Cross posted:

: Setting cacheValues=False does exactly what the name suggests -
: it prevents caching of the values (fields) associated with an object. 
: So with cacheValues=False obj.value retrieves value from the database.
: Explicitly removing an object from the object cache requires a call to obj.expire().
: If you're only creating objects (as I was) then things are only
: added to the cache (never fetched) and the cache
: grows indefinitely unless you manually expire objects.

I'd like to suggest a new SQLOBJECT feature that simply doesn't add newly inserted
Objects to the cache; this could be a simple sqlmeta property setting:

cacheInsertedValues=[True|False]

This would minimize the memory footprint somewhat, especially for programs that insert a lot. 

I'm too much of a newbie to implement this well, but I believe it would involve:

1.  Adding at line (approx) 357:
    ...
    _cacheValues = True

    # The _cacheInsertedValues attribute controls if you cache
    # values inserted into the database.  We make sure
    # it's set (default 1).
    _cacheInsertedValues = True

2.  Changing SQLObject.py line (approx) 913,

FROM:

def _SO_finishCreate(self, id=None):
    ...
    id = self._connection.queryInsertID(self._table, self._idName, id, names, values)
    cache = self._connection.cache
    cache.created(id, self.__class__, self)
    self._init(id)

TO:

def _SO_finishCreate(self, id=None):
    ...
    id = self._connection.queryInsertID(self._table, self._idName, id, names, values)
    if self._cacheInsertedValues:
        cache = self._connection.cache
        cache.created(id, self.__class__, self)
    self._init(id)


I'm intereseted in 3 things:

A.  This seems like a worthwhile feature for programs that scan for inputs, and insert if so;

B.  This might not be remotely the correct code to accomplish the task (don't know, can't test at the moment);

C.  It might be more prudent to put a cache size limit into cache instead. 

A cache size limit would seem more complicated in that there would
have to be size measurements of the SQLObjects.  Alas.  Another sqlmeta property
Created, as:
   
    _cacheValues = True            # existing
    _cacheInsertedValues = True    # new from above
    _cacheSizeLimit = 0            # 0 is unlimited

then, in Cache.__init__:

def __init__(self, cullFrequency=100, cullFraction=2, cache=True, cacheSize=0):   
    ...
    self.cacheSize = cacheSize

...
def get(self, id):
    if self.doCache:
        if self.cacheSize:
            amountNeeded = self.findSizeOfNewObject(id)
            while ((self.cacheUsed > 0) and (self.cacheUsed() + amountNeeded > self.cacheSize)):
                self.cullCount = 0
                self.cull()
        if self.cullCount > self.cullFrequency:
            # Two threads could hit the cull in a row, but
        ...

I don't know how to do the "def findSizeOfNewObject(self, id)" though.

Perhaps the first method (cacheInsertedValues=False) is a quick, simple
solution, but the latter one may build in a nicer feature set at the
expense of running slightly slower (computing sizes could be done in
an init and stored, but I'm not sure…)


-- Kevin Rice


___________________________________
Kevin J. Rice http://justanyone.com
Senior Software Engineer, Textura LLC http://texturallc.com
49-G Sherwood Terrace, Lake Bluff IL 
___________________________________