#29 COOL instances vs memory usage & environment rule triggering

[JFGuilbert]

Some various test scenarios in the attachment shows
that some instances that are explicitelly deleted seems
to still be in memory. A CLIPS test case equivalent is
also provided.

Also, the usage of COOL instances in rules don't seem
to trigger well with symbols conditions when using a
clips environment.

See <testscenario> file in tar.gz attachment for more
details on each topic.

[JFGuilbert]

PyClips COOL test scenarios

[Francesco Garosi]

Logged In: YES
user_id=328337

Hi!

I just received a notification about this bug - sorry for
the other one, but I've not been able to reproduce it: it
can really take me some time.

Regarding the first part - memory occupation - I think that
the origin of the problem may reside in the way CLIPS
objects are mapped to Python ones - even though for what I
see, you mostly create objects directly in CLIPS instead of
using the Build* functions. Probably there is a reference
counter somewhere that does not drop to zero: I will start
investigating right now. Just for the note: unfortunately
the Memory.Free() function is not of much help in freeing
real used memory, as it just reclaims buffers used for
pretty-printing. However the fact that COOL objects are not
correctly destroyed warns me of a poorly designed object
manager hidden somewhere.

As of the other issue (symbol matching behaving oddly when
using an environment), it looks also strange to me. Too bad,
it cannot be reproduced in a pure CLIPS session - as it does
not support environments.

Thank you for the detailed test cases, I think I've got some
stuff to work on.

Cheers,

F.

[Francesco Garosi]

Logged In: YES
user_id=328337

This is quite weird:

[BEGIN CODE]
## Here the file bstar.clp contains the following:
##
## (defclass TestOOCls (is-a USER)(slot a1))
## (make-instance tst of TestOOCls)
## (defrule testRule1
## (object (is-a TestOOCls) (a1 ?a1&HELLO))
## =>
## (printout t ?a1))

import clips
env = clips.Environment()
env.BatchStar("bstar.clp")
tst = env.FindInstance("tst")
#tst.Slots['a1'] = clips.Symbol("HELLO")
env.Eval("(active-modify-instance tst (a1 HELLO))")
print tst.PPForm()
env.Run()
print clips.StdoutStream.Read()

## this will output HELLO, as expected
[END CODE]

where the "active-modify-instance" function is supposed to
use the low-level DirectPutSlot function (as the Slots[]
pseudo-dictionary does). The Eval() happens at environment
level, thus should have the same effect. Next step will
consist of examining the active-modify-instance command
definition in the CLIPS source.

Maybe I will have to patch CLIPS itself to correct this
behaviour.

F.

[Francesco Garosi]

Logged In: YES
user_id=328337

Partial good news. I found that the file "clipsmodule.c" was
actually using non-environment functions to assign values to
instance slots. It seemed that it would have been the same,
but looking at the CLIPS source the DATA_OBJECT assignment
functions work differently when environment-aware or not.
This solves the problem of matching objects by symbols
assigned to slots (see COOL_Env_test1.py in the test case).

I prepared a patch using subversion, to be applied to the
pyclips-1.0_R3 source base: it possibly would not work with
previous "R" versions. Please download (see below) and apply
clipsmodule.c.diff to the existing source: it will be
incorporated in the next CVS commit and in the next release.

I'm still looking for a solution to the second part of the bug.

Cheers,

F.

[Francesco Garosi]

Patch for environment problem (SVN)

[Francesco Garosi]

Logged In: YES
user_id=328337

Second part of "partial good news": i discovered why the
"Free" function does not release all the memory that would
be expected. The reason is, that many of the items that
CLIPS marks as garbage are actually not released due to the
use of heuristics while performing the periodic cleanup.
This happens because, if there are any rules executing, they
might need to access the data of some of the invalid entities.

Probably the CLIPS shell, when returning to the prompt,
performs a periodic cleanup, and uses heuristics if there
are no more rules executing, and avoids heuristics otherwise.

This could actually not be solved using the documented CLIPS
API. However the latest CVS head implements a low-level
function (namely forceCleanup/env_forceCleanup) that allows
the user to attempt to force a garbage collection, possibly
avoiding heuristics. If you recompile the latest CVS, print
the function documentation to see how it can be used.

As a test, you can add the following lines to each of the
COOL_Mem_test?.py files:

clips._clips.forceCleanup(True, False)
clips.Memory.Free ( )
print "Memory After C/Free: ", clips.Memory.Used

You will notice that the result in the last line of output
is close to the expected one. Since this function is not
described in the "official" CLIPS API, I will probably not
implement it in the high-level interface.

However, as it tests whether or not any rule is executing
before actually trying to perform the garbage collection (it
raises an exception in case of rule execution), the use of
this utility should be safe in a production environment.

In my opinion this should close the bug.

Cheers,

F.

[Francesco Garosi]

Logged In: YES
user_id=328337

The fixes for this bug or request have been accepted and
committed to current CVS tree: next release will include these
fixes, possibly among other enhancements.