Duplicating an environment

[Fred]

Hi Gary,

I'm trying to duplicate an environment in an efficient way (using CLIPS 6.24)
I have an environment that contains 26000 facts.
I would like to duplcate these facts in a new environment.

At the moment, here's the way I'm proceeding:

/* initialize the new environment */
theEnv=CreateEnvironment();
iRet=EnvBload(theEnv, rule_file);

/* get facts from the common env and send them to the new one */
DATA_OBJECT rv;
int end, i;
void *FactPtr = NULL;
EnvGetFactList(CommonWksEnv, &rv, NULL);
end=EnvGetpDOEnd(CommonWksEnv, &rv);
for(i=EnvGetpDOBegin(CommonWksEnv, &rv); i<=end; i++) {
char *par = NULL;
char assert_fact;
FactPtr = EnvGetMFValue(CommonWksEnv, EnvGetpValue(CommonWksEnv, &rv), i);
EnvGetFactPPForm(CommonWksEnv,assert_fact,2048,FactPtr);
par = strchr(assert_fact, '(');
EnvAssertString(theEnv,par);
}

This code works fine, but it is a bit too slow : it takes 1,7 seconds to duplicate 26000 facts.
Is there a more efficient way to copy these facts?
I insist on the fact that this is only a copy of the common environment. The added facts to not trigger any rules.
I'm currently using CLIPS 6.24, but if a solution exists within the 6.30, I will migrate. This performance issue is quite important to us.

Thanks for your help.
Fred

[Gary Riley]

I think you can use some of the internal code as a model along with some of the documented API calls rather than converting each fact to a string and then parsing it again. The DuplicateModifyCommand in the file tmpltfun.c has some snippets that would be useful. This piece would be the core:

  
   templatePtr = oldFact->whichDeftemplate;
   newFact = (struct fact *) CreateFactBySize(theEnv,oldFact->theProposition.multifieldLength);
   newFact->whichDeftemplate = templatePtr;
   for (i = 0; i < (int) oldFact->theProposition.multifieldLength; i++)
     {
      newFact->theProposition.theFields[i].type = oldFact->theProposition.theFields[i].type;
     
      // Need to copy the value here
     }
   EnvAssert(theEnv,newFact);

In copying the values from the old environment to the new, you can't directly copy the values, but instead you'd have to use the APIs from the Advanced Programming Guide (section 4.4.4 has some examples). It would look like the following, but each of the primitive data types (integers, floats, strings, symbols, multifields, …) would have a section of code for copying it:

switch(newFact->theProposition.theFields[i].type)
     {
      case INTEGER:
        newFact->theProposition.theFields[i].value = EnvAddLong(theEnv,ValueToLong(oldFact->theProposition.theFields[i].value));
        break;
      case FLOAT:
        newFact->theProposition.theFields[i].value = EnvAddDouble(theEnv,ValueToDouble(oldFact->theProposition.theFields[i].value));
        break;

[Fred]

Hi Gary,
thanks for the quick reply.
I'll try this out and keep you informed.

[Fred]

Hi Gary,

I am trying to figure out how to parse my initial fact, and I'm stuck on one point : how do I parse multifields?
I have a template that looks like this:
(deftemplate my_fact
(slot factName)
(multislot otherData)
)
The 'otherData' part is either empty or it may contain strings.

I've started wrting my copy function and it looks like this:

DATA_OBJECT rv;
int end, i;
struct fact *FactPtr = NULL;;
EnvGetFactList(CommonWksEnv, &rv, NULL);

end=EnvGetpDOEnd(CommonWksEnv, &rv);

for(i=EnvGetpDOBegin(CommonWksEnv, &rv); i<=end; i++) {
char *par = NULL;
char assert_fact;
int j=0;
int size=0;
struct fact *newFact = (struct fact *) CreateFactBySize(theEnv,FactPtr->theProposition.multifieldLength);
FactPtr = (struct fact*)EnvGetMFValue(CommonWksEnv, EnvGetpValue(CommonWksEnv, &rv), i);
newFact->whichDeftemplate = FactPtr->whichDeftemplate;
for (j = 0; j < (int) FactPtr->theProposition.multifieldLength; j++) {
struct multifield *theList;
DATA_OBJECT rv2;
newFact->theProposition.theFields.type = FactPtr->theProposition.theFields.type;
switch(newFact->theProposition.theFields.type) {
case SYMBOL:
case STRING:
newFact->theProposition.theFields.value = EnvAddSymbol(theEnv,ValueToSymbol(oldFact->theProposition.theFields.value));
break;
case MULTIFIELD:
// How do I handle this?
break;
}
}
EnvAssert(theEnv,newFact);
}

Can you confirm that the STRING and SYMBOL are managed in the same way (i.e. with an AddSymbol)?
And how would you handle the multifield part?

Thanks again.
Fred

[Gary Riley]

Here's the basic structure for copying the old multifield value to the new environment. The GetMF… and SetMF… functions are described in sections 3.2.4 and 3.3.5 of the Advanced Programming Guide.

      struct multifield *oldMFV = oldFact->theProposition.theFields[i].value;
      struct multifield *newMFV = EnvCreateMultifield(theEnv,oldMFV->multifieldLength); 
      for (k = 0 ; k < (int) oldMFV->multifieldLength; k++)
         {
          SetMFType(newMFV,k,GetMFType(oldMFV,k));
          
          switch(GetMFType(oldMFV,k))
            {
             case STRING:
             case SYMBOL:
               SetMFValue(newMFV,k,EnvAddSymbol(theEnv,ValueToString(GetMFValue(oldMFV,k))));
               break;
            }
         }
      newFact->theProposition.theFields[j].value = newMFV;

STRING, SYMBOL, and INSTANCE_NAME all use AddSymbol for creating the value.

I'd also suggest using the GetNextFact API function to iterate over the fact list. Calling GetFactList is going to allocate a multifield to contain all of the values, which you really don't need.

[Fred]

Hi Gary,

here's my new try…
I did as you suggested : using the GetNextFact API. By the way, what is the way to check if a fact is the initial fact (I bet the way checked this is not the best one… ;) )

The problem, here is that the EnvAssert does not seem to do work properly : after having duplicated all my facts, the new environment remains empty (apart from the initial-fact)
I must obviously be doing something wrong since a DestroyEnvironment on the new environment crashes.

void *theEnv=CreateEnvironment();
EnvBload(theEnv, rule_file);
EnvReset(theEnv);
struct fact *oldFact = NULL;
for (oldFact = (struct fact *) EnvGetNextFact(CommonWksEnv,NULL); oldFact != NULL; oldFact = (struct fact *) EnvGetNextFact(CommonWksEnv,oldFact)) {
    int j=0;
    // skip the initial-fact...
    if((int) oldFact->theProposition.multifieldLength == 0) continue;
    struct fact *newFact = (struct fact *) CreateFactBySize(theEnv,oldFact->theProposition.multifieldLength);
    newFact->whichDeftemplate = oldFact->whichDeftemplate;
    for (j = 0; j < (int) oldFact->theProposition.multifieldLength; j++) {
        newFact->theProposition.theFields[j].type = oldFact->theProposition.theFields[j].type;
        switch(newFact->theProposition.theFields[j].type) {
            case SYMBOL:
            case STRING:
                newFact->theProposition.theFields[j].value = EnvAddSymbol(theEnv, ValueToString(oldFact->theProposition.theFields[j].value));
                break;
            case MULTIFIELD:
            {
                int k=0;
                struct multifield *oldMFV = (struct multifield *)oldFact->theProposition.theFields[j].value;
                struct multifield *newMFV = (struct multifield *)EnvCreateMultifield(theEnv,oldMFV->multifieldLength);
                for (k = 1 ; k <= (int) oldMFV->multifieldLength; k++) {
                    SetMFType(newMFV,k,GetMFType(oldMFV,k));
                    switch(GetMFType(oldMFV,k)) {
                        case STRING:
                        case SYMBOL:
                            SetMFValue(newMFV,k,EnvAddSymbol(theEnv,ValueToString(GetMFValue(oldMFV,k))));
                            break;
                    }
                }
                newFact->theProposition.theFields[j].value = newMFV;
            }
                break;
            default:
                break;
        }
    }
    EnvAssert(theEnv,newFact);
}

Any idea?
Thanks again for your help.

[Gary Riley]

You don't need to explicitly check for the initial-fact. If it already exists, it won't be asserted again. The test you're using will not only block the initial-fact, but any other fact that doesn't have any slots. Do any of your facts contain numbers? The code you've posted won't copy either integers or floats contained in the fact slots.

[Fred]

Hi Gary,

My facts only contain strings and symbols. That's why I didn't bother testing for any other type.

Any idea why the environment remains empty? And why destroying the new environment (DestroyEnvironment) crashes.

For the initial-fact issue: asserting a fact without having initialized it's value crashes. What should I create the new fact if the master (old) fact has no slots (e.g. like the initial fact)

Thanks again.
Fred

[Gary Riley]

Found it. This line:

newFact->whichDeftemplate = oldFact->whichDeftemplate;

Should be this:

newFact->whichDeftemplate = EnvFindDeftemplate(theEnv,EnvGetDeftemplateName(CommonWksEnv,oldFact->whichDeftemplate));

The deftemplate in the new environment needed to be assigned rather than the one from the old environment.

[Fred]

Hi Gary,

It works much better now :) And duplicating the environment is much more faster now (0.1 sec against 1.7 before).
There is still one problem : when I call the DestroyEnvironment method on the newly created environment it crashes.
Could you try out the following sequence please?
1) create a template (deftemplate my_fact (slot factName) (multislot otherData) )
2) create a fact in the commonEnvironment
3) create a new environment by duplicating the commonEnvironment (using the above code)
4) call DestroyEnvironment on the new environment => this crashes trying to illegaly free some memory…

The crash does not occur it I populate facts that obey to a deftemplate that does not contain multislots. It only occurs with multislots… there must be something wrong here…

Thanks.

[Fred]

Gary,

while debugging the code, I noticed that the address returned by the EnvCreateMultifield call overlaps some part of the memory initialized by the EnvReset function.
Is this normal?
Because the consequence is memory getting freed twice when I call DestroyEnvironment… should I add something in my code?

Thanks again.

[Gary Riley]

Replace the line

    newMFV = (struct multifield *) EnvCreateMultifield(theEnv,oldMFV->multifieldLength);

with

    newMFV = (struct multifield *) CreateMultifield2(theEnv,oldMFV->multifieldLength);

[Fred]

Gary,
It works perfectly now… and with a real speed improvement.

Thanks again !!!