CLIPSNET CaptureRouter memory leaks?

[Davide Archetti]

Hi, I was trying to do this thing:
I wish to have the clips environment running in a loop in a separated thread and from the main thread send commands to the environment
The problem is that when everything is running, without doing nothing, the memory occupation start increasing about 1MB per second, and looking to a memory snapshot, it says that the object CLIPSNET.CaptureRouter is continuosly allocating instances of about 160 bytes but at a rate of about 5000 per second.
Below there is the minimal code to reproduce the problem: what I'm doing wrong?

//Program.cs
// See https://aka.ms/new-console-template for more information

using TestSistemaEsperto;

ClipsEngine clipsEnvironment = new ClipsEngine();

Task clipsTask = new Task(() =>
{
    try
    {
        clipsEnvironment.Loop("..\\..\\..\\RuleBase\\");
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.ToString());
    }
});

clipsTask.Start();

string line = string.Empty;

while (line != "stop")
{

    line = Console.ReadLine() ?? "stop";

    if (!string.IsNullOrWhiteSpace(line))
    {
        //clipsEnvironment.Commands.Enqueue(line);
    }

}

namespace TestSistemaEsperto
{
    internal class ClipsEngine
    {
        CLIPSNET.Environment clipsEnvironment = new CLIPSNET.Environment();

        public void Loop(string rulePath)
        {
            foreach (string file in Directory.GetFiles(rulePath, "*.clp").OrderBy(x => x))
            {
                //...
                System.IO.StreamReader sr = new System.IO.StreamReader(file);
                string all = sr.ReadToEnd();
                try
                {
                    clipsEnvironment.LoadFromString(all);
                }
                catch (Exception ex)
                {
                    throw;
                }
            }

            string cmd = string.Empty;
            while (cmd != "quit")
            {
                clipsEnvironment.Run(1);

                if (cmd != string.Empty)
                {
                    //ExecCommand(cmd);

                }

                //cmd = Commands.Dequeue();
            }
        }
    }
}

(defrule initialize
    (declare (salience 9000))
=>
    (printout t "initialize" crlf)

    (assert (KEEP-ALIVE))
)



(defrule loop
    (declare (salience -1000))
    (KEEP-ALIVE)
    (not (SHUTDOWN))
=>

    (printout t "Looping" crlf)
    ;(UpdateSomething)

    (refresh loop)
)

[Gary Riley]

Download these updated files:

CLIPSNET_Router.cpp
CLIPSNET_Router.cpp

From here:

https://sourceforge.net/p/clipsrules/code/HEAD/tree/branches/64x/windows/MVS/CLIPSCLRWrapper/Source/Integration/

There was an issue with a strong reference in the router code that prevented the .NET garbage collection from reclaiming memory. See if that fixes your issue.

Gary

On Feb 20, 2023, at 5:26 PM, Davide Archetti davide-archetti@users.sourceforge.net wrote:

Hi, I was trying to do this thing:
I wish to have the clips environment running in a loop in a separated thread and from the main thread send commands to the environment
The problem is that when everything is running, without doing nothing, the memory occupation start increasing about 1MB per second, and looking to a memory snapshot, it says that the object CLIPSNET.CaptureRouter is continuosly allocating instances of about 160 bytes but at a rate of about 5000 per second.
Below there is the minimal code to reproduce the problem: what I'm doing wrong?

//Program.cs
// See https://aka.ms/new-console-template for more information

using TestSistemaEsperto;

ClipsEngine clipsEnvironment = new ClipsEngine();

Task clipsTask = new Task(() =>
{
try
{
clipsEnvironment.Loop("..\..\..\RuleBase\");
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
});

clipsTask.Start();

string line = string.Empty;

while (line != "stop")
{

line = Console.ReadLine() ?? "stop";

if (!string.IsNullOrWhiteSpace(line))
{
    //clipsEnvironment.Commands.Enqueue(line);
}

}
namespace TestSistemaEsperto
{
internal class ClipsEngine
{
CLIPSNET.Environment clipsEnvironment = new CLIPSNET.Environment();

    public void Loop(string rulePath)
    {
        foreach (string file in Directory.GetFiles(rulePath, "*.clp").OrderBy(x => x))
        {
            //...
            System.IO.StreamReader sr = new System.IO.StreamReader(file);
            string all = sr.ReadToEnd();
            try
            {
                clipsEnvironment.LoadFromString(all);
            }
            catch (Exception ex)
            {
                throw;
            }
        }

        string cmd = string.Empty;
        while (cmd != "quit")
        {
            clipsEnvironment.Run(1);

            if (cmd != string.Empty)
            {
                //ExecCommand(cmd);

            }

            //cmd = Commands.Dequeue();
        }
    }
}

}
(defrule initialize
(declare (salience 9000))
=>
(printout t "initialize" crlf)

(assert (KEEP-ALIVE))

)

(defrule loop
(declare (salience -1000))
(KEEP-ALIVE)
(not (SHUTDOWN))
=>

(printout t "Looping" crlf)
;(UpdateSomething)

(refresh loop)

)
CLIPSNET CaptureRouter memory leaks? https://sourceforge.net/p/clipsrules/discussion/776946/thread/454604ea4c/?limit=25#b791
Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/clipsrules/discussion/776946/

To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

[Davide Archetti]

I've changed CLIPSNET_Router.cpp and .h
It is now stable
Thank you

[Davide Archetti]

Hi again,
in another project I had defined a new ruoter, and now, after changing those files, I had a null reference exception here:

    bool CLIPSCPPRouterBridge::Query(
        CLIPSCPPEnv* theCPPEnv,
        const char* logicalName)
    {
        String^ cliLogicalName = Environment::CharStarToString(logicalName);
        Router^ r = (Router^)m_Router->Target;

        return r->Query(cliLogicalName);
    }

r is null
This is my implementation of router

internal class Trace : Router
    {
        private static readonly string routerName = "trace";
        CLIPSNET.Environment clipsEnvironment;
        Logger logger = NLog.LogManager.GetCurrentClassLogger();
        StringBuilder sb;

        string lastTrace = string.Empty;
        bool traceOn = false;

        internal Trace(CLIPSNET.Environment _env) : base(routerName, 40)
        {
            clipsEnvironment = _env;
            sb = new StringBuilder(1000);
        }

        public override bool Query(string logicalName)
        {
            return logicalName == routerName || logicalName == STDOUT;
        }

        public override void Write(string name, string buffer)
        {
            if (buffer.StartsWith("FIRE") ||
                buffer.StartsWith("==>") ||
                buffer.StartsWith("<==")) traceOn = true;

            if (traceOn)
            {
                name = routerName;
            }

            if (name == routerName)
            {
                if (buffer.Contains('\n'))
                {
                    if (sb.ToString() != lastTrace)
                    {
                        logger.Trace(sb);
                    }
                    lastTrace = sb.ToString();
                    sb.Clear();
                    if (traceOn) traceOn = false;
                }
                else
                {
                    sb.Append(buffer);
                }
            }
            else
            {
                clipsEnvironment.DeactivateRouter(this);
                clipsEnvironment.Write(name, buffer);
                clipsEnvironment.ActivateRouter(this);
            }
        }
    }

Is there something wrong?

[Gary Riley]

I can't tell much from just this. Can you provide enough code that I can reproduce the issue? The Target value should be set in this piece of code when the WeakReference is created:

CLIPSCPPRouterBridge::CLIPSCPPRouterBridge(msclr::gcroot<router^> the_Router)
{
m_Router = gcnew WeakReference(the_Router,false);
}</router^>

It would be helpful to know if this is either not initially set or if the value is set to null at some other point.

[Davide Archetti]

Hi,
I've found what is causing the problem, but I don't know why, I suppose it's related to WeakReference.
Anyways, this is the code that is running in a separated task

        public void Loop(string rulePath)
        {
            foreach (string file in Directory.GetFiles(rulePath, "*.clp").OrderBy(x => x))
            {

                System.IO.StreamReader sr = new System.IO.StreamReader(file);
                string all = sr.ReadToEnd();
                try
                {
                    clipsEnvironment.LoadFromString(all);
                }
                catch (Exception ex)
                {
                    throw;
                }
            }



            clipsEnvironment.AddRouter(new Routers.Trace(clipsEnvironment));
            //AddRouter();

            clipsEnvironment.Watch("rules");
            clipsEnvironment.Watch("facts");


            string cmd = string.Empty;
            while (cmd != "quit")
            {
                clipsEnvironment.Run(1);

                if (cmd != string.Empty)
                {
                    //ExecCommand(cmd);

                }

                //cmd = Commands.Dequeue();
            }
        }

And that code is working correctly. Now, instead of using the line

clipsEnvironment.AddRouter(new Routers.Trace(clipsEnvironment));

if instead, I use

AddRouter();
....

        private void AddRouter()
        {
            clipsEnvironment.AddRouter(new Routers.Trace(clipsEnvironment));
        }

now I get the null pointer exception.

Is it because the Garbage Collector has done its work because the Trace router has gone out of scope after exiting the AddRouter function?

[Gary Riley]

Yes, it seems like a plausible explanation that the router is getting garbage collected. I looked through my .NET code and in the cases where I saw routers being used they were either assigned to an member variable of a class or completely used within the scope of a method so they wouldn't be garbage collected when in use. I think what I need to is add a HashSet (or whatever it's called for .NET) to the Environment class and add any routers passed into AddRouter to that set so the garbage collecter won't try to collect them until they've been removed with a DeleteRouter call.

[Gary Riley]

Try these replacement files for the CLIPSCLRWrapper project.

[Davide Archetti]

Yes, it works now, no more errors.
Thank you