Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

Memory leak? (VB.NET)

Jason
2008-11-10
2013-04-22
  • Jason
    Jason
    2008-11-10

    Hi there,

    First of all thanks for creating this, it works great. I'm noticing an issue with memory however.

    I am using the dll & after a while (6-12 hours) the virtual memory of my application grows to be rather large... I've had my application grow to about 600-700mb & become quite unresponsive / unstable.

    Are predicates saved into memory? I know ProgramD would write all of these things into the ffm/usr directory..

    I'm pretty sure I'm not doing anything wrong as far as requesting, etc... just in case I've included my class code, it is fairly straight forward.

    Is there something I am missing here? Any suggestions on how to avoid this?

    ------

    Class:

    Imports AIMLbot

    Public Class ProgramNET

        Dim myBot As New AIMLbot.Bot
        Dim myUser As New AIMLbot.User("user", myBot)
        Dim myReq As AIMLbot.Request
        Dim myRes As String = ""

        Public Sub New()
            myBot.loadSettings()
            myBot.isAcceptingUserInput = False
            myBot.loadAIMLFromFiles()
            myBot.isAcceptingUserInput = True
        End Sub

        Public Function Request(ByVal sUser As String, ByVal sMessage As String) As String
            Dim myReq = New AIMLbot.Request(sMessage, myUser, myBot)
            myRes = myBot.Chat(myReq).ToString
            myReq = Nothing
            ' Console.WriteLine("response: " & myRes)
            Return myRes
        End Function

    End Class

    -----

    Request example:

    Public alice as new ProgramNET

        Public Function AliceRequest(ByVal sMsg As String) As String
            Dim sTmp As String = String.Empty
            sTmp = alice.Request("user", sMsg)
            Return sTmp
        End Function

     
    • Jason
      Jason
      2008-11-10

      I profiled my application for a while with ANTS & the memory problem happened quickly... i found that there are around 480,000 live instances of string & around 250,000 live instances of int32[] all related to the dll (my aiml is 446 categories)

      this time it happened fairly quickly (maybe a different issue all together)... typically the virtual memory consumption grows by about 20-30mb every hour in my application

       
    • Jason
      Jason
      2008-11-12

      after some more testing, it appears as if even though i created 1 new instance of this class in my form it simply reloads itself

      i have it right at the top of the class: Public alice as New ProgramNET (that is the only place)

      looks like the leak came from loadAIMLFromFiles()

      settings reload themselves randomly whenever: myUser = New AIMLBot.User("user", myBot) is called, the New() sub is fired or the entire class somehow reloads strangely

      i set a boolean variable to try to avoid loading settings more than once, but it is completely ignored - the class just reloads completely and resets the original boolean value of false

      i'm totally stumped, i don't think this dll likes vb.net :(

       
    • Hi Jason,

      OK... catching up on email - apologies for not getting back sooner.

      It'd be really helpful if you could let me know what version of the platform you're using - I'll try to re-create the problem myself and get back to you.

      Please be aware that AIMLBot is quite long in the tooth - I *was* starting a complete re-write to simplify and improve the project: AIMLBot3.0 - but in the end real-life (tm) got in the way as well as my growing frustration with the limitations of AIML based chat-bots in general.

      I've put quite a bit of effort into planning a new library called BotKit - a framework for writing conversational agents (so you could easily write an AIMLBot using BotKit).

      Who knows... one of these days I might find some time to code it up... too much interesting stuff to do and not enough time to do it in.

      Best wishes,

      Nicholas.

       
    • Jason
      Jason
      2008-11-17

      Hi Nicholas,

      Looks like I found the error...

      I had created the "alice" object on Form1...

      And then I created the actual code to request the response in a public module

      every time the public module called form1.alice.Request() the New() sub was fired in my ProgramNET class (reloading all of the aiml, etc) - I'm not sure why this happened.... obviously what I was doing wasn't proper but for it to always fire the New() sub seemed a bit odd

      In any case, that problem is solved! Now I'm onto a different issue where I'm not 100% sure if it is related to ProgramNET or not... I just added a large custom AIML set for ProgramNET to use within my application & now I am getting random StackOverflow Exceptions

      I'm trying to find out what is causing it, I didn't notice this problem until I started using the new aiml... hopefully it is something silly that I can find easily. Been quite a pain so far though!

       
    • Hi,

      StackOverflow exceptions with new AIML sets are usually indicative of an infinite loop within the AIML... i.e. you have an sr/srai element that loops back on itself. Check your AIML! :-) Perhaps I should provide a method so Program# can validate the AIML and check the graph doesn't include infinte loops.

      Hope this helps,

      Nicholas.

       
    • Jason
      Jason
      2008-11-18

      yeah, i figured that was the case... it was the only thing that made sense

      i thought Program# had something similar to ProgramD in which if an infinite loop was encountered, it would just default to "INFINITE LOOP" in the aiml

      the aiml i have is pretty large & it will take some time to look for this bug... i was searching everywhere online for something that would do that with no luck :(

       
      • Jason
        Jason
        2008-11-18

        anyone here know of an editor or software / script that can locate these things? i'm SOO close :)

         
        • The quickest (and dirtiest) solution would be to modify the source to catch the StackOverflow exception and take a look at the context - i.e. find out what the current Request instance contains and work backwards from that..?

           
          • Jason
            Jason
            2008-11-18

            yeah, hmm maybe a property that would allow you to enable logging system exceptions (infinite loops / stackoverflow, etc)

            that would be great for keeping an eye on the aiml - kind of like httpd error logs, simple information even if its not very informative... like a timestamp, bot, request string, error

            would be awesome

             
    • Jason
      Jason
      2008-11-29

      still getting this memory leak, i dont know what I'm doing wrong... it appears as if there are a ton of live instances related to system string & the aimlbot

      i couldnt find an implementation of this code in VB.NET so this was my attempt at porting it over to vb. The problem is after running for some time, the memory keeps growing & growing and eventually my system just runs out of VM.

      am i creating this wrong? calling it wrong?

      here is a screenshot of the mem profiler where you can see how many live instances there are here (memory grew about 150mb at this point).

      http://jenna.525i.net/memscreen.jpg

      (object class)

      Imports AIMLbot

      Public Class ProgramNET
          Private WithEvents myBot As AIMLbot.Bot
          Private myUser As AIMLbot.User
          Private bSettingsLoaded As Boolean = False

          Public Sub New()
              myBot = New AIMLbot.Bot
              myUser = New AIMLbot.User("user", myBot)
              myBot.loadSettings()
              myBot.isAcceptingUserInput = False
              myBot.loadAIMLFromFiles()
              myBot.isAcceptingUserInput = True
          End Sub
         
          Public Function Request(ByVal sUser As String, ByVal sMessage As String) As String
              Dim myRes As String = String.Empty
              Dim myReq As AIMLbot.Request
              If myUser Is Nothing Then
                  myUser = New AIMLbot.User("user", myBot)
              End If
              myReq = New AIMLbot.Request(sMessage, myUser, myBot)
              myRes = myBot.Chat(myReq).ToString
              myReq = Nothing
              Return myRes
          End Function
      End Class

      -----

      (in a public module)

      Private alice As New ProgramNET

      Public Function AliceRequest(ByVal sMsg As String) As String
          Dim sTmp As String = String.Empty
          Try
              If alice Is Nothing Then
                  Debug.WriteLine("alice object is nothing")
              Else
                  sTmp = alice.Request("user", sMsg)
              End If
          Catch ex As Exception
              Debug.WriteLine("AliceRequest Error: " & ex.Message)
          End Try
         
          Return sTmp
      End Function

       
    • Hmmm,

      Part of the point of having a managed run-time is that memory leaks shouldn't happen (unless you're in an infinite loop or simply run out of memory because you're not de-referencing).

      Therein lies the secret - in the public module - can you confirm that the line:

      Private alice As New ProgramNet

      isn't being called for each AliceRequest.

      I'm no expert in VB (C# is my thing) - but in C# there is the keyword "using" that can define the scope of an instance of a thing allowing it to be marked for GC as soon as the "using" block of code is exited. Is there something like this in VB? Can you check you've not got hanging references to strings (meaning they are never GC'd)?

      Like I said - VB isn't my thing so I'm not quite sure what to suggest. Of course, if it turns out to be something in Program# then I'll fix it.

      Best wishes,

      Nicholas.