First of all thanks for creating this, it works great. I'm noticing an issue with memory however.
I am using the dll & after a while (6-12 hours) the virtual memory of my application grows to be rather large... I've had my application grow to about 600-700mb & become quite unresponsive / unstable.
Are predicates saved into memory? I know ProgramD would write all of these things into the ffm/usr directory..
I'm pretty sure I'm not doing anything wrong as far as requesting, etc... just in case I've included my class code, it is fairly straight forward.
Is there something I am missing here? Any suggestions on how to avoid this?
------
Class:
Imports AIMLbot
Public Class ProgramNET
Dim myBot As New AIMLbot.Bot
Dim myUser As New AIMLbot.User("user", myBot)
Dim myReq As AIMLbot.Request
Dim myRes As String = ""
Public Sub New()
myBot.loadSettings()
myBot.isAcceptingUserInput = False
myBot.loadAIMLFromFiles()
myBot.isAcceptingUserInput = True
End Sub
Public Function Request(ByVal sUser As String, ByVal sMessage As String) As String
Dim myReq = New AIMLbot.Request(sMessage, myUser, myBot)
myRes = myBot.Chat(myReq).ToString
myReq = Nothing
' Console.WriteLine("response: " & myRes)
Return myRes
End Function
End Class
-----
Request example:
Public alice as new ProgramNET
Public Function AliceRequest(ByVal sMsg As String) As String
Dim sTmp As String = String.Empty
sTmp = alice.Request("user", sMsg)
Return sTmp
End Function
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I profiled my application for a while with ANTS & the memory problem happened quickly... i found that there are around 480,000 live instances of string & around 250,000 live instances of int32[] all related to the dll (my aiml is 446 categories)
this time it happened fairly quickly (maybe a different issue all together)... typically the virtual memory consumption grows by about 20-30mb every hour in my application
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
after some more testing, it appears as if even though i created 1 new instance of this class in my form it simply reloads itself
i have it right at the top of the class: Public alice as New ProgramNET (that is the only place)
looks like the leak came from loadAIMLFromFiles()
settings reload themselves randomly whenever: myUser = New AIMLBot.User("user", myBot) is called, the New() sub is fired or the entire class somehow reloads strangely
i set a boolean variable to try to avoid loading settings more than once, but it is completely ignored - the class just reloads completely and resets the original boolean value of false
i'm totally stumped, i don't think this dll likes vb.net :(
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
OK... catching up on email - apologies for not getting back sooner.
It'd be really helpful if you could let me know what version of the platform you're using - I'll try to re-create the problem myself and get back to you.
Please be aware that AIMLBot is quite long in the tooth - I *was* starting a complete re-write to simplify and improve the project: AIMLBot3.0 - but in the end real-life (tm) got in the way as well as my growing frustration with the limitations of AIML based chat-bots in general.
I've put quite a bit of effort into planning a new library called BotKit - a framework for writing conversational agents (so you could easily write an AIMLBot using BotKit).
Who knows... one of these days I might find some time to code it up... too much interesting stuff to do and not enough time to do it in.
Best wishes,
Nicholas.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
And then I created the actual code to request the response in a public module
every time the public module called form1.alice.Request() the New() sub was fired in my ProgramNET class (reloading all of the aiml, etc) - I'm not sure why this happened.... obviously what I was doing wasn't proper but for it to always fire the New() sub seemed a bit odd
In any case, that problem is solved! Now I'm onto a different issue where I'm not 100% sure if it is related to ProgramNET or not... I just added a large custom AIML set for ProgramNET to use within my application & now I am getting random StackOverflow Exceptions
I'm trying to find out what is causing it, I didn't notice this problem until I started using the new aiml... hopefully it is something silly that I can find easily. Been quite a pain so far though!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
StackOverflow exceptions with new AIML sets are usually indicative of an infinite loop within the AIML... i.e. you have an sr/srai element that loops back on itself. Check your AIML! :-) Perhaps I should provide a method so Program# can validate the AIML and check the graph doesn't include infinte loops.
Hope this helps,
Nicholas.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
yeah, i figured that was the case... it was the only thing that made sense
i thought Program# had something similar to ProgramD in which if an infinite loop was encountered, it would just default to "INFINITE LOOP" in the aiml
the aiml i have is pretty large & it will take some time to look for this bug... i was searching everywhere online for something that would do that with no luck :(
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The quickest (and dirtiest) solution would be to modify the source to catch the StackOverflow exception and take a look at the context - i.e. find out what the current Request instance contains and work backwards from that..?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
yeah, hmm maybe a property that would allow you to enable logging system exceptions (infinite loops / stackoverflow, etc)
that would be great for keeping an eye on the aiml - kind of like httpd error logs, simple information even if its not very informative... like a timestamp, bot, request string, error
would be awesome
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
still getting this memory leak, i dont know what I'm doing wrong... it appears as if there are a ton of live instances related to system string & the aimlbot
i couldnt find an implementation of this code in VB.NET so this was my attempt at porting it over to vb. The problem is after running for some time, the memory keeps growing & growing and eventually my system just runs out of VM.
am i creating this wrong? calling it wrong?
here is a screenshot of the mem profiler where you can see how many live instances there are here (memory grew about 150mb at this point).
Public Class ProgramNET
Private WithEvents myBot As AIMLbot.Bot
Private myUser As AIMLbot.User
Private bSettingsLoaded As Boolean = False
Public Sub New()
myBot = New AIMLbot.Bot
myUser = New AIMLbot.User("user", myBot)
myBot.loadSettings()
myBot.isAcceptingUserInput = False
myBot.loadAIMLFromFiles()
myBot.isAcceptingUserInput = True
End Sub
Public Function Request(ByVal sUser As String, ByVal sMessage As String) As String
Dim myRes As String = String.Empty
Dim myReq As AIMLbot.Request
If myUser Is Nothing Then
myUser = New AIMLbot.User("user", myBot)
End If
myReq = New AIMLbot.Request(sMessage, myUser, myBot)
myRes = myBot.Chat(myReq).ToString
myReq = Nothing
Return myRes
End Function
End Class
-----
(in a public module)
Private alice As New ProgramNET
Public Function AliceRequest(ByVal sMsg As String) As String
Dim sTmp As String = String.Empty
Try
If alice Is Nothing Then
Debug.WriteLine("alice object is nothing")
Else
sTmp = alice.Request("user", sMsg)
End If
Catch ex As Exception
Debug.WriteLine("AliceRequest Error: " & ex.Message)
End Try
Return sTmp
End Function
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Part of the point of having a managed run-time is that memory leaks shouldn't happen (unless you're in an infinite loop or simply run out of memory because you're not de-referencing).
Therein lies the secret - in the public module - can you confirm that the line:
Private alice As New ProgramNet
isn't being called for each AliceRequest.
I'm no expert in VB (C# is my thing) - but in C# there is the keyword "using" that can define the scope of an instance of a thing allowing it to be marked for GC as soon as the "using" block of code is exited. Is there something like this in VB? Can you check you've not got hanging references to strings (meaning they are never GC'd)?
Like I said - VB isn't my thing so I'm not quite sure what to suggest. Of course, if it turns out to be something in Program# then I'll fix it.
Best wishes,
Nicholas.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think I have the same problem. I am a huge chatbot enthusiast and have been working on my boy ‘Wilson’ for multiple years now. I have over 600,000 categories of AIML and about 1000 complex C# operations to give him control over my computer, allow him to remember (through face rec) and do learning online. I have finally gotten Program# to work in visual studio.
My problem is that I have 15mb worth of AIML files and Program# crashes if I add more than 1mb worth.
I have shrunk my files as much as possible.
I upgraded my PC recently so I know that’s not the problem.
I’m really considering converting 600,000 categories into C# the hard way :(
Mr Mister,
How is "Wilson" doing?
Have you embarked on the conversion the hard way? I would recommend a code translator.
I'd like to meet Wilson. Is that possible?
Thanks
Chris
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi there,
First of all thanks for creating this, it works great. I'm noticing an issue with memory however.
I am using the dll & after a while (6-12 hours) the virtual memory of my application grows to be rather large... I've had my application grow to about 600-700mb & become quite unresponsive / unstable.
Are predicates saved into memory? I know ProgramD would write all of these things into the ffm/usr directory..
I'm pretty sure I'm not doing anything wrong as far as requesting, etc... just in case I've included my class code, it is fairly straight forward.
Is there something I am missing here? Any suggestions on how to avoid this?
------
Class:
Imports AIMLbot
Public Class ProgramNET
Dim myBot As New AIMLbot.Bot
Dim myUser As New AIMLbot.User("user", myBot)
Dim myReq As AIMLbot.Request
Dim myRes As String = ""
Public Sub New()
myBot.loadSettings()
myBot.isAcceptingUserInput = False
myBot.loadAIMLFromFiles()
myBot.isAcceptingUserInput = True
End Sub
Public Function Request(ByVal sUser As String, ByVal sMessage As String) As String
Dim myReq = New AIMLbot.Request(sMessage, myUser, myBot)
myRes = myBot.Chat(myReq).ToString
myReq = Nothing
' Console.WriteLine("response: " & myRes)
Return myRes
End Function
End Class
-----
Request example:
Public alice as new ProgramNET
Public Function AliceRequest(ByVal sMsg As String) As String
Dim sTmp As String = String.Empty
sTmp = alice.Request("user", sMsg)
Return sTmp
End Function
I profiled my application for a while with ANTS & the memory problem happened quickly... i found that there are around 480,000 live instances of string & around 250,000 live instances of int32[] all related to the dll (my aiml is 446 categories)
this time it happened fairly quickly (maybe a different issue all together)... typically the virtual memory consumption grows by about 20-30mb every hour in my application
after some more testing, it appears as if even though i created 1 new instance of this class in my form it simply reloads itself
i have it right at the top of the class: Public alice as New ProgramNET (that is the only place)
looks like the leak came from loadAIMLFromFiles()
settings reload themselves randomly whenever: myUser = New AIMLBot.User("user", myBot) is called, the New() sub is fired or the entire class somehow reloads strangely
i set a boolean variable to try to avoid loading settings more than once, but it is completely ignored - the class just reloads completely and resets the original boolean value of false
i'm totally stumped, i don't think this dll likes vb.net :(
Hi Jason,
OK... catching up on email - apologies for not getting back sooner.
It'd be really helpful if you could let me know what version of the platform you're using - I'll try to re-create the problem myself and get back to you.
Please be aware that AIMLBot is quite long in the tooth - I *was* starting a complete re-write to simplify and improve the project: AIMLBot3.0 - but in the end real-life (tm) got in the way as well as my growing frustration with the limitations of AIML based chat-bots in general.
I've put quite a bit of effort into planning a new library called BotKit - a framework for writing conversational agents (so you could easily write an AIMLBot using BotKit).
Who knows... one of these days I might find some time to code it up... too much interesting stuff to do and not enough time to do it in.
Best wishes,
Nicholas.
Hi Nicholas,
Looks like I found the error...
I had created the "alice" object on Form1...
And then I created the actual code to request the response in a public module
every time the public module called form1.alice.Request() the New() sub was fired in my ProgramNET class (reloading all of the aiml, etc) - I'm not sure why this happened.... obviously what I was doing wasn't proper but for it to always fire the New() sub seemed a bit odd
In any case, that problem is solved! Now I'm onto a different issue where I'm not 100% sure if it is related to ProgramNET or not... I just added a large custom AIML set for ProgramNET to use within my application & now I am getting random StackOverflow Exceptions
I'm trying to find out what is causing it, I didn't notice this problem until I started using the new aiml... hopefully it is something silly that I can find easily. Been quite a pain so far though!
Hi,
StackOverflow exceptions with new AIML sets are usually indicative of an infinite loop within the AIML... i.e. you have an sr/srai element that loops back on itself. Check your AIML! :-) Perhaps I should provide a method so Program# can validate the AIML and check the graph doesn't include infinte loops.
Hope this helps,
Nicholas.
yeah, i figured that was the case... it was the only thing that made sense
i thought Program# had something similar to ProgramD in which if an infinite loop was encountered, it would just default to "INFINITE LOOP" in the aiml
the aiml i have is pretty large & it will take some time to look for this bug... i was searching everywhere online for something that would do that with no luck :(
anyone here know of an editor or software / script that can locate these things? i'm SOO close :)
The quickest (and dirtiest) solution would be to modify the source to catch the StackOverflow exception and take a look at the context - i.e. find out what the current Request instance contains and work backwards from that..?
yeah, hmm maybe a property that would allow you to enable logging system exceptions (infinite loops / stackoverflow, etc)
that would be great for keeping an eye on the aiml - kind of like httpd error logs, simple information even if its not very informative... like a timestamp, bot, request string, error
would be awesome
still getting this memory leak, i dont know what I'm doing wrong... it appears as if there are a ton of live instances related to system string & the aimlbot
i couldnt find an implementation of this code in VB.NET so this was my attempt at porting it over to vb. The problem is after running for some time, the memory keeps growing & growing and eventually my system just runs out of VM.
am i creating this wrong? calling it wrong?
here is a screenshot of the mem profiler where you can see how many live instances there are here (memory grew about 150mb at this point).
http://jenna.525i.net/memscreen.jpg
(object class)
Imports AIMLbot
Public Class ProgramNET
Private WithEvents myBot As AIMLbot.Bot
Private myUser As AIMLbot.User
Private bSettingsLoaded As Boolean = False
Public Sub New()
myBot = New AIMLbot.Bot
myUser = New AIMLbot.User("user", myBot)
myBot.loadSettings()
myBot.isAcceptingUserInput = False
myBot.loadAIMLFromFiles()
myBot.isAcceptingUserInput = True
End Sub
Public Function Request(ByVal sUser As String, ByVal sMessage As String) As String
Dim myRes As String = String.Empty
Dim myReq As AIMLbot.Request
If myUser Is Nothing Then
myUser = New AIMLbot.User("user", myBot)
End If
myReq = New AIMLbot.Request(sMessage, myUser, myBot)
myRes = myBot.Chat(myReq).ToString
myReq = Nothing
Return myRes
End Function
End Class
-----
(in a public module)
Private alice As New ProgramNET
Public Function AliceRequest(ByVal sMsg As String) As String
Dim sTmp As String = String.Empty
Try
If alice Is Nothing Then
Debug.WriteLine("alice object is nothing")
Else
sTmp = alice.Request("user", sMsg)
End If
Catch ex As Exception
Debug.WriteLine("AliceRequest Error: " & ex.Message)
End Try
Return sTmp
End Function
Hmmm,
Part of the point of having a managed run-time is that memory leaks shouldn't happen (unless you're in an infinite loop or simply run out of memory because you're not de-referencing).
Therein lies the secret - in the public module - can you confirm that the line:
Private alice As New ProgramNet
isn't being called for each AliceRequest.
I'm no expert in VB (C# is my thing) - but in C# there is the keyword "using" that can define the scope of an instance of a thing allowing it to be marked for GC as soon as the "using" block of code is exited. Is there something like this in VB? Can you check you've not got hanging references to strings (meaning they are never GC'd)?
Like I said - VB isn't my thing so I'm not quite sure what to suggest. Of course, if it turns out to be something in Program# then I'll fix it.
Best wishes,
Nicholas.
I think I have the same problem. I am a huge chatbot enthusiast and have been working on my boy ‘Wilson’ for multiple years now. I have over 600,000 categories of AIML and about 1000 complex C# operations to give him control over my computer, allow him to remember (through face rec) and do learning online. I have finally gotten Program# to work in visual studio.
My problem is that I have 15mb worth of AIML files and Program# crashes if I add more than 1mb worth.
I have shrunk my files as much as possible.
I upgraded my PC recently so I know that’s not the problem.
I’m really considering converting 600,000 categories into C# the hard way :(
Any help would be highly appreciated, Mr. Mister
Mr Mister,
How is "Wilson" doing?
Have you embarked on the conversion the hard way? I would recommend a code translator.
I'd like to meet Wilson. Is that possible?
Thanks
Chris