Uwe:
What do you think of Statistics-Package? This could help to evaluate CLA in more depth. Apart from computing the basic statistics this could later be extended to detect sequences and stuff...
David also sent his "Ideas for Measuring HTM" once, so are we going to do that?
Me not, I think was the own Uwe.
Anyway, I agree with the idea. Once, longer sequences were not recovered, only T+1, T+2, no more.
When I coded my HTM last year I was using the Windows performance counters to get stats and graph them. it was easy to implement and this feature is built into the .NET framework and the windows OS. I think it is a good approach for collecting and displaying information, but not cross-platform. Anyone implementing HTM in Java or on another platform would have to make their own statistics package.
So I would suggest an API again for this, a plug-in API. One of the first plug-ins could be a PerfMon / performance counter plug in.
If you are not familiar with perfmon, just go to the command prompt or windows search box and type in perfmon. Play around a bit with the performance counters in windows to get a feel for what you can do. You can spawn multiple windows, and I have seen IT rooms that have many monitors and many instances of perfmon running against remote machines.
I can implement the counters for an HTM that would count connections made, synapses, segments, cells, predictions good/bad, etc. in a few minutes. I will be using perfmon to monitor HTMs and the local cpu, memory, etc when I start doing performance optimization (soon I hope :-))
I have been waiting for an API to be created before doing that because I don't want to 'hard-code' a stats package into the app. I think it should be left up to the user to pick and choose from existing plug-ins, or create their own.
So I guess if others agree with this approach (vote), the action item would be to create an API and plug-in framework (similar to input processor) and then create a 'default' plug-in.
Our stats framework should have a means of collecting events or counts of the objects in the HTM and a way to transform/aggregate counts (sum, average, etc) and a way to display them or pass them on to a display / graph / chart or log output.
Using log4net would be a good way to leverage our existing libraries. log4net is pretty flexible, and well thought out in the way it collects and logs data.
That sounds promising.
It's not clear however, whether Uwe meant measuring performance statistics or in-depth accuracy statistics.
Note to self (and anyone else :-)
Creating a custom log4net appender to log to performance counters should be relatively easy: http://www.codeproject.com/Articles/406634/Creating-a-custom-log4net-appender
Also, log4net ships with about 20 different appenders allready: https://logging.apache.org/log4net/release/sdk/log4net.Appender.html