Re: [DL-Learner discussion] Jamon logging

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hm, yes, there are actually two use cases:
1. Finding bottlenecks and improve performance, for this we can really 
use the tools you are suggesting. I used VisualVM sometimes, but I 
always forget that it exists and code without it ^^   So, I guess this 
is tackled, well.  I might apply for yourkit as well.
2. Whenever we are measuring times, however, to put them in a paper we 
will probably need something else than a profiler. On the other hand 
this part is highly client specific and the script that runs the 
experiment should probably do the measurements.  Then everybody can use 
the prefered library.

Thanks, Chris, for clarifying this. I will try and remove all the time 
logging from the core code and put it in interfaces and use Yourkit 
instead.
VisualVM also does not need an eclipse plugin. I think it is a 
standalone app also.

All the best,
Sebastian

On 04/19/2012 10:19 PM, Chris Shellenbarger wrote:
> Hi Sebastian,
>
> It sounds like what you're trying to accomplish is to determine bottlenecks in the DL-Learner software in order to increase performance/reduce memory footprint.  I think this is a really great idea that all projects should do but very few actually think about.
>
> Spring is the object factory responsible for the instantiation of the high level components (KS, LearningProblem, LearningAlgorithm, Reasoners,etc) and their configured dependencies.  So, we could hook into the object instantiation code and pass any instantiated object to the Jamon MonitorFactory.  The downside of this is that it will only monitor the methods defined on an interface as it's using Java dynamic proxies.  So, we'd be able to see performance characteristics at the interface method level.  Basically, we'd see that LearningAlgorithm.start() would take a lot of time, but not the individual components.
>
> If we want to identify bottlenecks in DL-Learner, why don't we use a profiling tool to profile the application?  This would give us performance metrics for every method called during program execution regardless of whether it was behind an interface or not.   Also, there would be no modifications to code to achieve this - we just run the app with the profiler attached.  VisualVM provides this functionality and plugs in with Eclipse (http://bit.ly/JoAv2X).  Personally, I use YourKit (http://yourkit.com/) which is available for free to open source application developers by following the instructions at (http://bit.ly/JoBsse).  It is an incredible tool which I use for all of my programs that I write.
>
> I don't prefer one metrics library over another - I just want to separate the metrics concern away from the algorithms themselves.  This keeps the code lean and easy to understand/maintain.  We've done a lot in the last year towards to further this goal by removing the configuration from the algorithms.
>
> Please let me know what you think about these points and if I'm misunderstanding something.
>
> Thanks,
> Chris
>
>
> -----Original Message-----
> From: Sebastian Hellmann [mailto:hel...@in...]
> Sent: Thursday, April 19, 2012 9:37 AM
> To: Chris Shellenbarger
> Cc: dl-...@li...
> Subject: Re: [DL-Learner discussion] Jamon logging
>
> The general problem I have with AOP is that it adds quite a technology stack on top of DL-Learner. Normally, you will need to install additional eclipse plugins and whatnot to weave the aspect into the program and I would not do this for logging alone.
>
> Jamon is fairly easy to use, but needs some extra cross concern code. On their site however, they state that:
> http://jamonapi.sourceforge.net/#jamonmisc
> "In fact when totally disabled there is NO PERFORMANCE IMPACT whatsoever, as the actual object is returned and not a monitored object."
>
> I really don't care, where we insert the time profiling code, but we should decide on one framework to use and then use it consistently.
>
> I am not so much concerned about third parties in general. DL-Learner's impact would benefit greatly from better speed performance compared to the disadvantage of some cross_cutting Jamon code here and there. I would always opt for the ease of faster finding bottlenecks.
>
> Have you ever used the Metrics system? The page looks nice, but so does the page of http://projectlombok.org/ (with many disadvantages in the end).  Compatibility with VisualVM is also a plus, but I can not judge, whether it is unnecessary overhead.
>
> All th best,
> Sebastian
>
>
>
> On 04/19/2012 04:53 PM, Chris Shellenbarger wrote:
>> Hi Sebastian,
>>      I think leveraging metrics is a good idea, and along with other cross cutting concerns<http://bit.ly/JnbghC>, I think we need to be careful and not let it into the components-core main library.  A better place for it is up at the interfaces(CLI, etc) level as these are the clients which leverage the core library and do all of the instantiation of the objects they require.  The same goes for third parties who are leveraging just the core jar - the responsibility is on the client to inject these types of concerns at runtime.  This is important for various reasons, but an important reason that affects third parties is that some may want to use another metrics library like Metrics<http://metrics.codahale.com/>   rather than Jamon.  Some may want no metrics at all - this is a client specific choice and shouldn't be at the library level.
>>
>> A nice way of implementing cross cutting concerns like these is to use dynamic proxies/aspect oriented programming (AOP).  We can inject these in behind any interface without changing any code.  For the clients produced by the DL-Learner team, this can be injected as an aspect to wrap all interfaces instantiated by the Spring ApplicationContext with monitoring code.
>>
>> Jamon looks like it's using dynamic proxies to wrapper implementations of interfaces - this typically means it can only monitor methods which are on interfaces.  The same holds true for Spring's dynamic proxies - you can't inject proxies with abstract or concrete classes other than through some byte code manipulation (compile time weaving).   A lot of DL-Learner's methods are defined at the Abstract class level - we can probably analyze these and move the appropriate methods up to an interface.  There will be other benefits to components interacting only with interfaces as well.
>>
>> So, to summarize, in our clients (CLI, etc) we can inject an aspect to leverage Jamon monitoring, but we can't monitor methods which aren't defined in an interface.  I believe the client is the place for this type of code to go as it is responsible for instantiating objects- DL-Learner uses Spring for that, but that's not a requirement for third parties.
>>
>> Please let me know your thoughts on this.
>>
>> Thanks,
>> Chris
>>
>> From: Sebastian Hellmann [mailto:hel...@in...]
>> Sent: Wednesday, April 18, 2012 11:55 PM
>> To: dl-...@li...
>> Subject: [DL-Learner discussion] Jamon logging
>>
>> Hi all,
>> maybe we can add these three lines, at a place where all the objects such as Knowledgesources are created?
>> http://jamonapi.sourceforge.net/#monitoringinterfaces
>>
>> import com.jamonapi.proxy.*;
>>
>>
>>
>> // The MyObject() class has to implement MyInterface for monitoring to work.
>>
>> MyInterface myObject = (MyInterface) MonProxyFactory.monitor(new
>> MyObject());
>>
>> myObject.myMethod();// method call will be monitored with JAMon.
>>
>>
>> All the best,
>> Sebastian
>>
>>
>>
>> --
>>
>> Dipl. Inf. Sebastian Hellmann
>>
>> Department of Computer Science, University of Leipzig
>>
>> Projects: http://nlp2rdf.org , http://dbpedia.org
>>
>> Homepage: http://bis.informatik.uni-leipzig.de/SebastianHellmann
>>
>> Research Group: http://aksw.org
>>
>
> --
> Dipl. Inf. Sebastian Hellmann
> Department of Computer Science, University of Leipzig
> Projects: http://nlp2rdf.org , http://dbpedia.org
> Homepage: http://bis.informatik.uni-leipzig.de/SebastianHellmann
> Research Group: http://aksw.org
>
>
>
>

-- 
Dipl. Inf. Sebastian Hellmann
Department of Computer Science, University of Leipzig
Projects: http://nlp2rdf.org , http://dbpedia.org
Homepage: http://bis.informatik.uni-leipzig.de/SebastianHellmann
Research Group: http://aksw.org