webmacro-user — List for WebMacro users, developers, competitors, and friends.

Re: [WebMacro-user] patches to avoid FastWriter creation

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Thu 9 March 2006 16:58, Eric B. Ridge wrote:
> On Mar 9, 2006, at 11:50 AM, Alex Twisleton-Wykeham-Fiennes wrote:
> > Which jar is org.apache.tools.ant.taskdefs.optional.javacc.JavaCC
> > actually
> > supposed to be in because I don't find it on a cursory glance.  The
> > rule
> > which specifies it is this one:-
>
> Oh right, you also need the .jar of optional And taskdefs.  Back in
> the Ant 1.2 days it was an additional download from Jakarta called
> "ant-optional.jar" (or something similar).  I'm not sure what the
> latest versions of Ant do, but I suspect it's similar.
>
> That jar also needs to be in $ANT_HOME/lib.

This is prooving to be a very frustrating twisty turny web of passages all 
alike.

If anyone has the appropriate jar file that I should be linking to get the 
javacc build target to work with ant-1.6.2-5 on SuSE Linux 10.0 then it would 
be very much appreciated.

(the ant-javadoc-1.6.2-7 on SuSE 10.0 contains the javadoc for the class in 
question, I just can't find / they've ommitted the rpm with the actual jar 
file in it)

many thanks

Alex

Re: [WebMacro-user] Accessing property via "Unary Accessor"

[Mark C. <mcr...@ma...>]

On 2006-03-09 04:37, Marc Palmer wrote:

> Well, what doesn't quite make sense (but I like it anyway) is that 
> WebMacro assumes the first letter of properties should be capitalized - 
> I think that is your real problem.

> i.e. in any code that uses introspection and the java bean 
> specification, a property called "title" is accessed as "title" - this 
> is true of languages such as Groovy also.

> However WM, for some reason - I imagine aesthetics - requires you to 
> access a property  like this as "Title". As far as I recall it doesn't 
> matter whether you have a getTitle() or a public "title" field, it must 
> always be accessed as "Title".

> This has always jarred slightly in terms of my 'Java programmer' 
> expectations, but of course to non-Java end-users of WM this may not be 

On 2006-03-09 11:10, Keats Kirsch wrote:

> Actually you're not completely correct.  A public variable name
> "title" would be referred to as $obj.title.  This is the point Eric
> was trying to make; WM allows you to be completely unambiguous about
> the property you are referring to.  You can have a fields called
> "title", "Title", "TITLE", or accessors called "gettitle",
> "getTitle", getTITLE".  So the reason you refer to a property as
> "Title" is that you are generally referring to the "getTitle"
> accessor, not the non-public "title" member variable.


Maybe this is one of those places were an annotation might be useful for those 
who use Java 1.5.  If you have some field accessor method that, for some reason, 
you can't name as getAccessor then a standard annotation indicating that it is a 
getter could be used to still allow the template author, who may not understand 
such subtleties, to use the simple reference as if it did follow the old style 
javabean naming convention.

This is just a suggestion... I don't have any burning need for such a feature, 
but thought I'd run it up the flag pole and see if anyone salutes.  Also, I'd be 
interested in knowing if it's possible to do something like this in a way that 
doesn't break for users of older versions of the Java.

Re: [WebMacro-user] Optimising #include creation of FastWriter

[<Web...@St...>]

On Wed, 8 Mar 2006, Eric B. Ridge wrote:

| I've been thinking back to when I write IncludeDirective and I simply cannot
| remember why it makes a new FastWriter.  Was it because the old ParseDirective
| did that and I was just coping it?  Was it out of sheer stupidity?  Or maybe
| some reason about not wanting to muck with the real FastWriter in the event of
| a parsing/evaluation exception.  I dunno.

You should have commented it in the code.

..

One comment about the FastWriter/Writer thingy: A "resetable" writer would 
be fantastic, preferrably having unlimited resetable space (just growing 
adding byte-arrays into some list as needed).
  I am responsible for a portal, which renders every portlet in its own 
context and with its own "StringWriter", so that if it throws an 
exception, I can throw it away, and instead render an error-portlet. If 
all goes well, I commit the fully rendered portlet to the FastWriter, and 
move along.

However, this would be hugely better if I could just _always_ pass along 
the same fastwriter recursing down includes and macros and whatnots. 
Before I venture into the next portlet, i do a mark(). Then I let it 
render the portlet, with its recursing. Then if an exception turns up, 
I'll just do a reset(), and then output an error-template instead, while 
if it goes well, i'll do a _flush()_ to commit the stuff all the way to 
the browser at the other end of the stream (I feel this should be up to 
me, the webapp writer, to decide when to flush). That would have been 
absolutely great, and absolutely more performant.

Regards,
Endre.

Re: [WebMacro-user] patches to avoid FastWriter creation

[Eric B. R. <eb...@tc...>]

On Mar 9, 2006, at 11:50 AM, Alex Twisleton-Wykeham-Fiennes wrote:

> Which jar is org.apache.tools.ant.taskdefs.optional.javacc.JavaCC  
> actually
> supposed to be in because I don't find it on a cursory glance.  The  
> rule
> which specifies it is this one:-

Oh right, you also need the .jar of optional And taskdefs.  Back in  
the Ant 1.2 days it was an additional download from Jakarta called  
"ant-optional.jar" (or something similar).  I'm not sure what the  
latest versions of Ant do, but I suspect it's similar.

That jar also needs to be in $ANT_HOME/lib.

eric

Re: [WebMacro-user] patches to avoid FastWriter creation

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Thu 9 March 2006 16:36, Eric B. Ridge wrote:
> On Mar 9, 2006, at 11:29 AM, Alex Twisleton-Wykeham-Fiennes wrote:
> > BUILD FAILED
> > file:/home/alex/cvsroot/webmacro/build.xml:76: taskdef class
> > org.apache.tools.ant.taskdefs.optional.javacc.JavaCC cannot be found
>
> You need to have JavaCC installed on your system, and its main .jar
> file in your $ANT_HOME/lib directory.
>
> You can find JavaCC here:
> 	https://javacc.dev.java.net/

done that but no joy.  

Which jar is org.apache.tools.ant.taskdefs.optional.javacc.JavaCC actually 
supposed to be in because I don't find it on a cursory glance.  The rule 
which specifies it is this one:-

    <taskdef name="javacc"
        classname="org.apache.tools.ant.taskdefs.optional.javacc.JavaCC">
            <classpath refid="class.path"/>
    </taskdef>

Am I just being really stupid?

A

Re: [WebMacro-user] patches to avoid FastWriter creation

[Eric B. R. <eb...@tc...>]

On Mar 9, 2006, at 11:29 AM, Alex Twisleton-Wykeham-Fiennes wrote:

> BUILD FAILED
> file:/home/alex/cvsroot/webmacro/build.xml:76: taskdef class
> org.apache.tools.ant.taskdefs.optional.javacc.JavaCC cannot be found

You need to have JavaCC installed on your system, and its main .jar  
file in your $ANT_HOME/lib directory.

You can find JavaCC here:
	https://javacc.dev.java.net/

eric

Re: [WebMacro-user] patches to avoid FastWriter creation

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Thu 9 March 2006 16:13, Keats Kirsch wrote:
> Alex,
>
> I haven't found the time to evaluate these changes, but I have pretty
> good faith in the unit tests.  If they all still pass I say let's put
> these out there for folks to try.

I haven't been using the ant build system and the unit tests so far.  I'll 
check out my changes with respect to the unit tests and let you know how I 
get on.

One quick thing, where would I find the documentation on the ant build system, 
or what does this error mean?

Buildfile: build.xml

BUILD FAILED
file:/home/alex/cvsroot/webmacro/build.xml:76: taskdef class 
org.apache.tools.ant.taskdefs.optional.javacc.JavaCC cannot be found

Total time: 1 second

What should my classpath be set to when I run ant?

Alex

> Keats
>
> Alex Twisleton-Wykeham-Fiennes wrote:
> >On Thu 9 March 2006 12:29, Alex Twisleton-Wykeham-Fiennes wrote:
> >>All,
> >>
> >>here are the patched files to avoid unrequired FastWriter creation during
> >>template evaluation.
> >
> >(there is another email to the list before this one that is stuck in the
> > queue due to the attachments with the patches for the clean unloading of
> > templates. However, the patches don't overlap each other so the order of
> > application should not matter)
> >
> >>Changes:-
> >>
> >>- Template
> >>added in a write(FastWriter,Context) method.  This is already present in
> >>the only implementing class (WMTemplate) so it has no affect other than
> >>exposing the API to classes that evaluate against the interface.
> >>
> >>- IncludeDirective
> >>- EvalDirective
> >>Don't pre-cache the results of the sub-template parsing in a String but
> >>instead get them to write directly out to the FastWriter that is passed
> >>into the method.  This avoids the creation of a FastWriter to handle the
> >>template evaluation that is not required and which was taking up a load
> >> of processing time.
> >>
> >>Alex
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live
> webcast and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] patches to avoid FastWriter creation

[Keats K. <ke...@xa...>]

Alex,

I haven't found the time to evaluate these changes, but I have pretty 
good faith in the unit tests.  If they all still pass I say let's put 
these out there for folks to try.

Keats

Alex Twisleton-Wykeham-Fiennes wrote:

>On Thu 9 March 2006 12:29, Alex Twisleton-Wykeham-Fiennes wrote:
>  
>
>>All,
>>
>>here are the patched files to avoid unrequired FastWriter creation during
>>template evaluation.
>>    
>>
>
>(there is another email to the list before this one that is stuck in the queue 
>due to the attachments with the patches for the clean unloading of templates.  
>However, the patches don't overlap each other so the order of application 
>should not matter)
>
>  
>
>>Changes:-
>>
>>- Template
>>added in a write(FastWriter,Context) method.  This is already present in
>>the only implementing class (WMTemplate) so it has no affect other than
>>exposing the API to classes that evaluate against the interface.
>>
>>- IncludeDirective
>>- EvalDirective
>>Don't pre-cache the results of the sub-template parsing in a String but
>>instead get them to write directly out to the FastWriter that is passed
>>into the method.  This avoids the creation of a FastWriter to handle the
>>template evaluation that is not required and which was taking up a load of
>>processing time.
>>
>>Alex
>>    
>>

Re: [WebMacro-user] Accessing property via "Unary Accessor"

[Keats K. <ke...@xa...>]

Actually you're not completely correct.  A public variable name "title" 
would be referred to as $obj.title.  This is the point Eric was trying 
to make; WM allows you to be completely unambiguous about the property 
you are referring to.  You can have a fields called "title", "Title", 
"TITLE", or accessors called "gettitle", "getTitle", getTITLE".  So the 
reason you refer to a property as "Title" is that you are generally 
referring to the "getTitle" accessor, not the non-public "title" member 
variable.

It's also worthwhile to note that WM pre-dates JSP, so the property 
naming conventions for Java scripting were not well established at that 
time.  Personally I prefer the WM approach, for the reasons mentioned, 
and also aesthetically, but it's basically just a religious preference.  
Just be glad you're not using C# which got it all mixed up. ;-}

Keats

Marc Palmer wrote:

>
> On 8 Mar 2006, at 00:38, Sven Schliesing wrote:
>
>> Ok, that makes sense. Many Thanks for your quick and comprehensive  
>> answer!
>>
>
> Well, what doesn't quite make sense (but I like it anyway) is that  
> WebMacro assumes the first letter of properties should be capitalized  
> - I think that is your real problem.
>
> i.e. in any code that uses introspection and the java bean  
> specification, a property called "title" is accessed as "title" -  
> this is true of languages such as Groovy also.
>
> However WM, for some reason - I imagine aesthetics - requires you to  
> access a property  like this as "Title". As far as I recall it  
> doesn't matter whether you have a getTitle() or a public "title"  
> field, it must always be accessed as "Title".
>
> This has always jarred slightly in terms of my 'Java programmer'  
> expectations, but of course to non-Java end-users of WM this may not  
> be odd at all.
>
> Cheers
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting 
> language
> that extends applications into web and mobile media. Attend the live 
> webcast
> and join the prime developer group breaking into this new coding 
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user
>

Re: [WebMacro-user] patches to avoid FastWriter creation

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Thu 9 March 2006 12:29, Alex Twisleton-Wykeham-Fiennes wrote:
> All,
>
> here are the patched files to avoid unrequired FastWriter creation during
> template evaluation.

(there is another email to the list before this one that is stuck in the queue 
due to the attachments with the patches for the clean unloading of templates.  
However, the patches don't overlap each other so the order of application 
should not matter)

> Changes:-
>
> - Template
> added in a write(FastWriter,Context) method.  This is already present in
> the only implementing class (WMTemplate) so it has no affect other than
> exposing the API to classes that evaluate against the interface.
>
> - IncludeDirective
> - EvalDirective
> Don't pre-cache the results of the sub-template parsing in a String but
> instead get them to write directly out to the FastWriter that is passed
> into the method.  This avoids the creation of a FastWriter to handle the
> template evaluation that is not required and which was taking up a load of
> processing time.
>
> Alex

[WebMacro-user] patches to avoid FastWriter creation

[Alex Twisleton-Wykeham-F. <al...@fi...>]

All,

here are the patched files to avoid unrequired FastWriter creation during 
template evaluation.

Changes:-

- Template
added in a write(FastWriter,Context) method.  This is already present in the 
only implementing class (WMTemplate) so it has no affect other than exposing 
the API to classes that evaluate against the interface.

- IncludeDirective
- EvalDirective
Don't pre-cache the results of the sub-template parsing in a String but 
instead get them to write directly out to the FastWriter that is passed into 
the method.  This avoids the creation of a FastWriter to handle the template 
evaluation that is not required and which was taking up a load of processing 
time.

Alex 

Re: [WebMacro-user] Accessing property via "Unary Accessor"

[Marc P. <ma...@an...>]

On 8 Mar 2006, at 00:38, Sven Schliesing wrote:

> Ok, that makes sense. Many Thanks for your quick and comprehensive  
> answer!
>

Well, what doesn't quite make sense (but I like it anyway) is that  
WebMacro assumes the first letter of properties should be capitalized  
- I think that is your real problem.

i.e. in any code that uses introspection and the java bean  
specification, a property called "title" is accessed as "title" -  
this is true of languages such as Groovy also.

However WM, for some reason - I imagine aesthetics - requires you to  
access a property  like this as "Title". As far as I recall it  
doesn't matter whether you have a getTitle() or a public "title"  
field, it must always be accessed as "Title".

This has always jarred slightly in terms of my 'Java programmer'  
expectations, but of course to non-Java end-users of WM this may not  
be odd at all.

Cheers

Re: [WebMacro-user] Optimising #include creation of FastWriter

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Wed 8 March 2006 17:58, Eric B. Ridge wrote:
> I've been thinking back to when I write IncludeDirective and I simply
> cannot remember why it makes a new FastWriter.  Was it because the
> old ParseDirective did that and I was just coping it?  Was it out of
> sheer stupidity?  Or maybe some reason about not wanting to muck with
> the real FastWriter in the event of a parsing/evaluation exception.
> I dunno.

The only reason for the exception handling being an issue would be if you were 
going to be doing some kind of intelligent rollback on your nested template 
evaluation.  However, this doesn't smell write from an encapsulation point of 
view as I would expect any calculations that had rollbacks would take place 
in java space before you handed over control to the webmacro template, but if 
anyone has any real world situations that this would break then let me know.

> At any rate, I can't think of a reason why your idea won't work.  So
> I say go for it.

OK.  I'll get the destroy() stuff out of the way first and then have a closer 
look at the whole streaming thing...

> Go for the #eval thing too.  I know #setblock uses the same pattern,
> but it might actually be necessary for #setblock to do this.

I think that it is probably worth it.  On a relatively trivial application 
that was doing a fair amount of nested including and evaluating of templates, 
OptimizeIt was reporting that roughly 10 - 15 % of time was spent purely in 
creating FastWriters which just feels wrong to me...

I'll have a sweep through the entire tree looking for the usage patterns of 
FastWriter and will see if there are any other points that look like they 
should be tweaked as well.

Alex

Re: [WebMacro-user] Optimising #include creation of FastWriter

[Eric B. R. <eb...@tc...>]

I've been thinking back to when I write IncludeDirective and I simply  
cannot remember why it makes a new FastWriter.  Was it because the  
old ParseDirective did that and I was just coping it?  Was it out of  
sheer stupidity?  Or maybe some reason about not wanting to muck with  
the real FastWriter in the event of a parsing/evaluation exception.   
I dunno.

At any rate, I can't think of a reason why your idea won't work.  So  
I say go for it.

Go for the #eval thing too.  I know #setblock uses the same pattern,  
but it might actually be necessary for #setblock to do this.

eric

On Mar 8, 2006, at 7:32 AM, Alex Twisleton-Wykeham-Fiennes wrote:

> All,
>
> following on from my earlier optimising of Thread shutdowns inside  
> webmacro,
> I've been doing a bit of actual CPU profiling of the running  
> application and
> I found that a significant percentage of processor time was spent  
> creating
> new FastWriter objects as part of the
> Template.evaluateAsBytes(String,Context) method invoked from the
> IncludeDirective.write(FastWriter,Context).
>
> I couldn't see any reason why you should need to create a new  
> FastWriter just
> to write the template into it, convert it to a byte array and then  
> write it
> into the outer FastWriter - especially as the inner FastWriter is  
> only used
> for the duration of the evaluateAsBytes method and is initialised  
> using the
> same encoding as the outer FastWriter.
>
> I therefore made the following changes as an experiment:-
>
> Added:-
> public interface Template {
>   public void write(FastWriter out, Context context)
>     throws PropertyException, IOException;
> }
>
> changed:-
> public class IncludeDirective {
>   public void write(FastWriter out, Context context)
>     throws PropertyExcception, IOException
>   {
>     <snip>
>     case TYPE_TEMPLATE:
>       out.write(((Template) toInclude).evaluateAsBytes 
> (out.getEncoding(),
>                 context));
>     <snip>
>   }
> }
> to:-
> public class IncludeDirective {
>   public void write(FastWriter out, Context context)
>     throws PropertyExcception, IOException
>   {
>     <snip>
>     case TYPE_TEMPLATE:
>       ((Template) toInclude).write(out, context);
>     <snip>
>   }
> }
>
>
> which has yielded a considerable performance increase in the #include
> approach, not to mention a much happier garbage collector (not  
> having to
> allocate a char[512] every time that you include a template).
>
> Are there any major gotcha's in this approach that I haven't  
> spotted, and is
> it worth trying to look through the rest of the code at FastWriter  
> creation
> and try and move it more towards a "streaming" style of model with  
> a single
> FastWriter passed down through the items which is written to  
> directly rather
> than a tree of FastWriters that bubble up their byte[] outputs?
>
> Alex
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting  
> language
> that extends applications into web and mobile media. Attend the  
> live webcast
> and join the prime developer group breaking into this new coding  
> territory!
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] Thread leak with ReloadingCacheManager

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Wed 8 March 2006 15:54, Keats Kirsch wrote:
> I haven't had a chance to validate what you are saying, but I know there
> has been a longstanding issue with getting WM Web apps to unload
> properly.  It sounds like you've finally nailed it.
>
> I think your approach of adding a destroy method to the Broker makes
> sense.  If you're willing to work on it that would be great.

no problem at all.  I'll grab the latest version from CVS and have a play 
around with it.

I think that somepoint in the past n years I might have had a CVS account, but 
if I did then it has passed into the mists of time.  What is the procedure 
for changes and tweaks in the current webmacro world?  Do you want me to send 
something to the list about what it is that I'm planning to do or just go 
ahead with refactorings?

Finally, do you want me to work on the FastWriter optimisations that I posted 
about at as well?

Alex

> Nice work.
>
> Keats
>
> Alex Twisleton-Wykeham-Fiennes wrote:
> >All,
> >
> >I've just been spending a couple of days trying to track down why memory
> > gets consumed when reloading the same web application under tomcat which
> > according to my research is related to class loader confusion [1].
> >
> >During the course of my testing (running OptimizeIt across the tomcat5
> > process and analysing class loading and object creation across multiple
> > reloads), I've been noticing the number of Threads growing with each
> > reload which didn't seem correct at all.
> >
> >The increasing threads were getting allocated by ReloadingCacheManager and
> >where being utilised by the ClockDaemon process for triggering the
> > expiration of assets from the template and url Providers (using the
> > standard webmacro defaults file).
> >
> >These would normally get destroyed when the destroy() method was invoked
> > on the ReloadingCacheManager, but I couldn't find any where that enabled
> > me to do this easily.  I would have expected this to be invoked in the
> > destroy() method of WMServlet but this is not the case:-
> >    public synchronized void destroy ()
> >    {
> >        stop();
> >        _log.notice("stopped: " + this);
> >        _wm = null;
> >        _started = false;
> >        super.destroy();
> >    }
> >
> >In the end, I found it very hard to find a way to cleanly persuade the
> >webmacro instance to shut itself down properly, and ended up putting
> >something like this in my stop() method for my subclass of WMServlet:-
> >
> >  protected void stop()
> >  {
> >    // tidy up my stuff... <snip>
> >    Broker broker = getBroker();
> >    try {
> >      Provider templateProvider = broker.getProvider("template");
> >      templateProvider.destroy();
> >      Provider urlProvider = broker.getProvider("url");
> >      urlProvider.destroy();
> >    } catch (NotFoundException nfEx) {
> >      nfEx.printStackTrace();
> >    }
> >    super.stop();
> >  }
> >
> >Now this nicely tidies up the appropriate resources and when I reload this
> >application lots of times the thread count stays consistent, my memory is
> >much more under control and everything is a bit smoother.
> >
> >So, my question is:-
> >
> >Assuming that ReloadingCacheManager has to be destroy()ed correctly to get
> > it to shut down, and assuming that the application author doesn't know
> > which Providers are going to be implemented using a ReloadingCacheManager
> > (as it is defined by the properties classes at runtime), then what would
> > be the "correct" way of ensuring that the framework is correctly shut
> > down?
> >
> >Previously there was a definition of WebMacro.destroy() that stated "Call
> > this method when you are finished with WebMacro. If you don't call this
> > method, the Broker and all of WebMacro's caches may not be properly shut
> > down, potentially resulting in loss of data, and wasted memory. This
> > method is called in the finalizer, but it is best to call it as soon as
> > you know you are done with WebMacro.", but this has since been dropped
> > from the package.
> >
> >My gut feeling is that there should be a destroy() method on the Broker
> > that then invokes destroy() on all of the sub-components that it is
> > maintaining references to, but I am not that familiar with the webmacro
> > hierarchy...
> >
> >All tests have been run against 2.0b1, but the changelog to 2.0RC1 doesn't
> >touch upon any of these changes as far as I can see.
> >
> >Alex
> >
> >[1]
> >http://opensource2.atlassian.com/confluence/spring/pages/viewpage.action?p
> >ageId=2669
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live
> webcast and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] Thread leak with ReloadingCacheManager

[Keats K. <ke...@xa...>]

I haven't had a chance to validate what you are saying, but I know there 
has been a longstanding issue with getting WM Web apps to unload 
properly.  It sounds like you've finally nailed it.

I think your approach of adding a destroy method to the Broker makes 
sense.  If you're willing to work on it that would be great.

Nice work.

Keats

Alex Twisleton-Wykeham-Fiennes wrote:

>All,
>
>I've just been spending a couple of days trying to track down why memory gets 
>consumed when reloading the same web application under tomcat which according 
>to my research is related to class loader confusion [1].
>
>During the course of my testing (running OptimizeIt across the tomcat5 process 
>and analysing class loading and object creation across multiple reloads), 
>I've been noticing the number of Threads growing with each reload which 
>didn't seem correct at all.
>
>The increasing threads were getting allocated by ReloadingCacheManager and 
>where being utilised by the ClockDaemon process for triggering the expiration 
>of assets from the template and url Providers (using the standard webmacro 
>defaults file).
>
>These would normally get destroyed when the destroy() method was invoked on 
>the ReloadingCacheManager, but I couldn't find any where that enabled me to 
>do this easily.  I would have expected this to be invoked in the destroy() 
>method of WMServlet but this is not the case:-
>    public synchronized void destroy ()
>    {
>        stop();
>        _log.notice("stopped: " + this);
>        _wm = null;
>        _started = false;
>        super.destroy();
>    }
>
>In the end, I found it very hard to find a way to cleanly persuade the 
>webmacro instance to shut itself down properly, and ended up putting 
>something like this in my stop() method for my subclass of WMServlet:-
>
>  protected void stop() 
>  {
>    // tidy up my stuff... <snip>
>    Broker broker = getBroker();
>    try {
>      Provider templateProvider = broker.getProvider("template");
>      templateProvider.destroy();
>      Provider urlProvider = broker.getProvider("url");
>      urlProvider.destroy();
>    } catch (NotFoundException nfEx) {
>      nfEx.printStackTrace();
>    }
>    super.stop();
>  }
>
>Now this nicely tidies up the appropriate resources and when I reload this 
>application lots of times the thread count stays consistent, my memory is 
>much more under control and everything is a bit smoother.
>
>So, my question is:-
>
>Assuming that ReloadingCacheManager has to be destroy()ed correctly to get it 
>to shut down, and assuming that the application author doesn't know which 
>Providers are going to be implemented using a ReloadingCacheManager (as it is 
>defined by the properties classes at runtime), then what would be the 
>"correct" way of ensuring that the framework is correctly shut down?
>
>Previously there was a definition of WebMacro.destroy() that stated "Call this 
>method when you are finished with WebMacro. If you don't call this method, 
>the Broker and all of WebMacro's caches may not be properly shut down, 
>potentially resulting in loss of data, and wasted memory. This method is 
>called in the finalizer, but it is best to call it as soon as you know you 
>are done with WebMacro.", but this has since been dropped from the package.
>
>My gut feeling is that there should be a destroy() method on the Broker that 
>then invokes destroy() on all of the sub-components that it is maintaining 
>references to, but I am not that familiar with the webmacro hierarchy...
>
>All tests have been run against 2.0b1, but the changelog to 2.0RC1 doesn't 
>touch upon any of these changes as far as I can see.
>
>Alex
>
>[1] 
>http://opensource2.atlassian.com/confluence/spring/pages/viewpage.action?pageId=2669
>
>  
>

Re: [WebMacro-user] Optimising #eval creation of FastWriter

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Wed 8 March 2006 12:32, Alex Twisleton-Wykeham-Fiennes wrote:
> All,
>
> following on from my earlier optimising of Thread shutdowns inside
> webmacro, I've been doing a bit of actual CPU profiling of the running
> application and I found that a significant percentage of processor time was
> spent creating new FastWriter objects as part of the
> Template.evaluateAsBytes(String,Context) method invoked from the
> IncludeDirective.write(FastWriter,Context).

I've found a similar situation inside the EvalDirective that was creating 
FastWriters for each level of Eval.  Therefore a self-recursive #eval 
statement (such as discussed on the list recently) would generate a char[512] 
for every time it recursed which would all then be flattened down onto the 
parent one and eventually written out onto the original FastWriter.

I patched this as well to pass the master FastWriter down the chain, and it 
does indeed give a major performance increase (assuming that your templating 
structure uses a fair amount of #eval) (summary of patch at end of email).

The only point of concern for me about this is what happens to the state of 
the master FastWriter if the process fails at some point down the line.  In 
the default model, the PropertyException is thrown and the content of the 
FastWriter is not required and an error template is returned.  However, there 
may be situations when the PropertyException is caught and handled 
differently.  In this case, the master FastWriter would contain data up to 
the point when the PropertyException was thrown, whereas the previous model 
would contain data up to the start of the block which triggered the 
PropertyException.  Would this be a problem for anyone if it changed in this 
way?

Alex


------------------------------------------------------------

Changes to EvalDirective:-

    public void write(org.webmacro.FastWriter out, org.webmacro.Context 
context) throws org.webmacro.PropertyException, java.io.IOException
    {
        try
        {
// removed this as no longer necessary...
//             String s = null;
            Context c = null;
            
            Macro macro = (Macro)_evalTarget.getValue(context);
            if (_mapExpr == null)
            {
                // no map specified, use current context
// shift to write directly to master FastWriter:-
//               (String)macro.evaluate(context);
              macro.write(out, context);
            }
            else
            {
// <SNIP>
// shift to writ directly to master FastWriter
//                 s = (String)macro.evaluate(c);
                macro.write(out, c);
            }
// removed this as no longer necessary...
//             out.write(s);
        }
        catch (Exception e)
        {
            if (e instanceof PropertyException) throw (PropertyException)e;
            throw new PropertyException("#eval: Unable to evaluate macro.", 
e);
        }
    }

[WebMacro-user] Optimising #include creation of FastWriter

[Alex Twisleton-Wykeham-F. <al...@fi...>]

All,

following on from my earlier optimising of Thread shutdowns inside webmacro, 
I've been doing a bit of actual CPU profiling of the running application and 
I found that a significant percentage of processor time was spent creating 
new FastWriter objects as part of the 
Template.evaluateAsBytes(String,Context) method invoked from the 
IncludeDirective.write(FastWriter,Context).

I couldn't see any reason why you should need to create a new FastWriter just 
to write the template into it, convert it to a byte array and then write it 
into the outer FastWriter - especially as the inner FastWriter is only used 
for the duration of the evaluateAsBytes method and is initialised using the 
same encoding as the outer FastWriter.

I therefore made the following changes as an experiment:-

Added:-
public interface Template {
  public void write(FastWriter out, Context context)
    throws PropertyException, IOException;
}

changed:-
public class IncludeDirective {
  public void write(FastWriter out, Context context)
    throws PropertyExcception, IOException
  {
    <snip>
    case TYPE_TEMPLATE:
      out.write(((Template) toInclude).evaluateAsBytes(out.getEncoding(),
                context));
    <snip>
  }
}
to:-
public class IncludeDirective {
  public void write(FastWriter out, Context context)
    throws PropertyExcception, IOException
  {
    <snip>
    case TYPE_TEMPLATE:
      ((Template) toInclude).write(out, context);
    <snip>
  }
}


which has yielded a considerable performance increase in the #include 
approach, not to mention a much happier garbage collector (not having to 
allocate a char[512] every time that you include a template).

Are there any major gotcha's in this approach that I haven't spotted, and is 
it worth trying to look through the rest of the code at FastWriter creation 
and try and move it more towards a "streaming" style of model with a single 
FastWriter passed down through the items which is written to directly rather 
than a tree of FastWriters that bubble up their byte[] outputs?

Alex

[WebMacro-user] Thread leak with ReloadingCacheManager

[Alex Twisleton-Wykeham-F. <al...@fi...>]

All,

I've just been spending a couple of days trying to track down why memory gets 
consumed when reloading the same web application under tomcat which according 
to my research is related to class loader confusion [1].

During the course of my testing (running OptimizeIt across the tomcat5 process 
and analysing class loading and object creation across multiple reloads), 
I've been noticing the number of Threads growing with each reload which 
didn't seem correct at all.

The increasing threads were getting allocated by ReloadingCacheManager and 
where being utilised by the ClockDaemon process for triggering the expiration 
of assets from the template and url Providers (using the standard webmacro 
defaults file).

These would normally get destroyed when the destroy() method was invoked on 
the ReloadingCacheManager, but I couldn't find any where that enabled me to 
do this easily.  I would have expected this to be invoked in the destroy() 
method of WMServlet but this is not the case:-
    public synchronized void destroy ()
    {
        stop();
        _log.notice("stopped: " + this);
        _wm = null;
        _started = false;
        super.destroy();
    }

In the end, I found it very hard to find a way to cleanly persuade the 
webmacro instance to shut itself down properly, and ended up putting 
something like this in my stop() method for my subclass of WMServlet:-

  protected void stop() 
  {
    // tidy up my stuff... <snip>
    Broker broker = getBroker();
    try {
      Provider templateProvider = broker.getProvider("template");
      templateProvider.destroy();
      Provider urlProvider = broker.getProvider("url");
      urlProvider.destroy();
    } catch (NotFoundException nfEx) {
      nfEx.printStackTrace();
    }
    super.stop();
  }

Now this nicely tidies up the appropriate resources and when I reload this 
application lots of times the thread count stays consistent, my memory is 
much more under control and everything is a bit smoother.

So, my question is:-

Assuming that ReloadingCacheManager has to be destroy()ed correctly to get it 
to shut down, and assuming that the application author doesn't know which 
Providers are going to be implemented using a ReloadingCacheManager (as it is 
defined by the properties classes at runtime), then what would be the 
"correct" way of ensuring that the framework is correctly shut down?

Previously there was a definition of WebMacro.destroy() that stated "Call this 
method when you are finished with WebMacro. If you don't call this method, 
the Broker and all of WebMacro's caches may not be properly shut down, 
potentially resulting in loss of data, and wasted memory. This method is 
called in the finalizer, but it is best to call it as soon as you know you 
are done with WebMacro.", but this has since been dropped from the package.

My gut feeling is that there should be a destroy() method on the Broker that 
then invokes destroy() on all of the sub-components that it is maintaining 
references to, but I am not that familiar with the webmacro hierarchy...

All tests have been run against 2.0b1, but the changelog to 2.0RC1 doesn't 
touch upon any of these changes as far as I can see.

Alex

[1] 
http://opensource2.atlassian.com/confluence/spring/pages/viewpage.action?pageId=2669

Re: [WebMacro-user] Accessing property via "Unary Accessor"

[Sven S. <sv...@sc...>]

Ok, that makes sense. Many Thanks for your quick and comprehensive answer!

Eric B. Ridge wrote:
> WebMacro has to be precise in its spelling of properties because it's 
> the only way to provide predictable results.

<snip>

Re: [WebMacro-user] "recycle()" method not available anymore

[Eric B. R. <eb...@tc...>]

On Mar 7, 2006, at 1:28 PM, Holger King wrote:

> Dear group,
>
> in former webmacro releases a "Context"-class method called "recycle 
> ()" was available. This method has changed in current releases. Can  
> you tell me the corresponding java method in webmacro release 2.0.1?

I think we got rid of this because Context creation is a pretty cheap  
operation nowadays, and the advantages of caching/pooling/recycling  
Contexts didn't outweigh the added code we had to maintain.

Simply make a new Context in the same manner that you created the  
original.

eric

Re: [WebMacro-user] Accessing property via "Unary Accessor"

[Eric B. R. <eb...@tc...>]

WebMacro has to be precise in its spelling of properties because it's  
the only way to provide predictable results.

For example, what if your Bean also had:
	gettitle()
	getTitle()
         getTiTle()
	getTITLE()
	gettitlE()
	etc, etc

Which one would you want WebMacro to choose when the template  
designer typed $board.title?

This is somewhat of a hack, but if you want case-insensitive matching  
for property names, don't expose them at all through member  
functions, but instead implement a single get() method:

	public Object get(String proname) {
		return _propmap.get(proname.toLowerCase());
	}

This would allow any of these to return the same value:
	$board.title
	$board.Title
	$board.TITLE
	$board.TitLe
	etc, etc

But really, it's a horrible horrible idea and you shouldn't do it.   
Instead, just get used to WebMacro honoring the case of property  
names as they were defined in your class.  :)

eric


On Mar 7, 2006, at 7:15 PM, Sven Schliesing wrote:

> Hi,
>
> I'm having a class "Board" which has a private property "title".  
> For this property there is also a public getter "getTitle()", so  
> that webmacro can access it via "$board.Title"
>
> Unfortunately this was so uncommon to me, that I tried  
> "$board.title" for nearly two hours now before realizing what I was  
> doing wrong.
>
> What is the reason (I'm pretty sure that there is a good one :) )  
> that the accessor is not (also) trying "getTitle()" when using  
> "$board.title"? The property is "title" and not "Title", so. In  
> general a template designer should not even care about that. He  
> should know what properties the Bean got ("title" in my case) and  
> not how their getters are spelled.
>
>
> Or am I getting something totally wrong? :)
>
> Thanks for your help!
>
> Greetings from snowy Hamburg
>
> Sven
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting  
> language
> that extends applications into web and mobile media. Attend the  
> live webcast
> and join the prime developer group breaking into this new coding  
> territory!
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user

[WebMacro-user] Accessing property via "Unary Accessor"

[Sven S. <sv...@sc...>]

Hi,

I'm having a class "Board" which has a private property "title". For 
this property there is also a public getter "getTitle()", so that 
webmacro can access it via "$board.Title"

Unfortunately this was so uncommon to me, that I tried "$board.title" 
for nearly two hours now before realizing what I was doing wrong.

What is the reason (I'm pretty sure that there is a good one :) ) that 
the accessor is not (also) trying "getTitle()" when using 
"$board.title"? The property is "title" and not "Title", so. In general 
a template designer should not even care about that. He should know what 
properties the Bean got ("title" in my case) and not how their getters 
are spelled.


Or am I getting something totally wrong? :)

Thanks for your help!

Greetings from snowy Hamburg

Sven

[WebMacro-user] "recycle()" method not available anymore

[Holger K. <ki...@di...>]

Dear group,

in former webmacro releases a "Context"-class method called "recycle()" 
was available. This method has changed in current releases. Can you tell 
me the corresponding java method in webmacro release 2.0.1?

Thanx for helping me so far!

-- 
Herzliche Grüße,
Holger King

---------------------------------------------------
DIG Digitale Medienberatungs- und Produktions- GmbH
Neckarstrasse 1-5, D-78727 Oberndorf, Germany
Phone: +49 7423 8750-0
Fax: +49 7423 8750-23

mailto:ki...@di...
http://www.dig.de
---------------------------------------------------

Re: [WebMacro-user] Recursion within webmacro templates

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Tue 7 March 2006 09:48, Holger King wrote:

<snip>

> is it even possible to access plugged in WebContext-Tools like URLTool
> within a "templet" directive like
>
> >#templet $counter {
> >  #set $digit = $digit + 1
> >  #if ($digit < 10) {
> >    [$digit < 10]
>
>              $URL.getRequestURL()?counter=$digit

Yes it is possible to access variables from the main context.  You use a 
syntax like so (I've also fixed typos in previous copy paste):-

#set $outsidevariable = "finished!"
#templet $counter {
  #set $digit = $digit + 1
  #if ($digit < 10) {
    [$digit < 10]
    #eval $Self using { "digit": $digit }
  } #else {
    [$OuterVars.outsidevariable]
  }
}
#eval $counter using { "digit": 0 }

gives:-

[1 < 10] [2 < 10] [3 < 10] [4 < 10] [5 < 10] [6 < 10] [7 < 10] [8 < 10] [9 < 
10] [finished!]

For more docs see here:-
http://www.webmacro.org/TempletDirective

Alex