webmacro-user — List for WebMacro users, developers, competitors, and friends.

Re: [WebMacro-user] Optimising #include creation of FastWriter

[<Web...@St...>]

On Mon, 13 Mar 2006, Alex Twisleton-Wykeham-Fiennes wrote:

| >
| > I do it using _a new_ FastWriter with a temporary backing, then if
| > everything went right, I put the result to the FastWriter.
| 
| Do you use a seperate Context for each portlet as well?

Yes.

But assuming that the Portlets aren't _pure_ evil, this would have been a 
big thing: I could just have recreated the initial state before each 
rendering, by using a "weird-long-named" context-var for "my state", 
hoping that the corelet wouldn't intentially mess things up.

But I do make a new Context, due to this concern.

| 
| > This temporary backing paradigm obviously isn't the best, and a
| > mark/reset/flush paradigm would be much better for my part.
| 
| I can see where you are coming from on this, but it is very hard to state what 
| the side effects of flushing the FastWriter early and / or rolling it back to 
| an earlier state. 

Why is that? This seems completely illogical to me, I can't see at all 
where you're coming from.

If I still make new Contexts, and you understand that each portlet is "on 
its own" (with a logical separate context too), and if I don't commit 
bytes to the browser-stream before flush, then "rolling back" is _exactly_ 
the same as ditching the buffer, _as I do now_.

| I think that any such mechanism would need to be integrated with the 
| same ability on the Context that was being used as well for it to be 
| safe.

Make a new one?!

.. or as suggested above.

| 
| However, my gut reaction from the profiling that I've done suggests that if 
| what you are after is maximum performance from your system then the best way 
| of evaluating your portlets would be:-
| 
| #foreach $portlet in $portlets {
|   #include as template "$portlet.prepareView"
| }

Of course - which can't contain errors.

| 
| where the $portlet.prepareView prepares all the necessary state information 
| associated with the current view.  If this all goes smoothly in javaspace 
| then it returns the path to the standard view of the portlet, otherwise it 
| returns the path to the error template.

People don't program fantastic code. Suddenly, some method returns null. 
Suddenly, some seldom-used if-case kicks in, and everything goes haywire.

I really can't understand why my needs are so non-understandable?!

I am not constructing some obscure use-case here: This "damage control" is 
what I do now, I'd just like to have a more optimized way of doing it 
using the features that actually are present in the fastwriter already.

| 
| I would highly recommend getting hold of a copy of OptimizeIt and actually 
| benchmarking the performance of your portlet application before embarking on 
| refactoring the FastWriter in such a manner.

I use JProfiler, but thanks for the tip.

Endre

Re: [WebMacro-user] Can a FastWriter ever be recycled in the current model?

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Thu 16 March 2006 16:18, Keats Kirsch wrote:
> Alex Twisleton-Wykeham-Fiennes wrote:
> >... obvious to remove is
> >the ByteBufferOutputStream.reset() which currently is implemented as:-
> >
> >  /**
> >   * Clear the contents of the byte buffer.  Also shrinks the byte buffer
> >   * to the size specified during construction of this
> > ByteBufferOutputStream */
> >  public void reset ()
> >  {
> >      _pos = 0;
> >      _buf = new byte[_initialSize];
> >  }
> >
> >This is currently (in my testbed) only ever getting invoked during:-
> >
> >	ByteBufferOutputStream.reset()
>
> [I assume this is a typo and it doesn't actually call itself. ]

sorry, I was just copying from the call tree on OptimizeIt - the FastWriter 
flush() method invokes reset() on the ByteBufferOutputStream which does not 
call reset() on itself again...

> >	FastWriter.flush()
>
> I agree that this shouldn't re-allocate.
>
> >	FastWriter.close()
>
> No sense allocating on a close.
>
> >	WMTemplate.evaluateAsBytes()
>
> I'm not quite sure why this would reset anything.  It seems like you
> should explicitly flush or close after evaluating if you want to.

I think that there used to be a pool of FastWriters so it was tidying up the 
contents before returning it to the pool.  This is no longer there (as 
pointed out by Eric) so the code is no longer required.

A

> Keats
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live
> webcast and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] Can a FastWriter ever be recycled in the current model?

[Keats K. <ke...@xa...>]

Alex Twisleton-Wykeham-Fiennes wrote:

>... obvious to remove is 
>the ByteBufferOutputStream.reset() which currently is implemented as:-
>
>  /**
>   * Clear the contents of the byte buffer.  Also shrinks the byte buffer
>   * to the size specified during construction of this ByteBufferOutputStream
>   */
>  public void reset ()
>  {
>      _pos = 0;
>      _buf = new byte[_initialSize];
>  }
>
>This is currently (in my testbed) only ever getting invoked during:-
>
>	ByteBufferOutputStream.reset()
>  
>
[I assume this is a typo and it doesn't actually call itself. ]

>	FastWriter.flush()
>  
>
I agree that this shouldn't re-allocate.

>	FastWriter.close()
>  
>
No sense allocating on a close.

>	WMTemplate.evaluateAsBytes()
>  
>
I'm not quite sure why this would reset anything.  It seems like you 
should explicitly flush or close after evaluating if you want to.

Keats

Re: [WebMacro-user] Can a FastWriter ever be recycled in the current model?

[Marc P. <ma...@an...>]

On 16 Mar 2006, at 16:01, Keats Kirsch wrote:

> Alex Twisleton-Wykeham-Fiennes wrote:
>
>> ps any news on CVS status?  It's getting harder to keep track of  
>> what is experimental in my tree and what has been tested and  
>> posted to the list...
>>
> Alex, send me your SourceForge user name and I'll give you commit  
> rights (assuming no one objects).

Give the man the rights quick! Alex - truly appreciate your time on  
these sanity checks and optimisations.

Is the SF web CVS access working again? It was down yesterday.  
Somebody has mailed me about the Spring MVC integration code I've  
written and I wanted to be sure I have committed it.

Cheers

Re: [WebMacro-user] Can a FastWriter ever be recycled in the current model?

[Lane S. <la...@OP...>]

+1 from me.

Lane

--- ke...@xa... wrote:

From: Keats Kirsch <ke...@xa...>
To: web...@li...
Subject: Re: [WebMacro-user] Can a FastWriter ever be recycled in the current model?
Date: Thu, 16 Mar 2006 11:01:37 -0500

Alex Twisleton-Wykeham-Fiennes wrote:

>ps any news on CVS status?  It's getting harder to keep track of what is 
>experimental in my tree and what has been tested and posted to the list...
>  
>
Alex, send me your SourceForge user name and I'll give you commit rights 
(assuming no one objects).

Keats




-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Webmacro-user mailing list
Web...@li...
https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] Can a FastWriter ever be recycled in the current model?

[Eric B. R. <eb...@tc...>]

On Mar 16, 2006, at 6:01 AM, Alex Twisleton-Wykeham-Fiennes wrote:
> My questions are:-
>
> - is the FastWriter ever actually recycled?  It feels a bit like  
> there was a
> thought to recycle the FastWriters at some point, but currently (I  
> think) all
> FastWriters are being allocated via the FastWriter.getInstance()  
> methods that
> all allocate a new FastWriter instance.

FastWriter keeps a pool of FastWriter instances, doesn't it?  I know  
it did, maybe that got removed for 2.0?  Or are you saying that the  
pooling is broken and FastWriter.getInstance() always returns new  
instances.

<long pause>

Okay, the pool stuff has been ripped out of FastWriter.   
Interesting.  Since this is the now the case, I see no need for any  
of the .reset() business.  You're absolutely right, there's no point  
in re-allocating a smaller byte[] if the FastWriter instance is about  
to be gc'd.  I say get rid of it.

eric

Re: [WebMacro-user] Can a FastWriter ever be recycled in the current model?

[Keats K. <ke...@xa...>]

Alex Twisleton-Wykeham-Fiennes wrote:

>ps any news on CVS status?  It's getting harder to keep track of what is 
>experimental in my tree and what has been tested and posted to the list...
>  
>
Alex, send me your SourceForge user name and I'll give you commit rights 
(assuming no one objects).

Keats

[WebMacro-user] Can a FastWriter ever be recycled in the current model?

[Alex Twisleton-Wykeham-F. <al...@fi...>]

All,

I'm having a little look at memory consumption now.

The biggest culprit (by a factor of 2) in terms of size of allocated memory 
across a couple of hundred requests is byte[] buffers with the size 
percentages given as:-

	24% ByteBufferOutputStream.ensureCapacity()
	17% JDBC
	16% Tomcat
	9% ByteBufferOutputStream.getBytes()
	8% FastWriter()
	4% Buffer.getBytes()
	4% ByteBufferOutputStream.reset()
	4% ByteBufferOutputStream()
	3% JDBC
	2% Tomcat
	1% StreamTemplate()
	...

at a cursory glance, the only one that seems relatively obvious to remove is 
the ByteBufferOutputStream.reset() which currently is implemented as:-

  /**
   * Clear the contents of the byte buffer.  Also shrinks the byte buffer
   * to the size specified during construction of this ByteBufferOutputStream
   */
  public void reset ()
  {
      _pos = 0;
      _buf = new byte[_initialSize];
  }

This is currently (in my testbed) only ever getting invoked during:-

	ByteBufferOutputStream.reset()
	FastWriter.flush()
	FastWriter.close()
	WMTemplate.evaluateAsBytes()

My questions are:-

- is the FastWriter ever actually recycled?  It feels a bit like there was a 
thought to recycle the FastWriters at some point, but currently (I think) all 
FastWriters are being allocated via the FastWriter.getInstance() methods that 
all allocate a new FastWriter instance.

- if the FastWriter *is* recycled and the reset() funtion is actually 
required, then does it really matter if there is garbage in the buffer of the 
ByteBufferOutputStream left over from the earlier usage?  Surely so long as 
the pointer is reset back to the 0 point then everything is fine?  

The only issue with this that I can see is that if you were to create a very 
big ByteBufferOutputStream and then recycle it many times for small tasks, 
then you would hold onto a big byte[] array for a long time, but my feeling 
is that this is a minority case and not allocating a 4k (or 
FastWriter.DefaultBufferSize if defined) block of memory each time you finish 
doing your useful work with the buffer is a no-brainer.

If you just comment out the _buf = new byte[_initialSize] then all the unit 
tests still pass, but you immediately lose the 4% of memory comsumption, and 
gain the speed increase from not allocating the memory or having to garbage 
collect it.

Alex

ps any news on CVS status?  It's getting harder to keep track of what is 
experimental in my tree and what has been tested and posted to the list...

Re: [WebMacro-user] Optimising Variable to reduce instantiation overhead

[Keats K. <ke...@xa...>]

Endre Stølsvik wrote:

>On Sun, 12 Mar 2006, Alex Twisleton-Wykeham-Fiennes wrote:
>
>| All,
>| 
>| more performance tweaks, this one to the org.webmacro.engine.Variable class.
>
>Excellent performance tuneup you're into, Alex!
>
>| - made the getVariableName() method check to see if _vname has been 
>| initialised, and if not then build the name before returning it.  This is now 
>| synchronized as it changes the state of the object.
>
>So?
>
>Is it shared across threads?
>  
>
The answer is yes, as we were so rudely reminded when the dynamic 
#include bug was discovered recently.  Compiled template instances are 
cached and may be shared by multiple threads.

Keats

>Regards,
>Endre
>
>  
>

Re: [WebMacro-user] Optimising Variable to reduce instantiation overhead

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Mon 13 March 2006 12:34, Endre St=F8lsvik wrote:
> On Sun, 12 Mar 2006, Alex Twisleton-Wykeham-Fiennes wrote:
> | All,
> |
> | more performance tweaks, this one to the org.webmacro.engine.Variable
> | class.
>
> Excellent performance tuneup you're into, Alex!
>
> | - made the getVariableName() method check to see if _vname has been
> | initialised, and if not then build the name before returning it.  This =
is
> | now synchronized as it changes the state of the object.
>
> So?

If you don't synchronize then you could potentially have two threads modify=
ing=20
the state of the Object simultaneously.  In this situation it is not the en=
d=20
of the world, because the generation of the result is a) not that expensive=
,=20
and b) not dependent on other mutable properties, but I find it best to app=
ly=20
thread-safe approaches to everything that changes state and then remove the=
m=20
if profiling shows it to be a problem.  Much better to start safe and then=
=20
remove redundant safety when required than to start unsafe and try and make=
=20
things safe when a weird multi-threading bug occurs...

> Is it shared across threads?

I'd have to double check this one.  It's a bit of a twisty turny passage of=
=20
Macro inheritance and it isn't necessarily obvious...

Alex

ps synchronizing is really not the overhead that it used to be in older JVM=
s=20
and it *really* makes your life easier to not have any multi-threading=20
bugs...

Re: [WebMacro-user] Optimising #include creation of FastWriter

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Mon 13 March 2006 12:30, Endre St=F8lsvik wrote:
> | Just out of interest - how were you envisioning this rolling back
> | happening? Was it going to be a java space action or would it take place
> | during the template rendering phase?
>
> I already do this, dude - so all your arguments about why it ain't smart
> is simply not applicable.
>
> The clue is that this is a _portlet engine_, right? Every portlet operates
> on its own, and you may view the portal as an operating system with the
> portlets as applications. If you understand this view, then I guess you
> understand why containment of errors are both smart and correct, and the
> "internal state change" arguments are simply not interesting (Better to
> contain _one_ error than make it propagate all the way out to the users
> browser, so that he potentially can't see the portal page at all (due to
> messed up nesting of tables etc)).
>
> I do it using _a new_ FastWriter with a temporary backing, then if
> everything went right, I put the result to the FastWriter.

Do you use a seperate Context for each portlet as well?  If the portlet fai=
ls,=20
then how do you roll the Context back to the state that it was in at the=20
beginning of the evalaution of the portlet?

> This temporary backing paradigm obviously isn't the best, and a
> mark/reset/flush paradigm would be much better for my part.

I can see where you are coming from on this, but it is very hard to state w=
hat=20
the side effects of flushing the FastWriter early and / or rolling it back =
to=20
an earlier state.  I think that any such mechanism would need to be=20
integrated with the same ability on the Context that was being used as well=
=20
for it to be safe.

Either way, I'm trying not to create any functional changes to the code as =
a=20
whole at present unless it is really broken as I am not up to speed enough =
on=20
the internal architecture to state what the effect of the changes might be =
so=20
I suspect that I'm not going to look at this just now.

However, my gut reaction from the profiling that I've done suggests that if=
=20
what you are after is maximum performance from your system then the best wa=
y=20
of evaluating your portlets would be:-

#foreach $portlet in $portlets {
  #include as template "$portlet.prepareView"
}

where the $portlet.prepareView prepares all the necessary state information=
=20
associated with the current view.  If this all goes smoothly in javaspace=20
then it returns the path to the standard view of the portlet, otherwise it=
=20
returns the path to the error template.

of course you can still get an error while rendering your standard view, bu=
t=20
if all of the classes and methods that you are exposing to the template are=
=20
purely read only (ie the template is just a view rather than the controller=
)=20
then the performance increase that you get from not scripting the work up i=
n=20
webmacro will be greater than the performance increase that you would get b=
y=20
not generating a new FastWriter for each portlet. =20

WebMacro is great and quite surprisingly fast, but it isn't by definition a=
s=20
fast as doing the same thing in direct java code, so the less processing wo=
rk=20
that you perform inside the template, the better the ultimate performance=20
would be.

I would highly recommend getting hold of a copy of OptimizeIt and actually=
=20
benchmarking the performance of your portlet application before embarking o=
n=20
refactoring the FastWriter in such a manner.

Alex

Re: [WebMacro-user] Optimising Variable to reduce instantiation overhead

[<Web...@St...>]

On Sun, 12 Mar 2006, Alex Twisleton-Wykeham-Fiennes wrote:

| All,
| 
| more performance tweaks, this one to the org.webmacro.engine.Variable class.

Excellent performance tuneup you're into, Alex!

| - made the getVariableName() method check to see if _vname has been 
| initialised, and if not then build the name before returning it.  This is now 
| synchronized as it changes the state of the object.

So?

Is it shared across threads?

Regards,
Endre

Re: [WebMacro-user] Optimising #include creation of FastWriter

[<Web...@St...>]

| 
| Just out of interest - how were you envisioning this rolling back happening?  
| Was it going to be a java space action or would it take place during the 
| template rendering phase?

I already do this, dude - so all your arguments about why it ain't smart 
is simply not applicable.

The clue is that this is a _portlet engine_, right? Every portlet operates 
on its own, and you may view the portal as an operating system with the 
portlets as applications. If you understand this view, then I guess you 
understand why containment of errors are both smart and correct, and the 
"internal state change" arguments are simply not interesting (Better to 
contain _one_ error than make it propagate all the way out to the users 
browser, so that he potentially can't see the portal page at all (due to 
messed up nesting of tables etc)).

I do it using _a new_ FastWriter with a temporary backing, then if 
everything went right, I put the result to the FastWriter.

This temporary backing paradigm obviously isn't the best, and a 
mark/reset/flush paradigm would be much better for my part.

Regards,
Endre.

[WebMacro-user] Version of concurrent.jar?

[Alex Twisleton-Wykeham-F. <al...@fi...>]

Which version of concurrent.jar is packaged in the webmacro distribution?

I want to grab the identical source code so that my profiler can point me at 
what is going on inside the utilities invoked by webmacro...

Alex

Re: [WebMacro-user] Optimising Variable to reduce instantiation overhead

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Sun 12 March 2006 21:20, Alex Twisleton-Wykeham-Fiennes wrote:
> On Sun 12 March 2006 21:14, Lane Sharman wrote:
> > alex,
> >
> > thanks for this improvement and catch. Are there any issues or concerns
> > you have with this proposed update?
>
> none that I can see - there is a very slight performance hit checking to
> see whether or not _vname has been initialised inside getVariableName()
> (along with the synchronized lock acquisition), but this is only ever
> invoked while constructing a PropertyException which has such a huge
> overhead from building the stack trace that it is irrelevant.
>
> The only potential problem is that there are some 3rd party extensions of
> Variable that aren't included in the standard ant build which might break
> by making _vname private, but these are easily refactored to use the method
> call instead.

I spoke to soon (sort of).

Being relatively new to ant, I didn't realise that you had to call clean 
before compile and so there were some broken subclasses of Variable that no 
longer compile due to the private _vname change.

I've refactored these classes, and attach them to this email.

The (properly) compiled tree still passes all the unit tests.

Alex

> Alex
>
> ps I posted some patches to the list to make Broker shutdown cleanly and
> ensure that webmacro applications unload cleanly when reloading under
> tomcat. I was about 20k over the list limit, and am in a blocking queue. 
> Do you know who is the list admin to let this through, or should I repost
> with the fixes chopped into smaller amounts?
>
> > Lane
> >
> > --- al...@fi... wrote:
> >
> > From: Alex Twisleton-Wykeham-Fiennes <al...@fi...>
> > To: web...@li...
> > Subject: [WebMacro-user] Optimising Variable to reduce instantiation
> > overhead Date: Sun, 12 Mar 2006 18:50:17 +0000
> >
> > All,
> >
> > more performance tweaks, this one to the org.webmacro.engine.Variable
> > class.
> >
> > performance sampling showed a large proportion of CPU time being
> > allocated to the makeName(Object[] names) class that is invoked in the
> > constructor to create the human-readable version of the Object[] that
> > stores the names, which is then stored in _vname.
> >
> > However, _vname is only ever used in situations when you generating
> > PropertyException Objects to provide sensible error messages, and for all
> > "correctly functioning" instances, this is completely redundant.
> >
> > What I've done is:-
> >
> > - disabled the call to makeName(names) in the constructor
> > - made the getVariableName() method check to see if _vname has been
> > initialised, and if not then build the name before returning it.  This is
> > now synchronized as it changes the state of the object.
> > - made all previous references to _vname in the Variable class invoke
> > getVariableName() rather than accessing the variable directly.
> > - made _vname private rather than protected thereby forcing sub-classes
> > to access it via getVariableName()
> >
> > The resulting class (attached) passes all the unit tests and completely
> > removes this overhead for all normal usage patterns.
> >
> > Alex
> >
> >
> >
> > -------------------------------------------------------
> > This SF.Net email is sponsored by xPML, a groundbreaking scripting
> > language that extends applications into web and mobile media. Attend the
> > live webcast and join the prime developer group breaking into this new
> > coding territory!
> > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> > _______________________________________________
> > Webmacro-user mailing list
> > Web...@li...
> > https://lists.sourceforge.net/lists/listinfo/webmacro-user
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live
> webcast and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] Optimising Variable to reduce instantiation overhead

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Sun 12 March 2006 21:28, Lane Sharman wrote:
> Hi Alex,
>
> I would post the fixes one by one or send them to me as a zip.

I've forwarded you the blocked email directly.

> I am the lead release admin and I will look at them myself and wait for
> some more feedback before dropping them into cvs head.

OK.

> Also, do you want to do some development to webmacro as a cvs committer?

That would definitely make things a bit easier to keep track of what things 
are experimental (for me) and what I've sent to the list.  
Do you have standard formats for commit messages etc?  
and does everything happen in a single branch or does each developer work on 
their own branch? 

I don't envisage doing anything particularly radical, and I'm mainly 
interested in optimising and fine tuning existing algorithms and memory usage 
- webmacro is currently taking more overhead than my backend application 
server so it is the next obvious target for a bit of selective tweaking.

Are there any standard load tests for different elements of webmacro that 
measure performance rather than correctness?  
Is this something that would be useful to put together so that we can evaluate 
performance related versions against each other?
I'm very much aware that I'm only testing against my usage patterns, and 
although I'm pretty sure that I'm not slowing anything down for other people, 
there may be many other cases that I'm not touching on that could bring 
useful tweaks to move the process forwards.

Alex

> -Lane
>
> --- al...@fi... wrote:
>
> From: Alex Twisleton-Wykeham-Fiennes <al...@fi...>
> To: web...@li...
> Subject: Re: [WebMacro-user] Optimising Variable to reduce instantiation
> overhead Date: Sun, 12 Mar 2006 21:20:22 +0000
>
> On Sun 12 March 2006 21:14, Lane Sharman wrote:
> > alex,
> >
> > thanks for this improvement and catch. Are there any issues or concerns
> > you have with this proposed update?
>
> none that I can see - there is a very slight performance hit checking to
> see whether or not _vname has been initialised inside getVariableName()
> (along with the synchronized lock acquisition), but this is only ever
> invoked while constructing a PropertyException which has such a huge
> overhead from building the stack trace that it is irrelevant.
>
> The only potential problem is that there are some 3rd party extensions of
> Variable that aren't included in the standard ant build which might break
> by making _vname private, but these are easily refactored to use the method
> call instead.
>
> Alex
>
> ps I posted some patches to the list to make Broker shutdown cleanly and
> ensure that webmacro applications unload cleanly when reloading under
> tomcat. I was about 20k over the list limit, and am in a blocking queue. 
> Do you know who is the list admin to let this through, or should I repost
> with the fixes chopped into smaller amounts?
>
> > Lane
> >
> > --- al...@fi... wrote:
> >
> > From: Alex Twisleton-Wykeham-Fiennes <al...@fi...>
> > To: web...@li...
> > Subject: [WebMacro-user] Optimising Variable to reduce instantiation
> > overhead Date: Sun, 12 Mar 2006 18:50:17 +0000
> >
> > All,
> >
> > more performance tweaks, this one to the org.webmacro.engine.Variable
> > class.
> >
> > performance sampling showed a large proportion of CPU time being
> > allocated to the makeName(Object[] names) class that is invoked in the
> > constructor to create the human-readable version of the Object[] that
> > stores the names, which is then stored in _vname.
> >
> > However, _vname is only ever used in situations when you generating
> > PropertyException Objects to provide sensible error messages, and for all
> > "correctly functioning" instances, this is completely redundant.
> >
> > What I've done is:-
> >
> > - disabled the call to makeName(names) in the constructor
> > - made the getVariableName() method check to see if _vname has been
> > initialised, and if not then build the name before returning it.  This is
> > now synchronized as it changes the state of the object.
> > - made all previous references to _vname in the Variable class invoke
> > getVariableName() rather than accessing the variable directly.
> > - made _vname private rather than protected thereby forcing sub-classes
> > to access it via getVariableName()
> >
> > The resulting class (attached) passes all the unit tests and completely
> > removes this overhead for all normal usage patterns.
> >
> > Alex
> >
> >
> >
> > -------------------------------------------------------
> > This SF.Net email is sponsored by xPML, a groundbreaking scripting
> > language that extends applications into web and mobile media. Attend the
> > live webcast and join the prime developer group breaking into this new
> > coding territory!
> > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> > _______________________________________________
> > Webmacro-user mailing list
> > Web...@li...
> > https://lists.sourceforge.net/lists/listinfo/webmacro-user
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live
> webcast and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live
> webcast and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] Optimising Variable to reduce instantiation overhead

[Lane S. <la...@OP...>]

Hi Alex,

I would post the fixes one by one or send them to me as a zip.

I am the lead release admin and I will look at them myself and wait for some more feedback before dropping them into cvs head.

Also, do you want to do some development to webmacro as a cvs committer?

-Lane 

--- al...@fi... wrote:

From: Alex Twisleton-Wykeham-Fiennes <al...@fi...>
To: web...@li...
Subject: Re: [WebMacro-user] Optimising Variable to reduce instantiation overhead
Date: Sun, 12 Mar 2006 21:20:22 +0000

On Sun 12 March 2006 21:14, Lane Sharman wrote:
> alex,
>
> thanks for this improvement and catch. Are there any issues or concerns you
> have with this proposed update?

none that I can see - there is a very slight performance hit checking to see 
whether or not _vname has been initialised inside getVariableName() (along 
with the synchronized lock acquisition), but this is only ever invoked while 
constructing a PropertyException which has such a huge overhead from building 
the stack trace that it is irrelevant.

The only potential problem is that there are some 3rd party extensions of 
Variable that aren't included in the standard ant build which might break by 
making _vname private, but these are easily refactored to use the method call 
instead.

Alex

ps I posted some patches to the list to make Broker shutdown cleanly and 
ensure that webmacro applications unload cleanly when reloading under tomcat.  
I was about 20k over the list limit, and am in a blocking queue.  Do you know 
who is the list admin to let this through, or should I repost with the fixes 
chopped into smaller amounts?

> Lane
>
> --- al...@fi... wrote:
>
> From: Alex Twisleton-Wykeham-Fiennes <al...@fi...>
> To: web...@li...
> Subject: [WebMacro-user] Optimising Variable to reduce instantiation
> overhead Date: Sun, 12 Mar 2006 18:50:17 +0000
>
> All,
>
> more performance tweaks, this one to the org.webmacro.engine.Variable
> class.
>
> performance sampling showed a large proportion of CPU time being allocated
> to the makeName(Object[] names) class that is invoked in the constructor to
> create the human-readable version of the Object[] that stores the names,
> which is then stored in _vname.
>
> However, _vname is only ever used in situations when you generating
> PropertyException Objects to provide sensible error messages, and for all
> "correctly functioning" instances, this is completely redundant.
>
> What I've done is:-
>
> - disabled the call to makeName(names) in the constructor
> - made the getVariableName() method check to see if _vname has been
> initialised, and if not then build the name before returning it.  This is
> now synchronized as it changes the state of the object.
> - made all previous references to _vname in the Variable class invoke
> getVariableName() rather than accessing the variable directly.
> - made _vname private rather than protected thereby forcing sub-classes to
> access it via getVariableName()
>
> The resulting class (attached) passes all the unit tests and completely
> removes this overhead for all normal usage patterns.
>
> Alex
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live
> webcast and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Webmacro-user mailing list
Web...@li...
https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] Optimising Variable to reduce instantiation overhead

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Sun 12 March 2006 21:14, Lane Sharman wrote:
> alex,
>
> thanks for this improvement and catch. Are there any issues or concerns you
> have with this proposed update?

none that I can see - there is a very slight performance hit checking to see 
whether or not _vname has been initialised inside getVariableName() (along 
with the synchronized lock acquisition), but this is only ever invoked while 
constructing a PropertyException which has such a huge overhead from building 
the stack trace that it is irrelevant.

The only potential problem is that there are some 3rd party extensions of 
Variable that aren't included in the standard ant build which might break by 
making _vname private, but these are easily refactored to use the method call 
instead.

Alex

ps I posted some patches to the list to make Broker shutdown cleanly and 
ensure that webmacro applications unload cleanly when reloading under tomcat.  
I was about 20k over the list limit, and am in a blocking queue.  Do you know 
who is the list admin to let this through, or should I repost with the fixes 
chopped into smaller amounts?

> Lane
>
> --- al...@fi... wrote:
>
> From: Alex Twisleton-Wykeham-Fiennes <al...@fi...>
> To: web...@li...
> Subject: [WebMacro-user] Optimising Variable to reduce instantiation
> overhead Date: Sun, 12 Mar 2006 18:50:17 +0000
>
> All,
>
> more performance tweaks, this one to the org.webmacro.engine.Variable
> class.
>
> performance sampling showed a large proportion of CPU time being allocated
> to the makeName(Object[] names) class that is invoked in the constructor to
> create the human-readable version of the Object[] that stores the names,
> which is then stored in _vname.
>
> However, _vname is only ever used in situations when you generating
> PropertyException Objects to provide sensible error messages, and for all
> "correctly functioning" instances, this is completely redundant.
>
> What I've done is:-
>
> - disabled the call to makeName(names) in the constructor
> - made the getVariableName() method check to see if _vname has been
> initialised, and if not then build the name before returning it.  This is
> now synchronized as it changes the state of the object.
> - made all previous references to _vname in the Variable class invoke
> getVariableName() rather than accessing the variable directly.
> - made _vname private rather than protected thereby forcing sub-classes to
> access it via getVariableName()
>
> The resulting class (attached) passes all the unit tests and completely
> removes this overhead for all normal usage patterns.
>
> Alex
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live
> webcast and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] Optimising Variable to reduce instantiation overhead

[Lane S. <la...@OP...>]

alex,

thanks for this improvement and catch. Are there any issues or concerns you have with this proposed update?

Lane

--- al...@fi... wrote:

From: Alex Twisleton-Wykeham-Fiennes <al...@fi...>
To: web...@li...
Subject: [WebMacro-user] Optimising Variable to reduce instantiation overhead
Date: Sun, 12 Mar 2006 18:50:17 +0000

All,

more performance tweaks, this one to the org.webmacro.engine.Variable class.

performance sampling showed a large proportion of CPU time being allocated to 
the makeName(Object[] names) class that is invoked in the constructor to 
create the human-readable version of the Object[] that stores the names, 
which is then stored in _vname.

However, _vname is only ever used in situations when you generating 
PropertyException Objects to provide sensible error messages, and for all 
"correctly functioning" instances, this is completely redundant.

What I've done is:-

- disabled the call to makeName(names) in the constructor
- made the getVariableName() method check to see if _vname has been 
initialised, and if not then build the name before returning it.  This is now 
synchronized as it changes the state of the object.
- made all previous references to _vname in the Variable class invoke 
getVariableName() rather than accessing the variable directly.
- made _vname private rather than protected thereby forcing sub-classes to 
access it via getVariableName() 

The resulting class (attached) passes all the unit tests and completely 
removes this overhead for all normal usage patterns.

Alex

[WebMacro-user] Optimising Variable to reduce instantiation overhead

[Alex Twisleton-Wykeham-F. <al...@fi...>]

All,

more performance tweaks, this one to the org.webmacro.engine.Variable class.

performance sampling showed a large proportion of CPU time being allocated to 
the makeName(Object[] names) class that is invoked in the constructor to 
create the human-readable version of the Object[] that stores the names, 
which is then stored in _vname.

However, _vname is only ever used in situations when you generating 
PropertyException Objects to provide sensible error messages, and for all 
"correctly functioning" instances, this is completely redundant.

What I've done is:-

- disabled the call to makeName(names) in the constructor
- made the getVariableName() method check to see if _vname has been 
initialised, and if not then build the name before returning it.  This is now 
synchronized as it changes the state of the object.
- made all previous references to _vname in the Variable class invoke 
getVariableName() rather than accessing the variable directly.
- made _vname private rather than protected thereby forcing sub-classes to 
access it via getVariableName() 

The resulting class (attached) passes all the unit tests and completely 
removes this overhead for all normal usage patterns.

Alex

Re: [WebMacro-user] Optimising #include creation of FastWriter

[Tim P. <ti...@pa...>]

On Thursday 09 March 2006 23:11, Keats Kirsch wrote:
> If my memory serves (which is dubious) the FastWriter caches an entire
> page before flushing to the output stream.  This was a controversial
> design decision -- there were a lot of complaints about potential
> performance impacts, but no metrics that I'm aware of.  At one point I
> developed a #flush directive to work around this for a particular
> application.
>
Yes, Lane and I gave up on trying to reintroduce the lost functionality in 
Melati. 
It is still missed. 

If you are trying to paginate a book this is a process that might take fifteen 
or more minutes, so the status page needs to be updated every page or chapter 
to keep the browser believing something is happening. 

Really nice to see some activity on WM.

Cheers
tim pizey

-- 
Registered linux user #21337, http://counter.li.org.

Re: [WebMacro-user] Optimising #include creation of FastWriter

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Thu 9 March 2006 23:11, Keats Kirsch wrote:
> If my memory serves (which is dubious) the FastWriter caches an entire
> page before flushing to the output stream.  This was a controversial
> design decision -- there were a lot of complaints about potential
> performance impacts, but no metrics that I'm aware of.  At one point I
> developed a #flush directive to work around this for a particular
> application.

I have a vague memory about this - wasn't it to do with the time before the=
=20
first data is sent to the client that was under discussion rather than the=
=20
overall system throughput?

> In any case, if I'm correct, it should be a simple matter to do the sort
> of checkpoint/rollback mechanism that is being discussed. =20

until someone used your #flush directive (or called flush() on the FastWrit=
er=20
inside one of their own directives).  Then all your assumptions about=20
rollback go out the window - you'd have to restrict access to the flush=20
method to prevent this happening...

> But I'm not=20
> sure this is a good idea.  It would be adding more dependencies on the
> FastWriter and it seems like an overly complex error handling
> architecture.  We already have the pluggable EEH mechanism which seems
> pretty adequate.  Generally the error "culture" in WM is, if you
> anticipate errors, handle them in code.  Unanticipated errors should
> result in an error template being rendered.
>
> I think the bigger issue is with the FastWriter mechanism as a whole.
> It should be an interface so that you can plug in an implementation that
> serves your needs.  This is much more in line with the flexibility that
> permeates nearly every other aspect of WM.

I've just been having a quick scan through the source for which methods are=
=20
used in FastWriter, and it is pretty much everything across quite a wide=20
variety of situations.  If you were therefore going to make an interface fr=
om=20
it then it would pretty much duplicate the structure of the current=20
=46astWriter API.  What alterations would you make to the implemenation of=
=20
=46astWriter if you did have an extendable structure?

My feeling is that an easier to implement and probably functionally more=20
interesting approach would be to make the ByteBufferOutputStream into an=20
interface with a Factory method that gets past to the FastWriter on=20
construction.  Most of the changes that I could see being useful=20
(auto-flushing to the output stream to save memory etc etc) would take plac=
e=20
at this level in the code and wouldn't touch the FastWriter interface and=20
therefore wouldn't mean refactoring pretty much everything else.

Alex

> My 2p.
>
> Keats
>
> Alex Twisleton-Wykeham-Fiennes wrote:
> >On Thu 9 March 2006 17:00, Endre St=F8lsvik wrote:
> >>On Wed, 8 Mar 2006, Eric B. Ridge wrote:
> >>| I've been thinking back to when I write IncludeDirective and I simply
> >>| cannot remember why it makes a new FastWriter.  Was it because the old
> >>| ParseDirective did that and I was just coping it?  Was it out of sheer
> >>| stupidity?  Or maybe some reason about not wanting to muck with the
> >>| real FastWriter in the event of a parsing/evaluation exception.  I
> >>| dunno.
> >>
> >>You should have commented it in the code.
> >>
> >>..
> >>
> >>One comment about the FastWriter/Writer thingy: A "resetable" writer
> >> would be fantastic, preferrably having unlimited resetable space (just
> >> growing adding byte-arrays into some list as needed).
> >>  I am responsible for a portal, which renders every portlet in its own
> >>context and with its own "StringWriter", so that if it throws an
> >>exception, I can throw it away, and instead render an error-portlet. If
> >>all goes well, I commit the fully rendered portlet to the FastWriter, a=
nd
> >>move along.
> >>
> >>However, this would be hugely better if I could just _always_ pass along
> >>the same fastwriter recursing down includes and macros and whatnots.
> >>Before I venture into the next portlet, i do a mark(). Then I let it
> >>render the portlet, with its recursing. Then if an exception turns up,
> >>I'll just do a reset(), and then output an error-template instead, while
> >>if it goes well, i'll do a _flush()_ to commit the stuff all the way to
> >>the browser at the other end of the stream (I feel this should be up to
> >>me, the webapp writer, to decide when to flush). That would have been
> >>absolutely great, and absolutely more performant.
> >
> >It's not going to be extending FastWriter because after a cursory look at
> > the code it appears to be caching small (<4k blocks) of binary data in
> > the FastWriter and then pushing them out to the underlying OutputStream
> > whenever it needs to get it's internal buffer into some kind of known
> > state.
> >
> >Therefore if it had already pushed data out to the OutputStream when you
> > tried to rollback your code to the last point then it would fail because
> > it would no longer have access to or control of the data.
> >
> >To do this properly you would need to maintain a single data structure
> > that holds the entire contents of whatever is written to it in memory a=
nd
> > then shifts it out once you've satisfied yourself that it isn't going to
> > need to rollback.
> >
> >I don't think that you are going to do this by extending FastWriter - it
> > just doesn't feel right.  Unfortunately great chunks of WebMacro are
> > expecting FastWriter as their target output stream so at present you are
> > kind of stumped.  Might be interesting to look at what the difference is
> > between Writer and FastWriter and refactor the calling code if it is not
> > too much hassle?
> >
> >However, the more I think about it, the more I feel that this "rolling
> > back" is not a nice idea, for the following reasons:-
> >
> >- the success / failure thing should be a javaspace thing and to only ha=
ve
> > it occur inside the middle of a portlet feels a bit odd to me.  I kind =
of
> > feel that when the portlet rendering template is invoked, that the
> > context should already contain a set of calculated portlets (or error
> > portlets) that reflect the state of the system.  Then the rollback is
> > just not necessary.
> >
> >- rolling back the output is only half of the problem.  webmacro code can
> > make changes to the state of the backend java system and to the state of
> > the context.  Just because you roll the output stream back doesn't mean
> > that the java and context are in any way in a consistent state with what
> > they were before the rollback point.  To imply that they are is just
> > tempting fate...
> >
> >Just out of interest - how were you envisioning this rolling back
> > happening? Was it going to be a java space action or would it take place
> > during the template rendering phase?
> >
> >Alex
> >
> >
> >-------------------------------------------------------
> >This SF.Net email is sponsored by xPML, a groundbreaking scripting
> > language that extends applications into web and mobile media. Attend the
> > live webcast and join the prime developer group breaking into this new
> > coding territory!
> > http://sel.as-us.falkag.net/sel?cmd=3Dk&kid=110944&bid$1720&dat=121642
> >_______________________________________________
> >Webmacro-user mailing list
> >Web...@li...
> >https://lists.sourceforge.net/lists/listinfo/webmacro-user
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting langua=
ge
> that extends applications into web and mobile media. Attend the live
> webcast and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D110944&bid=3D241720&dat=
=3D121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] Optimising #include creation of FastWriter

[Keats K. <ke...@xa...>]

If my memory serves (which is dubious) the FastWriter caches an entire 
page before flushing to the output stream.  This was a controversial 
design decision -- there were a lot of complaints about potential 
performance impacts, but no metrics that I'm aware of.  At one point I 
developed a #flush directive to work around this for a particular 
application. 

In any case, if I'm correct, it should be a simple matter to do the sort 
of checkpoint/rollback mechanism that is being discussed.  But I'm not 
sure this is a good idea.  It would be adding more dependencies on the 
FastWriter and it seems like an overly complex error handling 
architecture.  We already have the pluggable EEH mechanism which seems 
pretty adequate.  Generally the error "culture" in WM is, if you 
anticipate errors, handle them in code.  Unanticipated errors should 
result in an error template being rendered.

I think the bigger issue is with the FastWriter mechanism as a whole.  
It should be an interface so that you can plug in an implementation that 
serves your needs.  This is much more in line with the flexibility that 
permeates nearly every other aspect of WM.

My 2p.

Keats

Alex Twisleton-Wykeham-Fiennes wrote:

>On Thu 9 March 2006 17:00, Endre Stølsvik wrote:
>  
>
>>On Wed, 8 Mar 2006, Eric B. Ridge wrote:
>>| I've been thinking back to when I write IncludeDirective and I simply
>>| cannot remember why it makes a new FastWriter.  Was it because the old
>>| ParseDirective did that and I was just coping it?  Was it out of sheer
>>| stupidity?  Or maybe some reason about not wanting to muck with the real
>>| FastWriter in the event of a parsing/evaluation exception.  I dunno.
>>
>>You should have commented it in the code.
>>
>>..
>>
>>One comment about the FastWriter/Writer thingy: A "resetable" writer would
>>be fantastic, preferrably having unlimited resetable space (just growing
>>adding byte-arrays into some list as needed).
>>  I am responsible for a portal, which renders every portlet in its own
>>context and with its own "StringWriter", so that if it throws an
>>exception, I can throw it away, and instead render an error-portlet. If
>>all goes well, I commit the fully rendered portlet to the FastWriter, and
>>move along.
>>
>>However, this would be hugely better if I could just _always_ pass along
>>the same fastwriter recursing down includes and macros and whatnots.
>>Before I venture into the next portlet, i do a mark(). Then I let it
>>render the portlet, with its recursing. Then if an exception turns up,
>>I'll just do a reset(), and then output an error-template instead, while
>>if it goes well, i'll do a _flush()_ to commit the stuff all the way to
>>the browser at the other end of the stream (I feel this should be up to
>>me, the webapp writer, to decide when to flush). That would have been
>>absolutely great, and absolutely more performant.
>>    
>>
>
>It's not going to be extending FastWriter because after a cursory look at the 
>code it appears to be caching small (<4k blocks) of binary data in the 
>FastWriter and then pushing them out to the underlying OutputStream whenever 
>it needs to get it's internal buffer into some kind of known state.
>
>Therefore if it had already pushed data out to the OutputStream when you tried 
>to rollback your code to the last point then it would fail because it would 
>no longer have access to or control of the data.
>
>To do this properly you would need to maintain a single data structure that 
>holds the entire contents of whatever is written to it in memory and then 
>shifts it out once you've satisfied yourself that it isn't going to need to 
>rollback.
>
>I don't think that you are going to do this by extending FastWriter - it just 
>doesn't feel right.  Unfortunately great chunks of WebMacro are expecting 
>FastWriter as their target output stream so at present you are kind of 
>stumped.  Might be interesting to look at what the difference is between 
>Writer and FastWriter and refactor the calling code if it is not too much 
>hassle?
>
>However, the more I think about it, the more I feel that this "rolling back" 
>is not a nice idea, for the following reasons:-
>
>- the success / failure thing should be a javaspace thing and to only have it 
>occur inside the middle of a portlet feels a bit odd to me.  I kind of feel 
>that when the portlet rendering template is invoked, that the context should 
>already contain a set of calculated portlets (or error portlets) that reflect 
>the state of the system.  Then the rollback is just not necessary.
>
>- rolling back the output is only half of the problem.  webmacro code can make 
>changes to the state of the backend java system and to the state of the 
>context.  Just because you roll the output stream back doesn't mean that the 
>java and context are in any way in a consistent state with what they were 
>before the rollback point.  To imply that they are is just tempting fate...
>
>Just out of interest - how were you envisioning this rolling back happening?  
>Was it going to be a java space action or would it take place during the 
>template rendering phase?
>
>Alex
>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by xPML, a groundbreaking scripting language
>that extends applications into web and mobile media. Attend the live webcast
>and join the prime developer group breaking into this new coding territory!
>http://sel.as-us.falkag.net/sel?cmd=k&kid0944&bid$1720&dat1642
>_______________________________________________
>Webmacro-user mailing list
>Web...@li...
>https://lists.sourceforge.net/lists/listinfo/webmacro-user
>
>  
>

Re: [WebMacro-user] Optimising #include creation of FastWriter

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Thu 9 March 2006 17:00, Endre St=F8lsvik wrote:
> On Wed, 8 Mar 2006, Eric B. Ridge wrote:
> | I've been thinking back to when I write IncludeDirective and I simply
> | cannot remember why it makes a new FastWriter.  Was it because the old
> | ParseDirective did that and I was just coping it?  Was it out of sheer
> | stupidity?  Or maybe some reason about not wanting to muck with the real
> | FastWriter in the event of a parsing/evaluation exception.  I dunno.
>
> You should have commented it in the code.
>
> ..
>
> One comment about the FastWriter/Writer thingy: A "resetable" writer would
> be fantastic, preferrably having unlimited resetable space (just growing
> adding byte-arrays into some list as needed).
>   I am responsible for a portal, which renders every portlet in its own
> context and with its own "StringWriter", so that if it throws an
> exception, I can throw it away, and instead render an error-portlet. If
> all goes well, I commit the fully rendered portlet to the FastWriter, and
> move along.
>
> However, this would be hugely better if I could just _always_ pass along
> the same fastwriter recursing down includes and macros and whatnots.
> Before I venture into the next portlet, i do a mark(). Then I let it
> render the portlet, with its recursing. Then if an exception turns up,
> I'll just do a reset(), and then output an error-template instead, while
> if it goes well, i'll do a _flush()_ to commit the stuff all the way to
> the browser at the other end of the stream (I feel this should be up to
> me, the webapp writer, to decide when to flush). That would have been
> absolutely great, and absolutely more performant.

It's not going to be extending FastWriter because after a cursory look at t=
he=20
code it appears to be caching small (<4k blocks) of binary data in the=20
=46astWriter and then pushing them out to the underlying OutputStream whene=
ver=20
it needs to get it's internal buffer into some kind of known state.

Therefore if it had already pushed data out to the OutputStream when you tr=
ied=20
to rollback your code to the last point then it would fail because it would=
=20
no longer have access to or control of the data.

To do this properly you would need to maintain a single data structure that=
=20
holds the entire contents of whatever is written to it in memory and then=20
shifts it out once you've satisfied yourself that it isn't going to need to=
=20
rollback.

I don't think that you are going to do this by extending FastWriter - it ju=
st=20
doesn't feel right.  Unfortunately great chunks of WebMacro are expecting=20
=46astWriter as their target output stream so at present you are kind of=20
stumped.  Might be interesting to look at what the difference is between=20
Writer and FastWriter and refactor the calling code if it is not too much=20
hassle?

However, the more I think about it, the more I feel that this "rolling back=
"=20
is not a nice idea, for the following reasons:-

=2D the success / failure thing should be a javaspace thing and to only hav=
e it=20
occur inside the middle of a portlet feels a bit odd to me.  I kind of feel=
=20
that when the portlet rendering template is invoked, that the context shoul=
d=20
already contain a set of calculated portlets (or error portlets) that refle=
ct=20
the state of the system.  Then the rollback is just not necessary.

=2D rolling back the output is only half of the problem.  webmacro code can=
 make=20
changes to the state of the backend java system and to the state of the=20
context.  Just because you roll the output stream back doesn't mean that th=
e=20
java and context are in any way in a consistent state with what they were=20
before the rollback point.  To imply that they are is just tempting fate...

Just out of interest - how were you envisioning this rolling back happening=
? =20
Was it going to be a java space action or would it take place during the=20
template rendering phase?

Alex

Re: [WebMacro-user] patches to avoid FastWriter creation

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Thu 9 March 2006 17:51, Alex Twisleton-Wykeham-Fiennes wrote:
> On Thu 9 March 2006 16:58, Eric B. Ridge wrote:
> > On Mar 9, 2006, at 11:50 AM, Alex Twisleton-Wykeham-Fiennes wrote:
> > > Which jar is org.apache.tools.ant.taskdefs.optional.javacc.JavaCC
> > > actually
> > > supposed to be in because I don't find it on a cursory glance.  The
> > > rule
> > > which specifies it is this one:-
> >
> > Oh right, you also need the .jar of optional And taskdefs.  Back in
> > the Ant 1.2 days it was an additional download from Jakarta called
> > "ant-optional.jar" (or something similar).  I'm not sure what the
> > latest versions of Ant do, but I suspect it's similar.
> >
> > That jar also needs to be in $ANT_HOME/lib.
>
> This is prooving to be a very frustrating twisty turny web of passages all
> alike.
>
> If anyone has the appropriate jar file that I should be linking to get the
> javacc build target to work with ant-1.6.2-5 on SuSE Linux 10.0 then it
> would be very much appreciated.
>
> (the ant-javadoc-1.6.2-7 on SuSE 10.0 contains the javadoc for the class in
> question, I just can't find / they've ommitted the rpm with the actual jar
> file in it)

Aha.   locked in the bottom of a filing cabinet in a disused toilet with a 
sign on the door saying "beware of the leopard".

why didn't I think of that before?

in the off chance that anyone else is looking for it, the rpm you want is 
called ant-nodeps.rpm

*sigh*

Anyway, the good news is that all of the unit tests pass apart from one:-

    [junit] Running org.webmacro.adapter.TestSpringView
    [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.839 sec
    [junit] Test org.webmacro.adapter.TestSpringView FAILED

I'm just waiting to be allowed into the sourceforge CVS server to check out a 
clean version and see if this is my fault or not but it is being veerrrrryyyy 
slow at present - is it just anonymous access that is throttled so badly?  If 
not then would you like the use of a slightly less loaded CVS server for the 
project?

Some time later:  OK, a fresh checkout from CVS fails on the TestSpringView as 
well as my patched version.  This "presumably" means my patches are OK (or at 
least haven't made it any worse)...  

Alex