webmacro-user — List for WebMacro users, developers, competitors, and friends.

Re: [WebMacro-user] Formatting a date

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Tue 28 March 2006 10:06, Endre St=F8lsvik wrote:
> With how many threads did you test this? 1000? 100? 10? 1? And how
> "real-life" was the scenario: how many date-formattings will one
> typically do within one rendering?

Much as I hate to rise to the occasion, I'll just attempt to put this one t=
o=20
bed.

creating: 4007
HashMap: 1295
ConcurrentHashMap: 1292

creating: 3787
HashMap: 1285
ConcurrentHashMap: 1289

creating: 3795
HashMap: 1284
ConcurrentHashMap: 1288

creating: 3787
HashMap: 1290
ConcurrentHashMap: 1287

creating: 3786
HashMap: 1287
ConcurrentHashMap: 1288

creating: 3797
HashMap: 1289
ConcurrentHashMap: 1457

creating: 4031
HashMap: 1320
ConcurrentHashMap: 1319

creating: 3882
HashMap: 1288
ConcurrentHashMap: 1305

creating: 3866
HashMap: 1308
ConcurrentHashMap: 1311

creating: 4244
HashMap: 1292
ConcurrentHashMap: 1328

This is the same as the original test, just repeated 10 times to give the j=
it=20
time to kick in.  In addition, the tests have been changed as follows:-

Creator: new SimpleDateFormat for each invocation
HashMap: using an unsynchronized HashMap for access
ConcurrentHashMap: using an unsynchronized ConcurrentHashMap for access fro=
m=20
the concurrent.jar shipped with webmacro (does anyone know what version thi=
s=20
is?)

So that deals with the JIT issues.

> This illustrates the point about synchs that I'm trying to make: synchs
> are pretty much not noticeable at all if you access them with one thread
> (but there was a difference, and it was in the bad direction, right?).
> However, _contended_ synchs are the bad stuff. A conteded synch forces a
> complete flush of your memory-caches (read the java spec on synch if you
> haven't yet). On "small systems", this isn't that big a deal, as they tend
> to a) have one CPU (thus really only one cache), and b) have coherent
> caches if they are more. On larger systems, where CPUs might be more
> independent ("NUMA"), this can be a huge overhead.

Hence the usage of ConcurrentHashMap in the updated test.  ConcurrentHashMa=
p=20
*does* sync but only if the first lookup fails to return a sensible result=
=20
(therefore implying concurrent modification and reading).  This has therefo=
re=20
moved the percentage chance of a contended sync from "two threads want to=20
look up a SimpleDateFormat in the time it takes to read from a HashMap" to=
=20
"one thread wants to look up a SimpleDateFormat from a ConcurrentHashMap=20
while another thread is storing a newly initialised SimpleDateFormat at=20
exactly the same time" which kind of is "good enough" odds to me that it=20
isn't going to cripple my application.

=46inally, if we are going to be planning on boycotting all "hidden caches"=
 of=20
code, have you actually looked at the source code for SimpleDateFormat?  On=
e=20
of the first things that it does when it boots up is to retrieve the=20
NumberFormat that it is going to use from a nice little HashMap of cached=20
NumberFormat items with Locale as the key.  So the outer Map of=20
SimpleDateFormats is evil and should be stopped, and the inner Map of=20
NumberFormats is good and we don't care about it?

The final thing to notice about SimpleDateFormat is that it really isn't a=
=20
nice thing to do to your garbage collector.  It really does generate a=20
*large* number of temporary items during the initialisation of=20
SimpleDateFormat, all of which will need garbage collecting (~40 Objects=20
taking up ~1k per init) .  Looking at the initialiser makes it pretty obvio=
us=20
that it is designed to be initialised once and then run multiple times as t=
he=20
parse() and format() methods have been designed to minimise object creation=
=20
and are quite streamlined whereas the creator is a bit of a beast.

Alex

Re: [WebMacro-user] Formatting a date

[Tim P. <ti...@pa...>]

On Wednesday 29 March 2006 00:41, Keats Kirsch wrote:
> Tim Pizey wrote:
> >I have every confidence that these plugable tools will not slow me or
> > webmacro down if I don't use them,
>
> Exactly.
>
> >but I cannot understand why anyone would use them.
>
> Personally, I think this fits well with the separation of concerns.  A
> template designer should be able to choose and modify the display format
> of a date without going back to the Java coder.
>
Ah, yes, I keep forgetting that not everyone is a one man band :)

I don't know how many WM users actually fit the organisational  paradigm 
that has always been espoused in the WM philosophical discussions of java 
gurus and template monkeys. 

To the extent that this paradigm ever had any actual instances I suspect 
that they migrated to VM land a while back. 

I share Alex's reasons for sticking with WM, its a good natured league of 
gentlemen. 

I very much appreciate the fact that WM has not been abandoned, and I 
still dream of committing something one day. 

thanks a lot everyone

Tim

-- 
Registered linux user #21337, http://counter.li.org.

Re: [WebMacro-user] Infinite loop with CountDirective

[Keats K. <ke...@xa...>]

Thanks Sven.

I've gone through the code and found one other place where this loop 
could occur; the TypeDirective.  I have committed a patch for that as 
well.  There is currently no unit test for this directive; I'll try to 
add one.  (Does anybody use this?)

It occurs to me that we are still vulnerable to infinite loops with 
other Macro implementations.  I noticed in the PropertyMethod class that 
it actually tests if the Macro evaluates to itself.  This takes us one 
step further, but you could still have an infinite loop if there is a 
multiple class circular reference.  E.g., MacroA evals to MacroB which 
evals to MacroA.  It's probably not worth guarding against this somewhat 
arcane case.  What do folks think?

Keats

Sven Schliesing wrote:

> Works for me.
> I get:
> ----
> Pages:    [<!-- Undefined expression in #count directive. at 
> templates\default\forum\showBoard.wm:4.12 -->]
> ----
> if not specifying an end-value.
>
> Thanks for the quick fix!
>
> Sven
>
>
> Keats Kirsch wrote:
>
>> OK, I've committed a fixed version along with a new unit test.  
>> Please check it out and let me know if you find any problems.
>>
>> Keats
>
>

Re: [WebMacro-user] Infinite loop with CountDirective

[Sven S. <sv...@sc...>]

Works for me.
I get:
----
Pages:	[<!-- Undefined expression in #count directive. at 
templates\default\forum\showBoard.wm:4.12 -->]
----
if not specifying an end-value.

Thanks for the quick fix!

Sven


Keats Kirsch wrote:
> OK, I've committed a fixed version along with a new unit test.  Please 
> check it out and let me know if you find any problems.
> 
> Keats

Re: [WebMacro-user] Formatting a date

[<Web...@St...>]

Hi!

| My gut feel would be to agree with Endre, ..

Thanks - it's actually nice to hear that someone thinks that not 
everything I say is 100% BS!

| 
| I have every confidence that these plugable tools will not slow me or webmacro 
| down if I don't use them,

.. but who's gonna tell you which tools have which strange side-effects? 
Maybe javadocs, but who reads _all_ classes' javadocs when you begin using 
a new library? Maybe other docs, but again ..
  "Here's a date tool! Excellent! I'll use that!". Whoops, you've got 
yourself a new bunch of potentially contended synchs for every single page 
rendering, and a new _VM-static_ cache.

That's the reason for my whining about "linear code" and "clear pathways" 
and all that stuff.

"Magic caches" (and with them, lots of synchs) don't fit with that 
paradigm.

| but I cannot understand why anyone would use them. 

Well - but when they're there, people will use them, and will end up with 
bad behaviour.

PS: I actually put another "date tool" within the context, that has a set 
of predefined patterns, and is tied to the system's "user object" (since 
he may choose his own Locale). Thus no extra synchs, but still 
user-controlled (user as in me-the-developer) caching.

Regards,
Endre.

Re: [WebMacro-user] Formatting a date

[Marc P. <ma...@an...>]

On 29 Mar 2006, at 00:41, Keats Kirsch wrote:

> Tim Pizey wrote:
>
>> I have every confidence that these plugable tools will not slow me  
>> or webmacro down if I don't use them,
> Exactly.
>
>> but I cannot understand why anyone would use them.
> Personally, I think this fits well with the separation of  
> concerns.  A template designer should be able to choose and modify  
> the display format of a date without going back to the Java coder.

Yes, plus changes to the display requirements that do not affect the  
model in a material way are possible when you include a raw date - i.e.

#if ($date.Time < $someReferenceDate.Time) #set $color = "red" #end

I know what you are saying Tim - and to tackle this in the past I  
spent a lot of time producing a system that would mutate objects in  
the model to a localized string form, and the code always put both  
sets of data into the context, so that the template had everything  
ready for display (according to application settings also), as well  
as the raw data.

It's a PITA but it gives great flexibility.

Formatting values within templates is often wordy and messy. Much  
nicer to have available a pre-formatted version if possible, as well  
as the raw :)

Cheers

Re: [WebMacro-user] Infinite loop with CountDirective

[Keats K. <ke...@xa...>]

OK, I've committed a fixed version along with a new unit test.  Please 
check it out and let me know if you find any problems.

Keats

Keats Kirsch wrote:

> This certainly looks like a bug.  Thanks for doing such an excellent 
> job of documenting it.  I'll try to commit a patch (and maybe a unit 
> test to go with it) in the next day or so.
>
> I'm pretty sure that the count directive was written before we added 
> the UndefinedMacro, and this edge case was apparently not tested.
>
> Keats
>
> Sven Schliesing wrote:
>
>> Hi,
>>
>> I think I just encountered an infinite loop with the CountDirective.
>>
>> I cut the template down to this snippet to verify the error:
>> #count $i from 1 to $pages {
>> }
>>
>> $pages is not set in the context, so we count from "1 to unknown"
>>
>> In the CountDirective this snippet is responsible for the handling of 
>> the end-value:
>>
>> CountDirective.java
>> -------------
>> 162 if ((tmp = _objEnd) != null)
>> 163        {
>> 164            while (tmp instanceof Macro)
>> 164                tmp = ((Macro) tmp).evaluate(context);
>> ...
>> 173        }
>> -------------
>> I added line numbers for better understanding.
>> Cause _objEnd (and therefore tmp) is a SimplePropertyVariable 
>> (implements Macro), the statement on line 164 gets true and so 
>> evaluate(context) (member of Variable) is called.
>>
>> Variable.java
>> -------------
>> 153 final public Object evaluate (Context context) throws 
>> PropertyException
>> 154    {
>> 155        try
>> 156        {
>> 157            Object val = getValue(context);
>> 158            if (val instanceof Macro)
>> 159            {
>> 160                val = ((Macro) val).evaluate(context); // recurse
>> 161            }
>> 162            return val;
>> 163        }
>> ...
>> ------------
>> getValue(context) on line 157 returns "UndefinedMacro", cause the 
>> value ($pages) is not set. Line 158 evaluates to true, and so 
>> evaluate (context) (member of UndefinedMacro) is called and returns 
>> the class' _singleton (new UndefinedMacro()).
>> So we get back to the beginning in CountDirective. tmp in line 164 
>> gets a new value (singleton of UndefinedMacro).
>> Now we enter the infinite loop: Because UndefinedMacro is a Macro, 
>> too, we call evaluate(context) on it again (line 164) and get 
>> returned: the _singleton of UndefinedMacro. So no change here. tmp 
>> will never change it's value from UndefinedMacro to something else. 
>> It will never ever evaluate to a value that exits our loop.
>>
>> So is it me? Am I getting something totally wrong? An unset 
>> end-variable should result in an error-message, not in an infinite 
>> loop, shouldn't it?
>>
>> Greets,
>>
>> Sven
>>

Re: [WebMacro-user] Formatting a date

[Keats K. <ke...@xa...>]

Tim Pizey wrote:

>I have every confidence that these plugable tools will not slow me or webmacro 
>down if I don't use them, 
>  
>
Exactly.

>but I cannot understand why anyone would use them. 
>
Personally, I think this fits well with the separation of concerns.  A 
template designer should be able to choose and modify the display format 
of a date without going back to the Java coder.

Keats

>cheers
>Tim
>  
>

Re: [WebMacro-user] Infinite loop with CountDirective

[Eric B. R. <eb...@tc...>]

On Mar 28, 2006, at 5:26 PM, Keats Kirsch wrote:

> This is what the DefaultEvaluationExceptionHandler does.  Other  
> EEH's do different thing -- like throw an exception that bounces  
> you to the error template for the CrankyEEH.

Right!  I couldn't remember "EvaluationExceptionHandler" off the top  
of my head this morning!

> This could be a bit tricky. Undefined evaluates to undefined.   
> (Expansion, or write, on undefined throws an exception.)

Agreed that it could be tricky.  But what use is  
UndefinedMaco.evaluate()?  Why do we need it to not throw?

> For now we should just take a look at any code that say "while  
> instanceof Macro ...".

That's a good idea, for sure.

eric

Re: [WebMacro-user] Formatting a date

[Tim P. <ti...@pa...>]

On Monday 27 March 2006 20:54, Endre St=F8lsvik wrote:
> On Mon, 27 Mar 2006, Alex Twisleton-Wykeham-Fiennes wrote:
> | On Mon 27 March 2006 11:28, Marc Palmer wrote:
> | > On 27 Mar 2006, at 11:13, Alex Twisleton-Wykeham-Fiennes wrote:
> | > > On Mon 27 March 2006 11:00, Marc Palmer wrote:
> | > >> On 27 Mar 2006, at 02:47, Sven Schliesing wrote:
> | > >>> That's exactly what I looked for.
> | > >>
> | > >> There's another, potentially cleaner way...
> | > >>
> | > >> ----------------- "mymacros.wmm":
> | > >>
> | > >> #macro formatDate($d)
> | > >>     $Text.formatDate($d, "dd.MM.yyyy, HH:mm")
> | > >> #end
> | > >>
> | > >> ----------------- "yourtemplate.wmt":
> | > >>
> | > >> #include as macro "mymacros.wmm"
> | > >>
> | > >> #formatDate($createdAt)
> | > >
> | > > Just to jump in for a moment - this will create a new
> | > > SimpleDateFormat object
> | > > every single time that it is invoked.  If you know that you are
> | > > going to be
> | > > formatting a lot of different dates with the same format then it is
> | > > considerably more efficient to drop in a dedicated wrapper around
> | > > an instance
> | > > of SimpleDateFormat that gives you the format that you want as
> | > > otherwise each
> | > > time that you invoke it, it will have to parse your "dd.MM.yyyy,
> | > > HH:mm"
> | > > format string, build the internal representation and then format
> | > > the Date
> | > > which is relatively expensive.
> | >
> | > You're absolutely right, but that can be dealt with using a macro
> | > plus a #bean to create a "static" shared instance of the formatter?
> |
> | I'll also look at making $Text.formatDate cache the SimpleTextFormat
> | objects that it creates and run some profiling to see whether or not th=
is
> | is really an advantage...
>
> And where should this cache reside? In which context? Who's gonna clean
> it? When? How? (And how do I reload my webapp in Tomcat?)
>
> Caches on every corner, under every rock, in every creek, and in other
> unknown places, is the way to evil code.
>
> Regards,
> Endre.

My gut feel would be to agree with Endre, but what do i know.=20

Getting back to the OP's requirement.=20
Why are you putting a raw date into the context?
Why does everyone think it would be neat to have model level code in the vi=
ew=20
layer?

We deal with dates in melati by formatting them before they hit the context=
,=20
by which I mean we have getters on our objects which return Strings, taking=
=20
into account locale.=20

I have every confidence that these plugable tools will not slow me or webma=
cro=20
down if I don't use them, but I cannot understand why anyone would use them=
=2E=20

cheers
Tim





=2D-=20
Registered linux user #21337, http://counter.li.org.

Re: [WebMacro-user] Infinite loop with CountDirective

[Keats K. <ke...@xa...>]

Eric B. Ridge wrote:

> On Mar 28, 2006, at 11:09 AM, Marc Palmer wrote:
>
>> Isn't this what gives us the "<!-- Variable XXXX is undefined -->"  
>> type output in the templates? That's much more helpful - and  
>> sometimes a quick fix for null checking in a hacked together  
>> prototype - than it spewing a property exception and failing to  
>> build the page...
>
>
> Yes and no.  The ExceptionHelper (or whatever it's called) generally  
> traps all the PropertyExceptions, looks at the innards for things  
> like UndefinedMacro, null objects, etc, and makes the determination  
> about what to do.  So he's the guy that writes out the <!-- Variable  
> XXX is undefined --> stuff.

This is what the DefaultEvaluationExceptionHandler does.  Other EEH's do 
different thing -- like throw an exception that bounces you to the error 
template for the CrankyEEH.

btw, I've refactored the CountDirective to use the EEH.  Should commit 
soon as I get the unit tests ready.

>
> We'd probably need to tighten up some try/catch blocks to make sure  
> we're passing exceptions into the ExceptionHelper (or whatever it's  
> called!), but that still seems better than having code that can  
> accidently cause an infinite loop.

This could be a bit tricky. Undefined evaluates to undefined.  
(Expansion, or write, on undefined throws an exception.)

For now we should just take a look at any code that say "while 
instanceof Macro ...".

Keats

>
> eric
>

Re: [WebMacro-user] Infinite loop with CountDirective

[Eric B. R. <eb...@tc...>]

On Mar 28, 2006, at 11:09 AM, Marc Palmer wrote:

> Isn't this what gives us the "<!-- Variable XXXX is undefined -->"  
> type output in the templates? That's much more helpful - and  
> sometimes a quick fix for null checking in a hacked together  
> prototype - than it spewing a property exception and failing to  
> build the page...

Yes and no.  The ExceptionHelper (or whatever it's called) generally  
traps all the PropertyExceptions, looks at the innards for things  
like UndefinedMacro, null objects, etc, and makes the determination  
about what to do.  So he's the guy that writes out the <!-- Variable  
XXX is undefined --> stuff.

We'd probably need to tighten up some try/catch blocks to make sure  
we're passing exceptions into the ExceptionHelper (or whatever it's  
called!), but that still seems better than having code that can  
accidently cause an infinite loop.

eric

Re: [WebMacro-user] Infinite loop with CountDirective

[Marc P. <ma...@an...>]

On 28 Mar 2006, at 17:05, Eric B. Ridge wrote:

> Should this be fixed in CountDirective.java or in Variable.java?
>
> I haven't studied the code in forever, but if we could change  
> Variable.evaluate() to throw a PropertyException whenever val is an  
> UndefinedMacro, then it seems like we'd never end up with this sort  
> of bug again.
>
> That, or maybe UndefinedMacro.evaluate() should throw?  What value  
> is there in trying to evaluate() an UndefinedMacro?

Isn't this what gives us the "<!-- Variable XXXX is undefined -->"  
type output in the templates? That's much more helpful - and  
sometimes a quick fix for null checking in a hacked together  
prototype - than it spewing a property exception and failing to build  
the page...

...or am I missing something?

Marc

~ ~ ~
Marc Palmer (ma...@an...)
Consultant/Analyst
AnyWare Ltd.
http://www.anyware.co.uk/

Re: [WebMacro-user] Infinite loop with CountDirective

[Eric B. R. <eb...@tc...>]

Should this be fixed in CountDirective.java or in Variable.java?

I haven't studied the code in forever, but if we could change  
Variable.evaluate() to throw a PropertyException whenever val is an  
UndefinedMacro, then it seems like we'd never end up with this sort  
of bug again.

That, or maybe UndefinedMacro.evaluate() should throw?  What value is  
there in trying to evaluate() an UndefinedMacro?

eric

On Mar 28, 2006, at 9:42 AM, Keats Kirsch wrote:

> This certainly looks like a bug.  Thanks for doing such an  
> excellent job of documenting it.  I'll try to commit a patch (and  
> maybe a unit test to go with it) in the next day or so.
>
> I'm pretty sure that the count directive was written before we  
> added the UndefinedMacro, and this edge case was apparently not  
> tested.
>
> Keats
>
> Sven Schliesing wrote:
>
>> Hi,
>>
>> I think I just encountered an infinite loop with the CountDirective.
>>
>> I cut the template down to this snippet to verify the error:
>> #count $i from 1 to $pages {
>> }
>>
>> $pages is not set in the context, so we count from "1 to unknown"
>>
>> In the CountDirective this snippet is responsible for the handling  
>> of the end-value:
>>
>> CountDirective.java
>> -------------
>> 162 if ((tmp = _objEnd) != null)
>> 163        {
>> 164            while (tmp instanceof Macro)
>> 164                tmp = ((Macro) tmp).evaluate(context);
>> ...
>> 173        }
>> -------------
>> I added line numbers for better understanding.
>> Cause _objEnd (and therefore tmp) is a SimplePropertyVariable  
>> (implements Macro), the statement on line 164 gets true and so  
>> evaluate(context) (member of Variable) is called.
>>
>> Variable.java
>> -------------
>> 153 final public Object evaluate (Context context) throws  
>> PropertyException
>> 154    {
>> 155        try
>> 156        {
>> 157            Object val = getValue(context);
>> 158            if (val instanceof Macro)
>> 159            {
>> 160                val = ((Macro) val).evaluate(context); // recurse
>> 161            }
>> 162            return val;
>> 163        }
>> ...
>> ------------
>> getValue(context) on line 157 returns "UndefinedMacro", cause the  
>> value ($pages) is not set. Line 158 evaluates to true, and so  
>> evaluate (context) (member of UndefinedMacro) is called and  
>> returns the class' _singleton (new UndefinedMacro()).
>> So we get back to the beginning in CountDirective. tmp in line 164  
>> gets a new value (singleton of UndefinedMacro).
>> Now we enter the infinite loop: Because UndefinedMacro is a Macro,  
>> too, we call evaluate(context) on it again (line 164) and get  
>> returned: the _singleton of UndefinedMacro. So no change here. tmp  
>> will never change it's value from UndefinedMacro to something  
>> else. It will never ever evaluate to a value that exits our loop.
>>
>> So is it me? Am I getting something totally wrong? An unset end- 
>> variable should result in an error-message, not in an infinite  
>> loop, shouldn't it?
>>
>> Greets,
>>
>> Sven
>>
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting  
> language
> that extends applications into web and mobile media. Attend the  
> live webcast
> and join the prime developer group breaking into this new coding  
> territory!
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] Infinite loop with CountDirective

[Keats K. <ke...@xa...>]

This certainly looks like a bug.  Thanks for doing such an excellent job 
of documenting it.  I'll try to commit a patch (and maybe a unit test to 
go with it) in the next day or so.

I'm pretty sure that the count directive was written before we added the 
UndefinedMacro, and this edge case was apparently not tested.

Keats

Sven Schliesing wrote:

> Hi,
>
> I think I just encountered an infinite loop with the CountDirective.
>
> I cut the template down to this snippet to verify the error:
> #count $i from 1 to $pages {
> }
>
> $pages is not set in the context, so we count from "1 to unknown"
>
> In the CountDirective this snippet is responsible for the handling of 
> the end-value:
>
> CountDirective.java
> -------------
> 162 if ((tmp = _objEnd) != null)
> 163        {
> 164            while (tmp instanceof Macro)
> 164                tmp = ((Macro) tmp).evaluate(context);
> ...
> 173        }
> -------------
> I added line numbers for better understanding.
> Cause _objEnd (and therefore tmp) is a SimplePropertyVariable 
> (implements Macro), the statement on line 164 gets true and so 
> evaluate(context) (member of Variable) is called.
>
> Variable.java
> -------------
> 153 final public Object evaluate (Context context) throws 
> PropertyException
> 154    {
> 155        try
> 156        {
> 157            Object val = getValue(context);
> 158            if (val instanceof Macro)
> 159            {
> 160                val = ((Macro) val).evaluate(context); // recurse
> 161            }
> 162            return val;
> 163        }
> ...
> ------------
> getValue(context) on line 157 returns "UndefinedMacro", cause the 
> value ($pages) is not set. Line 158 evaluates to true, and so evaluate 
> (context) (member of UndefinedMacro) is called and returns the class' 
> _singleton (new UndefinedMacro()).
> So we get back to the beginning in CountDirective. tmp in line 164 
> gets a new value (singleton of UndefinedMacro).
> Now we enter the infinite loop: Because UndefinedMacro is a Macro, 
> too, we call evaluate(context) on it again (line 164) and get 
> returned: the _singleton of UndefinedMacro. So no change here. tmp 
> will never change it's value from UndefinedMacro to something else. It 
> will never ever evaluate to a value that exits our loop.
>
> So is it me? Am I getting something totally wrong? An unset 
> end-variable should result in an error-message, not in an infinite 
> loop, shouldn't it?
>
> Greets,
>
> Sven
>

[WebMacro-user] Infinite loop with CountDirective

[Sven S. <sv...@sc...>]

Hi,

I think I just encountered an infinite loop with the CountDirective.

I cut the template down to this snippet to verify the error:
#count $i from 1 to $pages {
}

$pages is not set in the context, so we count from "1 to unknown"

In the CountDirective this snippet is responsible for the handling of 
the end-value:

CountDirective.java
-------------
162 if ((tmp = _objEnd) != null)
163        {
164            while (tmp instanceof Macro)
164                tmp = ((Macro) tmp).evaluate(context);
...
173        }
-------------
I added line numbers for better understanding.
Cause _objEnd (and therefore tmp) is a SimplePropertyVariable 
(implements Macro), the statement on line 164 gets true and so 
evaluate(context) (member of Variable) is called.

Variable.java
-------------
153 final public Object evaluate (Context context) throws PropertyException
154    {
155        try
156        {
157            Object val = getValue(context);
158            if (val instanceof Macro)
159            {
160                val = ((Macro) val).evaluate(context); // recurse
161            }
162            return val;
163        }
...
------------
getValue(context) on line 157 returns "UndefinedMacro", cause the value 
($pages) is not set. Line 158 evaluates to true, and so evaluate 
(context) (member of UndefinedMacro) is called and returns the class' 
_singleton (new UndefinedMacro()).
So we get back to the beginning in CountDirective. tmp in line 164 gets 
a new value (singleton of UndefinedMacro).
Now we enter the infinite loop: Because UndefinedMacro is a Macro, too, 
we call evaluate(context) on it again (line 164) and get returned: the 
_singleton of UndefinedMacro. So no change here. tmp will never change 
it's value from UndefinedMacro to something else. It will never ever 
evaluate to a value that exits our loop.

So is it me? Am I getting something totally wrong? An unset end-variable 
should result in an error-message, not in an infinite loop, shouldn't it?

Greets,

Sven

Re: [WebMacro-user] Formatting a date

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Tue 28 March 2006 10:06, Endre St=F8lsvik wrote:
> | > How? (And how do I reload my webapp in Tomcat?)
> |
> | Same answer for both question:  same as you always did.
>
> It doesn't really work properly, due to the combination of third party
> libraries' use of ThreadLocals (incl. WM if I remember my testing
> correct), Tomcat's "reuse" of its Thread-pool, and this bug:
>   http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=3D6254531
>
> Have you tried to reload webapps with Tomcat? It "works", but you'll leak
> memory: The ClassLoader isn't reaped, and all your caches' memory that
> can't be cleaned with some WM.shutdown() will _never_ be reclaimed.

not in the current CVS build.  My Class objects are getting cleanly unloade=
d=20
during reloading. Fixed that one with some of my recent patches.  If your=20
classes are not getting reloaded then it is not WebMacro's fault as far as =
I=20
can find out...

<snip>

> Re: the "lets not have discussions about proposed solutions" argument: I=
=20
> won't be using WebMacro much more from June on, I believe. Then that'll be
> good riddance for WebMacro! (.. and btw, I have actually tried to give
> some small pieces of code to WM, but it apparently wasn't accepted..)

coolio.  I'll not bother getting into a religious war with you about writin=
g=20
thread safe code and will get on with my job.  Enjoy whichever community yo=
u=20
move onto...

Alex

Re: [WebMacro-user] Formatting a date

[<Web...@St...>]

| 
| > How? (And how do I reload my webapp in Tomcat?)  
| 
| Same answer for both question:  same as you always did.

It doesn't really work properly, due to the combination of third party 
libraries' use of ThreadLocals (incl. WM if I remember my testing 
correct), Tomcat's "reuse" of its Thread-pool, and this bug:
  http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6254531 

Have you tried to reload webapps with Tomcat? It "works", but you'll leak 
memory: The ClassLoader isn't reaped, and all your caches' memory that 
can't be cleaned with some WM.shutdown() will _never_ be reclaimed.

Re: your other post:

|
| The last entry is doing exactly the same cached lookup as the second 
| entry on the same cache with a synchronized lock around the HashMap.  I 
| throw this in purely to illustrate that the performance overload of 
| synchronization is not going cripple your code.

With how many threads did you test this? 1000? 100? 10? 1? And how 
"real-life" was the scenario: how many date-formattings will one 
typically do within one rendering?

This illustrates the point about synchs that I'm trying to make: synchs 
are pretty much not noticeable at all if you access them with one thread 
(but there was a difference, and it was in the bad direction, right?). 
However, _contended_ synchs are the bad stuff. A conteded synch forces a 
complete flush of your memory-caches (read the java spec on synch if you 
haven't yet). On "small systems", this isn't that big a deal, as they tend 
to a) have one CPU (thus really only one cache), and b) have coherent 
caches if they are more. On larger systems, where CPUs might be more 
independent ("NUMA"), this can be a huge overhead.

The lookup of a HashMap is very fast (given that the hashCode is good), 
and won't make for a huge block for other threads - mostly, it will be in 
and out before it is pre-empted, and thus won't block at all (but it will 
still force a mem-synch). But it might still happen (pre-empted within a 
synch), and even _one_ context-swith due to a contended sync is bad. With 
a huge bunch of threads coming in to the different synch-blocks in your 
code, things might start to stack up: if one thread comes to a 
synch-point, not being able to go directly in, it will enter a queue. At 
that point, you'll have started a line - new threads coming in will enter 
it, and the CPU will have to context-switch to get things moving.

"One more synch" is never the thing that kills your app. But _lots_ of 
small, unnecessary synchs througout your code-path will give you a bunch 
of threads hanging on the different synchs throughout your code, and might 
then force _lots_ of context-switches.

So: If your "worker thread" _could have_ been finished within one "slot" 
of CPU time, but due to some synch have to wait one or more Context 
switches, this will start to give you noticeable performance hits. And 
when this starts to hit, you will have more and more of it, potentially 
leading to "thread thrashing".

  (Synchs are thus evil even without mentioning the different caches 
laying around, eating small amounts of memory without the user of the 
library neither being able to understand what's happening w/o firing up 
his profiler/heap-dumper, nor being able to control it when he finally 
figures out that the innocent-looking little $Text tool both synchs, and 
makes a cache of his date formatting invocations.)

Your test, which was good, reveals the following, as far as I can read:

1 "created formatting" takes 0.04026 milliseconds

1 "cached formatting" takes 0.01279 milliseconds

Which is 3.14 (hah!) times faster. (That low number really surprised me!)

(.. but maybe one ought to do the entire thing 10 times (loop it), so that 
one is _sure_ that the JIT have done all its deeds, and that the first 
test won't get the JIT impact)

But really, it was bleeding fast as it was. For this 3.14 fold "increase 
in speed" for something that will happen _not that often_, you're willing 
to create a _synched permanent cache_ in the code, even with all the 
potential pitfalls and crumbling of the code-pathway that this results in.

.. and that's before you've tested with a bunch of threads instead of one.


Re: the "lets not have discussions about proposed solutions" argument: I 
won't be using WebMacro much more from June on, I believe. Then that'll be 
good riddance for WebMacro! (.. and btw, I have actually tried to give 
some small pieces of code to WM, but it apparently wasn't accepted..)

Regards,
Endre.

Re: [WebMacro-user] Formatting a date

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Mon 27 March 2006 19:12, Keats Kirsch wrote:
> Alex Twisleton-Wykeham-Fiennes wrote:
> >Why not make it so that the wrapper class extends Date so that you can
> > then also used the wrapped class as a drop-in replacement for Date in
> > other methods.
> >
> >Also, why not make the wrapper include a hashtable accessor that enables
> > you to extend the number of registered formats on it?
>
> Exactly what I was thinking.  Don't know when I'll find the time, but it
> should be a fun little project.

the attached should be roughly what you are looking for.  I'm not sure what 
the correct way of wrapping it up should be to drop it into a context though 
as I have my own framework of utilities and I don't use the conventional 
context tools that much.

Alex

Re: [WebMacro-user] Formatting a date

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Mon 27 March 2006 21:51, Keats Kirsch wrote:
> Endre St=F8lsvik wrote:
> >| I'll also look at making $Text.formatDate cache the SimpleTextFormat
> >| objects that it creates and run some profiling to see whether or not
> >| this is really an advantage...
> >
> >And where should this cache reside? In which context? Who's gonna clean
> >it? When? How? (And how do I reload my webapp in Tomcat?)
> >
> >Caches on every corner, under every rock, in every creek, and in other
> >unknown places, is the way to evil code.
>
> I think your lyrical rejoinder is a bit overstretched.  Sometimes
> caching is good, sometimes it isn't.  In this case we are talking about
> caching a small number or small items, which may be a performance win.
>
> Notice that Alex said he would do some testing to see if this is
> worthwhile.  It may not be.  We've been reducing the amount of explicit
> caching in WM as JVMs have become much better at doing this for us.
> It's difficult to predict when you will see a payoff without testing.

attached is very crude test case that compares SimpleDateFormat creation=20
versus SimpleDateFormat caching using the example formats on the=20
SimpleDateFormat javadoc.  Output is as follows:-

creating: 4027
caching: 1279
synchronized: 1287

So we have a rough 3 or 4 x speed up when caching the value across 100000=20
iterations.

The last entry is doing exactly the same cached lookup as the second entry =
on=20
the same cache with a synchronized lock around the HashMap.  I throw this i=
n=20
purely to illustrate that the performance overload of synchronization is no=
t=20
going cripple your code.

Alex

> In this case the cache could just be a static map inside of the factory
> class.  This optional overhead would be minimal and not interfere with
> Servlet reloading in any way.
>
> Keats
>
> >Regards,
> >Endre.
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting langua=
ge
> that extends applications into web and mobile media. Attend the live
> webcast and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D110944&bid=3D241720&dat=
=3D121642
> _______________________________________________
> Webmacro-user mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webmacro-user

Re: [WebMacro-user] Formatting a date

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Mon 27 March 2006 20:54, Endre St=F8lsvik wrote:
> On Mon, 27 Mar 2006, Alex Twisleton-Wykeham-Fiennes wrote:
> | On Mon 27 March 2006 11:28, Marc Palmer wrote:
> | > On 27 Mar 2006, at 11:13, Alex Twisleton-Wykeham-Fiennes wrote:
> | > > On Mon 27 March 2006 11:00, Marc Palmer wrote:
> | > >> On 27 Mar 2006, at 02:47, Sven Schliesing wrote:
> | > >>> That's exactly what I looked for.
> | > >>
> | > >> There's another, potentially cleaner way...
> | > >>
> | > >> ----------------- "mymacros.wmm":
> | > >>
> | > >> #macro formatDate($d)
> | > >>     $Text.formatDate($d, "dd.MM.yyyy, HH:mm")
> | > >> #end
> | > >>
> | > >> ----------------- "yourtemplate.wmt":
> | > >>
> | > >> #include as macro "mymacros.wmm"
> | > >>
> | > >> #formatDate($createdAt)
> | > >
> | > > Just to jump in for a moment - this will create a new
> | > > SimpleDateFormat object
> | > > every single time that it is invoked.  If you know that you are
> | > > going to be
> | > > formatting a lot of different dates with the same format then it is
> | > > considerably more efficient to drop in a dedicated wrapper around
> | > > an instance
> | > > of SimpleDateFormat that gives you the format that you want as
> | > > otherwise each
> | > > time that you invoke it, it will have to parse your "dd.MM.yyyy,
> | > > HH:mm"
> | > > format string, build the internal representation and then format
> | > > the Date
> | > > which is relatively expensive.
> | >
> | > You're absolutely right, but that can be dealt with using a macro
> | > plus a #bean to create a "static" shared instance of the formatter?
> |
> | I'll also look at making $Text.formatDate cache the SimpleTextFormat
> | objects that it creates and run some profiling to see whether or not th=
is
> | is really an advantage...
>
> And where should this cache reside?=20

Within the same class as the accessor.

> In which context?=20

The ClassLoader that loaded the accessor.

> Who's gonna clean =20
> it?=20

No-one.  It will cache at most the same number of unique SimpleDateFormats=
=20
that you create during the lifespan of your servlet.  This is assuming that=
=20
you never use a SimpleDateFormat more than once.   Any other case will almo=
st=20
definitely be considerably better (pending testing).

> When?=20

When the ClassLoader ditches the class definition and therefore by definiti=
on=20
the static variables.

> How? (And how do I reload my webapp in Tomcat?) =20

Same answer for both question:  same as you always did.

> Caches on every corner, under every rock, in every creek, and in other
> unknown places, is the way to evil code.

That's a fair enough assumption to make in certain contexts.  However, I'd =
be=20
really happy if you were to back up your viewpoint with some reasoned=20
arguments rather than your somewhat overzealous lashing out that seems to=20
accompany anyone's postings to the list.  This is supposed to be a communit=
y,=20
and people are doing things for nothing so if you are (somewhat) aggressive=
ly=20
negative about almost everything that people do (and I'm not just referring=
=20
to my stuff, I'm a *long* time lurker), then either people will stop=20
contributing or they will stop listening to what you have to say which woul=
d=20
be a shame because I'm sure that you have lots of constructive things to=20
contribute.

One of the reasons that I stuck around with the webmacro project for as lon=
g=20
as I have (not that I regret it for a minute) is that during the discussion=
s=20
with the apache project which led into the forking of webmacro and the=20
creation of Velocity, I felt that the webmacro community was the more order=
ed=20
and calm side of the argument (regardless of who was "right").  Lets try an=
d=20
keep it that way?

Alex

Re: [WebMacro-user] Formatting a date

[Keats K. <ke...@xa...>]

Endre Stølsvik wrote:

>| I'll also look at making $Text.formatDate cache the SimpleTextFormat objects 
>| that it creates and run some profiling to see whether or not this is really 
>| an advantage...
>
>And where should this cache reside? In which context? Who's gonna clean 
>it? When? How? (And how do I reload my webapp in Tomcat?)
>
>Caches on every corner, under every rock, in every creek, and in other 
>unknown places, is the way to evil code.
>  
>
I think your lyrical rejoinder is a bit overstretched.  Sometimes 
caching is good, sometimes it isn't.  In this case we are talking about 
caching a small number or small items, which may be a performance win.  

Notice that Alex said he would do some testing to see if this is 
worthwhile.  It may not be.  We've been reducing the amount of explicit 
caching in WM as JVMs have become much better at doing this for us.  
It's difficult to predict when you will see a payoff without testing.

In this case the cache could just be a static map inside of the factory 
class.  This optional overhead would be minimal and not interfere with 
Servlet reloading in any way.

Keats

>Regards,
>Endre.
>  
>

Re: [WebMacro-user] Formatting a date

[<Web...@St...>]

On Mon, 27 Mar 2006, Alex Twisleton-Wykeham-Fiennes wrote:

| On Mon 27 March 2006 11:28, Marc Palmer wrote:
| > On 27 Mar 2006, at 11:13, Alex Twisleton-Wykeham-Fiennes wrote:
| > > On Mon 27 March 2006 11:00, Marc Palmer wrote:
| > >> On 27 Mar 2006, at 02:47, Sven Schliesing wrote:
| > >>> That's exactly what I looked for.
| > >>
| > >> There's another, potentially cleaner way...
| > >>
| > >> ----------------- "mymacros.wmm":
| > >>
| > >> #macro formatDate($d)
| > >>     $Text.formatDate($d, "dd.MM.yyyy, HH:mm")
| > >> #end
| > >>
| > >> ----------------- "yourtemplate.wmt":
| > >>
| > >> #include as macro "mymacros.wmm"
| > >>
| > >> #formatDate($createdAt)
| > >
| > > Just to jump in for a moment - this will create a new
| > > SimpleDateFormat object
| > > every single time that it is invoked.  If you know that you are
| > > going to be
| > > formatting a lot of different dates with the same format then it is
| > > considerably more efficient to drop in a dedicated wrapper around
| > > an instance
| > > of SimpleDateFormat that gives you the format that you want as
| > > otherwise each
| > > time that you invoke it, it will have to parse your "dd.MM.yyyy,
| > > HH:mm"
| > > format string, build the internal representation and then format
| > > the Date
| > > which is relatively expensive.
| >
| > You're absolutely right, but that can be dealt with using a macro
| > plus a #bean to create a "static" shared instance of the formatter?
| 
| I'll also look at making $Text.formatDate cache the SimpleTextFormat objects 
| that it creates and run some profiling to see whether or not this is really 
| an advantage...

And where should this cache reside? In which context? Who's gonna clean 
it? When? How? (And how do I reload my webapp in Tomcat?)

Caches on every corner, under every rock, in every creek, and in other 
unknown places, is the way to evil code.

Regards,
Endre.

Re: [WebMacro-user] Formatting a date

[Keats K. <ke...@xa...>]

Alex Twisleton-Wykeham-Fiennes wrote:

>Why not make it so that the wrapper class extends Date so that you can then 
>also used the wrapped class as a drop-in replacement for Date in other 
>methods.
>
>Also, why not make the wrapper include a hashtable accessor that enables you 
>to extend the number of registered formats on it?
>  
>
Exactly what I was thinking.  Don't know when I'll find the time, but it 
should be a fun little project.

Keats

Re: [WebMacro-user] Formatting a date

[Alex Twisleton-Wykeham-F. <al...@fi...>]

On Mon 27 March 2006 17:36, Keats Kirsch wrote:
> >I'll also look at making $Text.formatDate cache the SimpleTextFormat
> > objects that it creates and run some profiling to see whether or not this
> > is really an advantage...
>
> Yes the formatDate() method was kind of a quick hack.  At one point I
> had written a DateTool, but I never got it clean enough to commit to CVS
> (and now I've lost it).  Perhaps I'll give it another whack.  It's
> actually fairly complicated, given all the different date formats and
> localization etc.  It would be nice if the tool could pick up the locale
> from the request, but then it would be Servlet specific.  But something
> simple would probably be useful for parsing and formatting dates.

In the meantime, adding something like this to TextTool will help:-

private final static Map __simpleDateFormats = new HashMap();
public static String formatDate (Date date, String format)
{
  SimpleDateFormat formatter = null;
  synchronized (__simpleDateFormats) {
    formatter = (SimpleDateFormat) __simpleDateFormats.get(format);
    if (formatter == null) {
      formatter = new SimpleDateFormat(format);
      __formatters.put(format, formatter);
    }
  }
  return formatter.format(date);
}

I'll run some tests and see if it is worth the added complexity from a 
performance viewpoint.

> I'm thinking a factory that would produce date wrappers.  E.g.,
>
> #set $dt = $DateFactory.parse("27-Mar-2006", "dd-MMM-yyyy")
> #set $dt.Format.Short = "MM/dd/yyyy"
> #set $dt.Format.Default = "dd-MMM-yyyy"
>
> Default format: $dt                      ==> 27-Mar-2006
> Short format: $dt.Format.Short   ==> 02/27/2006
>
> #set $dt = $dt.addMonths(1)
> New date: $dt    ==> 27-Apr-2006

Why not make it so that the wrapper class extends Date so that you can then 
also used the wrapped class as a drop-in replacement for Date in other 
methods.

Also, why not make the wrapper include a hashtable accessor that enables you 
to extend the number of registered formats on it?

e.g.

$DateFactory.registerFormat("Short", "MM-dd-yyyy")
#set $dateWrapper = $DateFactory.parse("27-Mar-2006", "Short")
$dateWrapper.Time --> java.util.Date.getTime()
$dateWrapper.Short --> DateWrapper.get("Short") --> String (by querying 
DateFactory for the SimpleDateFormatter registered as "Short" and then 
formatting itself using this format)

If I was really looking for flexibility you could also make cascading parsers 
which would let you do things like this:-

$DateFactory.registerFormat("Long", "hh:MM:ss MM-dd-yyyy")
$DateFactory.registerFormat("Medium", "hh:MM MM-dd-yyyy")
$DateFactory.registerFormat("Short", "MM-dd-yyyy")
#set $dateWrapper = $DateFactory.parse("22-05-1971")

and it would iterate through the registered formatters in order finding one 
that can parse the supplied text...

<snip>

Alex