HTML::Template

On 3/19/08, Alex Teslik <alex@...> wrote:

> But then I can no longer do all my escaping in the template.

Well, you're effectively handing it a half-escaped string. That's kind
of a special case.

>  So I propose that the HTML::Template escaping does not squash HTML entities.

That would break its behavior for me - I'm handing it unescaped
strings, which (being somewhat self-referential discussions of HTML
and CSS) very often contain examples of HTML entities, which H::T
properly escapes so they are displayed the way the author expects them
to be. It's not terribly helpful if, for example, someone says "To
display an ampersand on the page, use '&amp;'" and the forum displays
"use '&'".

An alternate escaping method would be fine, but changing the default
behavior to kinda-sorta escape things, but not always, would IMHO be a
Really Bad Thing.

Have you tried using utf8 as the encoding?  You will get far more 
millage out of utf8 than using entities.

regards,
Mathew

Alex Teslik wrote:
> Hello,
>
>   I'm developing an app where there are some strings that have HTML entities
> in them, such as:
>
> This is a "Tést"
>
> These strings need to go into a template that is displayed in the iso-8859-1
> character set. These strings need to be html escaped so that the quotes don't
> break things. When I put it in a template:
>
> <tmpl_var name="sentence" escape="html">
>
> I get:
>
> This is a &quot;&amp;#233;st&quot;
>
> Doh. The entity got squashed.
>
> So the easy way out is "just decode the entity before dropping it in the
> template"... but if the decoded entity cannot be displayed in iso-8859-1 when
> it is decoded, as is the case with a Chinese character like &#25105;, than I'm
> out of luck and must pass the entity through to the template. But then I can
> no longer do all my escaping in the template.
>
> So I propose that the HTML::Template escaping does not squash HTML entities.
> Something like (quick off the cuff):
>
> sub entity_safe_escape {
>    my $t=shift;
>
>    $t=~s/&#(\d\d\d+);/ESCAPE_UNICODE_$1/g;
>    $t=~s/&/ESCAPE_AMP/g;
>    
>    $t=~s/"/&quot;/g;
>    $t=~s/</&lt;/g;
>    $t=~s/>/&gt;/g;
>    
>    $t=~s/ESCAPE_AMP/&amp;/g;
>    $t=~s/ESCAPE_UNICODE_(\d\d\d+)/&#$1;/g;
>    
>    return($t);
> }
>
> How do others typically get around this problem?
>
> Thanks,
> Alex
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Html-template-users mailing list
> Html-template-users@...
> https://lists.sourceforge.net/lists/listinfo/html-template-users
>

I completely agree. Unfortunately I've inherited a lot of the code and it
needs to support all charsets. :(

On Thu, 20 Mar 2008 14:31:54 +1100, Mathew Robertson wrote
> Have you tried using utf8 as the encoding?  You will get far more 
> millage out of utf8 than using entities.
> 
> regards,
> Mathew
> 
> Alex Teslik wrote:
> > Hello,
> >
> >   I'm developing an app where there are some strings that have HTML entities
> > in them, such as:
> >
> > This is a "Tést"
> >
> > These strings need to go into a template that is displayed in the iso-8859-1
> > character set. These strings need to be html escaped so that the quotes don't
> > break things. When I put it in a template:
> >
> > <tmpl_var name="sentence" escape="html">
> >
> > I get:
> >
> > This is a &quot;&amp;#233;st&quot;
> >
> > Doh. The entity got squashed.
> >
> > So the easy way out is "just decode the entity before dropping it in the
> > template"... but if the decoded entity cannot be displayed in iso-8859-1 when
> > it is decoded, as is the case with a Chinese character like &#25105;, than I'm
> > out of luck and must pass the entity through to the template. But then I can
> > no longer do all my escaping in the template.
> >
> > So I propose that the HTML::Template escaping does not squash HTML entities.
> > Something like (quick off the cuff):
> >
> > sub entity_safe_escape {
> >    my $t=shift;
> >
> >    $t=~s/&#(\d\d\d+);/ESCAPE_UNICODE_$1/g;
> >    $t=~s/&/ESCAPE_AMP/g;
> >    
> >    $t=~s/"/&quot;/g;
> >    $t=~s/</&lt;/g;
> >    $t=~s/>/&gt;/g;
> >    
> >    $t=~s/ESCAPE_AMP/&amp;/g;
> >    $t=~s/ESCAPE_UNICODE_(\d\d\d+)/&#$1;/g;
> >    
> >    return($t);
> > }
> >
> > How do others typically get around this problem?
> >
> > Thanks,
> > Alex
> >
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Microsoft
> > Defy all challenges. Microsoft(R) Visual Studio 2008.
> > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> > _______________________________________________
> > Html-template-users mailing list
> > Html-template-users@...
> > https://lists.sourceforge.net/lists/listinfo/html-template-users
> >

On Wed, 19 Mar 2008 22:24:36 -0500, Karen wrote
> On 3/19/08, Alex Teslik <alex@...> wrote:
> 
> > But then I can no longer do all my escaping in the template.
> 
> Well, you're effectively handing it a half-escaped string. That's 
> kind of a special case.

I disagree. The string I'm giving that contains entities is completely valid
and not half-escaped. An entity is not always an escaped string - aside from
the 4 common entities &amp, &lt, &gt, and &quot, any other entity can be
decoded and co-exist with no problem among the html code. &amp, &lt, &gt, and
&quot are escapes because they prevent specific characters from breaking the
html code - and happen to perform the escaping by converting those characters
to entities.

> 
> >  So I propose that the HTML::Template escaping does not squash HTML entities.
> 
> That would break its behavior for me

The code I submitted would not break your example because it does not change
the behavior regarding legitimate &, <, >, or " characters. It simply changes
the behavior to not _also_ escape other legitimate entities that also happen
to contain legitimate ampersands.

> An alternate escaping method would be fine

An alternate option is probably a good idea.

Thanks,
Alex

Alex Teslik wrote:

> So I propose that the HTML::Template escaping does not squash HTML entities.
> Something like (quick off the cuff)

This seems like a very specific case to me. You are having a problem because
your text contains some HTML encoded things, but not all and because of the
limitations of charsets that your project has. These aren't very generic and is
not a problem that H::T should solve.

You know that your data won't fit the standard, so just do the escaping you
propose on your own.

> How do others typically get around this problem?

We either don't have mixed strings like you do, or we've moved on to UTF8 :)

-- 
Michael Peters
Plus Three, LP

You will be able to support more characters descriptors by using only 
utf8 (ie: unicode) than trying to support individual charsets ** ie: if 
you are resorting to entities, then you should simply use utf8 as they 
both print the same character.

** I can say this from experience, eg: I had a requirement to support at 
least three different charsets on the same web page, at the same time -> 
using entites was hard work on the programmers brain and is bandwidth 
intensive.

You should seriously consider outputting to utf8 only.

cheers,
Mathew

Alex Teslik wrote:
> I completely agree. Unfortunately I've inherited a lot of the code and it
> needs to support all charsets. :(
>
> On Thu, 20 Mar 2008 14:31:54 +1100, Mathew Robertson wrote
>   
>> Have you tried using utf8 as the encoding?  You will get far more 
>> millage out of utf8 than using entities.
>>
>> regards,
>> Mathew
>>
>> Alex Teslik wrote:
>>     
>>> Hello,
>>>
>>>   I'm developing an app where there are some strings that have HTML entities
>>> in them, such as:
>>>
>>> This is a "Tést"
>>>
>>>       
>