Re: [Pagekit-devel] Character conversions

[Gabriel B. <gbu...@eb...>]

On Tue, 10 Dec 2002, Boris Zentner wrote:

> Hi Gabriel,
> 
> this is hopfully already fixed. With this patch, that works only for perl 5.8. 
> Perl 5.6.1 should be fine already. I hope the patch applied clean. please let 
> me know.

Actually, that made everything 2x as bad. Before the patch, strings
inside form fields and <CONTENT_VAR > were OK, and strings inside
<MODEL_VAR > had 2 characters where 1 should be. Now form fields and
<CONTENT_VAR > have 2 characters instead of 1, and <MODEL_VAR > has 4
characters where 1 should be (and where there were 2 characters before
the patch).

I'll un-patch for now and take a look at the patch tonight to see where
the conversions are done.

> The reason for your double conversion is that perl now converts some strings 
> to utf8 but Text::Iconv did not notice that and convert it again. So we 
> remove T::I for perl 5.8.
> 
> Maybe we should have a patchpage where all this fixes have a little 
> description and a tarball?
> 
> Am Dienstag, 10. Dezember 2002 21:23 schrieb Gabriel Burca:
> > PageKit v1.10 (checked out of CVS on Nov 29th).
> > Perl 5.8
> > HTML::Template 2.6 (see below)
> > HTML::FillInForm 1.0 (see below)
> >
> >
> > In the model code I have:
> > 	$model->fillinform(key => $utf8_string);
> > 	$model->output(key => $utf8_string);
> >
> > In the template I have:
> > 	<input type="text" name="key">
> > 	<MODEL_VAR NAME="key">
> > 	<CONTENT_VAR NAME="some_utf-8">
> >
> > Config.xml
> > 	default_input_charset = "UTF-8"
> > 	default_output_charset = "UTF-8"
> >
> > The form field and <CONTENT_VAR ...> contains the correct unicode
> > characters. If I try to display the same characters using <MODEL_VAR
> > ...> on the same page, it looks like the characters get converted to
> > unicode twice (a single 2-byte unicode character shows up as two
> > characters).
> >
> > I've patched both HTML::FillInForm and HTML::Template and replaced .=
> > with sprintf "%s%s". That was before I realized the problem only occurs
> > when using <MODEL_VAR ...>. The View.pm _preparse_model_tags sub has the
> > "use bytes;" fix.
> >
> > This might be the same problem reported by Leo Cambilargiu in the "Odd
> > Characters" thread. Did anyone figure out that problem? Any ideas of
> > where the double-conversion might be happening?