Re: [Cgi-session-user] utf8 problem

[Thomas G. <tho...@pi...>]

Hi Ron,

Thank you for the quick response.

You are right, turning the session data column into binary blob does work.
But in my present scenario the db column is preferred to be human readable.
Also, saving the session data in a binary column without a collation 
rather avoids the whole encoding issue right?

If I write the session data to a file with Path::Tiny::spew_utf8 the 
string is stored as expected 'jörg'.
Aside from the session table, the name is also stored in a 'person' db 
table but without using Data::Dumper of course, and reading/writing from 
there works fine with utf-8 characters.

I made some tests using Devel::Peek and before it gets stored to db via 
Data::Dumper, the session parameter 'jörg' has the utf8 flag set 
alright, after thawing it doesn't anymore.
So, I tried calling utf8::upgrade manually on each session parameter 
right after thawing and this would work indeed, but in general there 
might be various nested structures in the session which makes that 
workaround inconvenient.

Interestingly, using YAML as a serializer yields similar results. On 
thaw the session parameters are not decoded properly into perls internal 
format. Manually calling decode_utf8 for each session parameter right 
after thaw would fix it, but like above it's not convenient.

So I'm wondering if I'm just missing a parameter which could tell 
CGI::Session to expect utf8 data and deserialize it accordingly, thereby 
not making assumptions about it. Or is there indeed no other way than to 
manually iterate over the whole session data structure?

Best regards,
Thomas
> Ron Savage <mailto:ro...@sa...>
> Freitag, 10. Juli 2015 00:42
> Hi Thomas
>
> There's another issue, which I omitted from my 1st email on the
> assumption it was obvious, but perhaps it isn't.
>
> (Ideally) You should not be writing or using code which makes
> assumptions about the 'utf8 flag'.
>
> So, try opening a file with the utf-8 layer active, and just write the
> session data to the file, without using Data::Dumper.
>
> Use https://metacpan.org/release/Path-Tiny, say, for that.
>
>
> Thomas Glase <mailto:tho...@pi...>
> Donnerstag, 9. Juli 2015 14:36
> Hi,
>
> I'm having problems getting CGI::Session to work with utf8.
> I read the "A Warning about UTF8" section but I couldn't find the
> following issue anywhere:
>
> In my session I have utf8 encoded strings like 'jörg'.
>
> The session data gets serialized by default serializer Data::Dumper into
> a MySQL data column (mediumtext, charset utf8, collation
> utf8_unicode_ci) as expected with utf8 characters escaped like this:
> 'j\x{f6}rg'
>
> Now, the problem is that when this session data get's thawed, the
> resulting string 'jörg' is missing perls utf8 flag. Which causes
> Data::Dumper to just cut the data at the utf8 character 'ö' during next
> freeze. Result: the dump is stored corrupted in the data column and
> Data::Dumper crashes of course when trying to thaw that next time.
>
> I already posted this problem incl more details and code snippets at
> stackoverflow to no avail:
>
> http://stackoverflow.com/questions/31049125/perl-cgisession-mysql-thaw-utf-8-data 
>
>
>
> It looks like a bug to me but since I don't think that I'm the only one
> trying to use CGI::Session with utf8 characters I guess that I rather
> must be missing something here.
>
> I'd be really grateful for any hints on this.
>
> Best regards,
> Thomas
>
>
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
> _______________________________________________
> Cgi-session-user mailing list
> Cgi...@li...
> https://lists.sourceforge.net/lists/listinfo/cgi-session-user
> Thomas Glase <mailto:tho...@pi...>
> Donnerstag, 9. Juli 2015 13:29
> Hi,
>
> I'm having problems getting CGI::Session to work with utf8.
> I read the "A Warning about UTF8" section but I couldn't find the 
> following issue anywhere:
>
> In my session I have utf8 encoded strings like 'jörg'.
>
> The session data gets serialized by default serializer Data::Dumper 
> into a MySQL data column (mediumtext, charset utf8, collation 
> utf8_unicode_ci) as expected with utf8 characters escaped like this: 
> 'j\x{f6}rg'
>
> Now, the problem is that when this session data get's thawed, the 
> resulting string 'jörg' is missing perls utf8 flag. Which causes 
> Data::Dumper to just cut the data at the utf8 character 'ö' during 
> next freeze. Result: the dump is stored corrupted in the data column 
> and Data::Dumper crashes of course when trying to thaw that next time.
>
> I already posted this problem incl more details and code snippets at 
> stackoverflow to no avail:
>
> http://stackoverflow.com/questions/31049125/perl-cgisession-mysql-thaw-utf-8-data 
>
>
> It looks like a bug to me but since I don't think that I'm the only 
> one trying to use CGI::Session with utf8 characters I guess that I 
> rather must be missing something here.
>
> I'd be really grateful for any hints on this.
>
> Best regards,
> Thomas
>

-- 

Thomas Glase
Software Developer

Piranha Arts AG * Kreuzbergstr. 30 * 10965 Berlin * Germany
Tel+49 30  318 614-21  * Fax+49 30  318 614-10

www.piranha-arts.com

-----------------------------------------------------------
Mark your calendars ...
:::21 - 25  Oct 2015, WOMEX, Budapest, Hungary
:::25 - 28  May 2016, Classical:NEXT, Rotterdam, the Netherlands
----------------------------------------------------------

This email was sent by: Piranha Arts AG * Berlin HRB 63466
Vorstand F.Klaffs&  M.v.Petrykowski * AR-Vorsitzender C.Borkowsky

Disclaimer: Any statements in this email or attachment explicitly
do not constitute a contract or a legally binding agreement with
Piranha Arts AG.