Re: [Cgi-session-user] utf8 problem

[Ron S. <ro...@sa...>]

Hi Thomas

Thanx for the detailed reply.

I wrote Data::Session as a modernized version of CGI::Session, and in 
its docs:

https://metacpan.org/pod/Data::Session

you can search for 'utf8' to find a little bit of info. The context is 
UUID16, which I guess does not help, but it's a far as I've investigated 
in the past.

The other reference to 'utf8' is in:

https://metacpan.org/pod/Data::Session::ID::UUID16

If I hit this problem, my first idea would be to use SHA1 to encode the 
session. I have no idea if it would help.

HTH.

On 13/07/15 20:40, Thomas Glase wrote:
> Hi Ron,
>
> Thank you for the quick response.
>
> You are right, turning the session data column into binary blob does work.
> But in my present scenario the db column is preferred to be human readable.
> Also, saving the session data in a binary column without a collation
> rather avoids the whole encoding issue right?
>
> If I write the session data to a file with Path::Tiny::spew_utf8 the
> string is stored as expected 'jörg'.
> Aside from the session table, the name is also stored in a 'person' db
> table but without using Data::Dumper of course, and reading/writing from
> there works fine with utf-8 characters.
>
> I made some tests using Devel::Peek and before it gets stored to db via
> Data::Dumper, the session parameter 'jörg' has the utf8 flag set
> alright, after thawing it doesn't anymore.
> So, I tried calling utf8::upgrade manually on each session parameter
> right after thawing and this would work indeed, but in general there
> might be various nested structures in the session which makes that
> workaround inconvenient.
>
> Interestingly, using YAML as a serializer yields similar results. On
> thaw the session parameters are not decoded properly into perls internal
> format. Manually calling decode_utf8 for each session parameter right
> after thaw would fix it, but like above it's not convenient.
>
> So I'm wondering if I'm just missing a parameter which could tell
> CGI::Session to expect utf8 data and deserialize it accordingly, thereby
> not making assumptions about it. Or is there indeed no other way than to
> manually iterate over the whole session data structure?
>
> Best regards,
> Thomas
>> Ron Savage <mailto:ro...@sa...>
>> Freitag, 10. Juli 2015 00:42
>> Hi Thomas
>>
>> There's another issue, which I omitted from my 1st email on the
>> assumption it was obvious, but perhaps it isn't.
>>
>> (Ideally) You should not be writing or using code which makes
>> assumptions about the 'utf8 flag'.
>>
>> So, try opening a file with the utf-8 layer active, and just write the
>> session data to the file, without using Data::Dumper.
>>
>> Use https://metacpan.org/release/Path-Tiny, say, for that.
>>
>>
>> Thomas Glase <mailto:tho...@pi...>
>> Donnerstag, 9. Juli 2015 14:36
>> Hi,
>>
>> I'm having problems getting CGI::Session to work with utf8.
>> I read the "A Warning about UTF8" section but I couldn't find the
>> following issue anywhere:
>>
>> In my session I have utf8 encoded strings like 'jörg'.
>>
>> The session data gets serialized by default serializer Data::Dumper into
>> a MySQL data column (mediumtext, charset utf8, collation
>> utf8_unicode_ci) as expected with utf8 characters escaped like this:
>> 'j\x{f6}rg'
>>
>> Now, the problem is that when this session data get's thawed, the
>> resulting string 'jörg' is missing perls utf8 flag. Which causes
>> Data::Dumper to just cut the data at the utf8 character 'ö' during next
>> freeze. Result: the dump is stored corrupted in the data column and
>> Data::Dumper crashes of course when trying to thaw that next time.
>>
>> I already posted this problem incl more details and code snippets at
>> stackoverflow to no avail:
>>
>> http://stackoverflow.com/questions/31049125/perl-cgisession-mysql-thaw-utf-8-data
>>
>>
>>
>> It looks like a bug to me but since I don't think that I'm the only one
>> trying to use CGI::Session with utf8 characters I guess that I rather
>> must be missing something here.
>>
>> I'd be really grateful for any hints on this.
>>
>> Best regards,
>> Thomas
>>
>>
>> ------------------------------------------------------------------------------
>> Don't Limit Your Business. Reach for the Cloud.
>> GigeNET's Cloud Solutions provide you with the tools and support that
>> you need to offload your IT needs and focus on growing your business.
>> Configured For All Businesses. Start Your Cloud Today.
>> https://www.gigenetcloud.com/
>> _______________________________________________
>> Cgi-session-user mailing list
>> Cgi...@li...
>> https://lists.sourceforge.net/lists/listinfo/cgi-session-user
>> Thomas Glase <mailto:tho...@pi...>
>> Donnerstag, 9. Juli 2015 13:29
>> Hi,
>>
>> I'm having problems getting CGI::Session to work with utf8.
>> I read the "A Warning about UTF8" section but I couldn't find the
>> following issue anywhere:
>>
>> In my session I have utf8 encoded strings like 'jörg'.
>>
>> The session data gets serialized by default serializer Data::Dumper
>> into a MySQL data column (mediumtext, charset utf8, collation
>> utf8_unicode_ci) as expected with utf8 characters escaped like this:
>> 'j\x{f6}rg'
>>
>> Now, the problem is that when this session data get's thawed, the
>> resulting string 'jörg' is missing perls utf8 flag. Which causes
>> Data::Dumper to just cut the data at the utf8 character 'ö' during
>> next freeze. Result: the dump is stored corrupted in the data column
>> and Data::Dumper crashes of course when trying to thaw that next time.
>>
>> I already posted this problem incl more details and code snippets at
>> stackoverflow to no avail:
>>
>> http://stackoverflow.com/questions/31049125/perl-cgisession-mysql-thaw-utf-8-data
>>
>>
>> It looks like a bug to me but since I don't think that I'm the only
>> one trying to use CGI::Session with utf8 characters I guess that I
>> rather must be missing something here.
>>
>> I'd be really grateful for any hints on this.
>>
>> Best regards,
>> Thomas
>>
>
> --
>
> Thomas Glase
> Software Developer
>
> Piranha Arts AG * Kreuzbergstr. 30 * 10965 Berlin * Germany
> Tel+49 30  318 614-21  * Fax+49 30  318 614-10
>
> www.piranha-arts.com
>
> -----------------------------------------------------------
> Mark your calendars ...
> :::21 - 25  Oct 2015, WOMEX, Budapest, Hungary
> :::25 - 28  May 2016, Classical:NEXT, Rotterdam, the Netherlands
> ----------------------------------------------------------
>
> This email was sent by: Piranha Arts AG * Berlin HRB 63466
> Vorstand F.Klaffs & M.v.Petrykowski * AR-Vorsitzender C.Borkowsky
>
> Disclaimer: Any statements in this email or attachment explicitly
> do not constitute a contract or a legally binding agreement with
> Piranha Arts AG.
>
>
>
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
>
>
>
> _______________________________________________
> Cgi-session-user mailing list
> Cgi...@li...
> https://lists.sourceforge.net/lists/listinfo/cgi-session-user
>

-- 
Ron Savage - savage.net.au