perfparse-devel-int

[Perfparse-devel-int] Re: [Perfparse-users] schema incorrect use perfparse-db-tool --update

[Yves M. <yme...@li...>]

> Please edit libpp_mysql/dbms.c
>
> Near the top please add:
>
> #include <locale.h>
>
> Then after line 227 in function now_connect please add:
>
> setlocale(LC_NUMERIC, "POSIX");
>
> Now please 'make' and let us know.
>
> To other members:
>
> Would any of you know which component effects this definition?

Only the numeric format on localized builds of perfparse.
In reality : nearly nothing.

I have another question : cannot you ask mysql to read your queries as PO=
SIX instead of
localized ? Somebody knows how ?

> Is this fix supported by FreeBSD, SUN and others if added to the
> standard distribution?

It should be. We already use setlocale() in perfparsed.

Yves
--=20
- Homepage    - http://ymettier.free.fr - http://www.logicacmg.com -
- GPG key     - http://ymettier.free.fr/gpg.txt                    -
- Maitretarot - http://www.nongnu.org/maitretarot/                 -
- Perfparse   - http://perfparse.sf.net/                           -

Re: [Perfparse-devel-int] Re: [Perfparse-users] schema incorrect use perfparse-db-tool --update

[Ben C. <bcl...@pe...>]

Hi Yves,

The problem is not with MySQL but with 'c'.  The language translation 
introduced a line of code which forces the 'format' to output floating 
numbers in local format:

printf("%f", 0.17) -> "0,17"

I can see in the code the calls to setlocale.

Sample:

setlocale(LC_ALL, "");
bindtextdomain(PACKAGE, LOCALEDIR);
textdomain(PACKAGE);

Looking at the man page:

'If locale is "", each part of the locale that should be modified is set 
according to the  environment  variables.'

Therefore this code is looking at the language and causing the change in 
behavior of the 'printf' format.

I can see this being an attractive solution for data presentation.  I am 
sure many people would prefer the graph labels to show as '0,000' and 
not '0.000'.

There are some problems:

1. The 'atof' is not locale safe.  0,17 -> 0.00

2. The interface to the database.  DBMS expects numbers in POSIX format:

"INSERT (1,17)"   is understood as "INSERT ('1', '17')"
"INSERT ('1,17')" is understood as "INSERT ('1');

Users with this version in these countries are therefore sending corrupt 
data to their database.  Eg, 'show_load' will be storing only the 
integer part of the numbers.  THIS IS BAD!

3.  The HTML CGI interface.  Floating numbers are likely to be 
corruption as above.


The hack I suggested will fix this:

setlocale(LC_NUMERIC, "POSIX");

But it may be a little too blunt.

So what do we do?

1. Set locale to "POSIX" and loose the localized display?

2. Set locale to "POSIX" before any SQL, and back to "" afterwards:

setlocale(LC_NUMERIC, "POSIX");
sprintf(sql, "INSERT data = %f", 0.17);
setlocale(LC_NUMERIC, "");

3. Set the locale to "POSIX" for all the code, accept the presentation. 
  Here and here only, set back to "":

setlocale(LC_NUMERIC, "");
printf("Graph label:  %f", 0.17);
setlocale(LC_NUMERIC, "POSIX");

???

Ben






Yves Mettier wrote:
>>Please edit libpp_mysql/dbms.c
>>
>>Near the top please add:
>>
>>#include <locale.h>
>>
>>Then after line 227 in function now_connect please add:
>>
>>setlocale(LC_NUMERIC, "POSIX");
>>
>>Now please 'make' and let us know.
>>
>>To other members:
>>
>>Would any of you know which component effects this definition?
> 
> 
> Only the numeric format on localized builds of perfparse.
> In reality : nearly nothing.
> 
> I have another question : cannot you ask mysql to read your queries as POSIX instead of
> localized ? Somebody knows how ?
> 
> 
>>Is this fix supported by FreeBSD, SUN and others if added to the
>>standard distribution?
> 
> 
> It should be. We already use setlocale() in perfparsed.
> 
> Yves


-- 
Ben Clewett  bcl...@pe...
PerfParse    http://www.perfparse.org
PP FAQ       http://wiki.perfparse.org/tiki-list_faqs.php

Re: [Perfparse-devel-int] Re: [Perfparse-users] schema incorrect use perfparse-db-tool --update

[Yves <yme...@pe...>]

> Hi Yves,
>
> The problem is not with MySQL but with 'c'.  The language translation
> introduced a line of code which forces the 'format' to output floating
> numbers in local format:
>
> printf("%f", 0.17) -> "0,17"

OK :)



> 1. The 'atof' is not locale safe.  0,17 -> 0.00

I suppose that strtof and strtod are not locale safe either.

>
> 2. The interface to the database.  DBMS expects numbers in POSIX format=
:
>
> "INSERT (1,17)"   is understood as "INSERT ('1', '17')"
> "INSERT ('1,17')" is understood as "INSERT ('1');
>
> Users with this version in these countries are therefore sending corrup=
t
> data to their database.  Eg, 'show_load' will be storing only the
> integer part of the numbers.  THIS IS BAD!

I agree :)

> 3.  The HTML CGI interface.  Floating numbers are likely to be
> corruption as above.
>
>
> The hack I suggested will fix this:
>
> setlocale(LC_NUMERIC, "POSIX");
>
> But it may be a little too blunt.

Correct as a quick fix.

> So what do we do?
>
> 1. Set locale to "POSIX" and loose the localized display?

no :)

> 2. Set locale to "POSIX" before any SQL, and back to "" afterwards:
>
> setlocale(LC_NUMERIC, "POSIX");
> sprintf(sql, "INSERT data =3D %f", 0.17);
> setlocale(LC_NUMERIC, "");

yes, maybe this is best.

> 3. Set the locale to "POSIX" for all the code, accept the presentation.
>   Here and here only, set back to "":
>
> setlocale(LC_NUMERIC, "");
> printf("Graph label:  %f", 0.17);
> setlocale(LC_NUMERIC, "POSIX");

Probably more to print than floats to send to the database. 2 is better.

Yves
--=20
- Homepage    - http://ymettier.free.fr - http://www.logicacmg.com -
- GPG key     - http://ymettier.free.fr/gpg.txt                    -
- Maitretarot - http://www.nongnu.org/maitretarot/                 -
- Perfparse   - http://perfparse.sf.net/                           -

Re: [Perfparse-devel-int] Re: [Perfparse-users] schema incorrect

[Ben C. <bcl...@pe...>]

Ok.

When ever the CGI or SQL is called with floats, I will:

setlocale(LC_NUMERIC, "POSIX");
sprintf(sql, "INSERT data = %f", 0.17);
setlocale(LC_NUMERIC, "");

Since there are users battling this problem now.  And this will corrupt 
the database.  I'll have to do this now.  The summary tables will wait a 
day.

None of your work is with the CGI or SQL today?

Ben




Yves wrote:

>>Hi Yves,
>>
>>The problem is not with MySQL but with 'c'.  The language translation
>>introduced a line of code which forces the 'format' to output floating
>>numbers in local format:
>>
>>printf("%f", 0.17) -> "0,17"
> 
> 
> OK :)
> 
> 
> 
> 
>>1. The 'atof' is not locale safe.  0,17 -> 0.00
> 
> 
> I suppose that strtof and strtod are not locale safe either.
> 
> 
>>2. The interface to the database.  DBMS expects numbers in POSIX format:
>>
>>"INSERT (1,17)"   is understood as "INSERT ('1', '17')"
>>"INSERT ('1,17')" is understood as "INSERT ('1');
>>
>>Users with this version in these countries are therefore sending corrupt
>>data to their database.  Eg, 'show_load' will be storing only the
>>integer part of the numbers.  THIS IS BAD!
> 
> 
> I agree :)
> 
> 
>>3.  The HTML CGI interface.  Floating numbers are likely to be
>>corruption as above.
>>
>>
>>The hack I suggested will fix this:
>>
>>setlocale(LC_NUMERIC, "POSIX");
>>
>>But it may be a little too blunt.
> 
> 
> Correct as a quick fix.
> 
> 
>>So what do we do?
>>
>>1. Set locale to "POSIX" and loose the localized display?
> 
> 
> no :)
> 
> 
>>2. Set locale to "POSIX" before any SQL, and back to "" afterwards:
>>
>>setlocale(LC_NUMERIC, "POSIX");
>>sprintf(sql, "INSERT data = %f", 0.17);
>>setlocale(LC_NUMERIC, "");
> 
> 
> yes, maybe this is best.
> 
> 
>>3. Set the locale to "POSIX" for all the code, accept the presentation.
>>  Here and here only, set back to "":
>>
>>setlocale(LC_NUMERIC, "");
>>printf("Graph label:  %f", 0.17);
>>setlocale(LC_NUMERIC, "POSIX");
> 
> 
> Probably more to print than floats to send to the database. 2 is better.
> 
> Yves


-- 
Ben Clewett  bcl...@pe...
PerfParse    http://www.perfparse.org
PP FAQ       http://wiki.perfparse.org/tiki-list_faqs.php

Re: [Perfparse-devel-int] Re: [Perfparse-users] schema incorrect

[Ben C. <bcl...@pe...>]

The attached patch against 0.104.5 should fix the locale problems.

To all contributers: Please remember with all new work.  If the 
float/double is not being displayed, use this convention!

Ben

Ben Clewett wrote:

> Ok.
> 
> When ever the CGI or SQL is called with floats, I will:
> 
> setlocale(LC_NUMERIC, "POSIX");
> sprintf(sql, "INSERT data = %f", 0.17);
> setlocale(LC_NUMERIC, "");
> 
> Since there are users battling this problem now.  And this will corrupt 
> the database.  I'll have to do this now.  The summary tables will wait a 
> day.
> 
> None of your work is with the CGI or SQL today?
> 
> Ben
> 
> 
> 
> 
> Yves wrote:
> 
>>> Hi Yves,
>>>
>>> The problem is not with MySQL but with 'c'.  The language translation
>>> introduced a line of code which forces the 'format' to output floating
>>> numbers in local format:
>>>
>>> printf("%f", 0.17) -> "0,17"
>>
>>
>>
>> OK :)
>>
>>
>>
>>
>>> 1. The 'atof' is not locale safe.  0,17 -> 0.00
>>
>>
>>
>> I suppose that strtof and strtod are not locale safe either.
>>
>>
>>> 2. The interface to the database.  DBMS expects numbers in POSIX format:
>>>
>>> "INSERT (1,17)"   is understood as "INSERT ('1', '17')"
>>> "INSERT ('1,17')" is understood as "INSERT ('1');
>>>
>>> Users with this version in these countries are therefore sending corrupt
>>> data to their database.  Eg, 'show_load' will be storing only the
>>> integer part of the numbers.  THIS IS BAD!
>>
>>
>>
>> I agree :)
>>
>>
>>> 3.  The HTML CGI interface.  Floating numbers are likely to be
>>> corruption as above.
>>>
>>>
>>> The hack I suggested will fix this:
>>>
>>> setlocale(LC_NUMERIC, "POSIX");
>>>
>>> But it may be a little too blunt.
>>
>>
>>
>> Correct as a quick fix.
>>
>>
>>> So what do we do?
>>>
>>> 1. Set locale to "POSIX" and loose the localized display?
>>
>>
>>
>> no :)
>>
>>
>>> 2. Set locale to "POSIX" before any SQL, and back to "" afterwards:
>>>
>>> setlocale(LC_NUMERIC, "POSIX");
>>> sprintf(sql, "INSERT data = %f", 0.17);
>>> setlocale(LC_NUMERIC, "");
>>
>>
>>
>> yes, maybe this is best.
>>
>>
>>> 3. Set the locale to "POSIX" for all the code, accept the presentation.
>>>  Here and here only, set back to "":
>>>
>>> setlocale(LC_NUMERIC, "");
>>> printf("Graph label:  %f", 0.17);
>>> setlocale(LC_NUMERIC, "POSIX");
>>
>>
>>
>> Probably more to print than floats to send to the database. 2 is better.
>>
>> Yves
> 
> 
> 


-- 
Ben Clewett  bcl...@pe...
PerfParse    http://www.perfparse.org
PP FAQ       http://wiki.perfparse.org/tiki-list_faqs.php