#399 VCPP98 atol() / VCPP2005 atol() return different values

[Mark Miesfeld]

There has been a change in the CRT function atol() between VCPP98 (Microsoft's Visual C++ 6.0) and VCPP2005 (Microsoft's Visual Studio 2005) that can cause different (buggy) behavior in an ooRexx compiled with Visual Studio 2005 than one compiled with Visual C++ 6.0.

The data range for a long is –2,147,483,648 to 2,147,483,647. In VCPP98, for a string in the range of 2,147,483,647 to 4,294,967,295, atol() would return the equivalent of the unsigned long value cast to a long.

However, in VCPP2005 2,147,483,647 to 4,294,967,295 is considered an overflow condition and LONG_MAX is returned.

So, with a VCPP98 compiled ooRexx a string like 2345332467 could be converted using atol() and cast to an unsigned long without problem. In VCPP2005 this does not work.

Here is some output from a test program that shows this. The same source was compiled with VCPP98 and VCPP2005:

E:\work.ooRexx\other\bugs.new\font.bug>test_atolVC2005.exe
Function: atol( "2100000000" ) = 2100000000
Function: atol( "2147483647" ) = 2147483647

Function: atol( "2199999999" ) = 2147483647
Overflow condition occurred.
printf as an unsigned long
Function: atol( "2199999999" ) = 2147483647

Function: atol( "4294967295" ) = 2147483647
Overflow condition occurred.
printf as an unsigned long
Function: atol( "4294967295" ) = 2147483647

E:\work.ooRexx\other\bugs.new\font.bug>test_atolVC98.exe
Function: atol( "2100000000" ) = 2100000000
Function: atol( "2147483647" ) = 2147483647

Function: atol( "2199999999" ) = -2094967297
printf as an unsigned long
Function: atol( "2199999999" ) = 2199999999

Function: atol( "4294967295" ) = -1
printf as an unsigned long
Function: atol( "4294967295" ) = 4294967295

E:\work.ooRexx\other\bugs.new\font.bug>

This is difinitely causing bugs in ooDialog where many parameters to the Windows UI API are unsigned longs. The setFont() method is a good example (and where I first ran across this.)

Font handles are unsigned longs. In ooDialog, the font is created and the handle passed back. This handle is then used in the setFont method. As long as the created font handle was below the 2147483647 boundary, setFont worked. If the font handle was greater than 2147483647, then setFont did not work.

In ooDialog, it looks like this may be a pervasive problem. It probably exists in the other Windows specific external functions. And, we will need to check for it in the interpreter itself (when built on Windows.)

[Mark Miesfeld]

Simple test program that shows the difference between compilers

[Mark Miesfeld]

Logged In: YES
user_id=191588
Originator: YES

Rick,

I only see one place in ooDialog where this is causing a bug. It is only when a font object is created. (And when the font is deleted.)

I see two ways to fix it. Both work fine, but both have some drawback. So, I thought I'd get your advice.

The problem happens in SendWinMsg("ANY",...) in oovmsg.c in this code:

if (ISHEX(argv[i+1].strptr))
n[i] = (ULONG)strtol(argv[i+1].strptr,'\0',16);
else
n[i] = atol(argv[i+1].strptr);

Font handles are often greater than 2147483647. If the font handle is larger, then under Visual Studio 2005 it gets changed to LONG_MAX and is no longer a valid font handle.

Fix 1: Do:

if (ISHEX(argv[i+1].strptr))
n[i] = strtoul(argv[i+1].strptr,'\0',16);
else
n[i] = strtoul(argv[i+1].strptr,'\0',10);

instead.

All args used in SendWinMsg("ANY",...) are unsigned longs so this works just fine. A similar fix is needed for DeleteObject() in HandleDC_Obj().

Fix 2: In all places where values are returned to ooRexx from the ooDialog C functions, some form of ltoa() is used to return the string value, except for when a font is created. For the creation of a font only, ultoa() is used. This fix would use ltoa() to return the font handle to ooRexx. Then, when it is sent back to the C function, strtol() can be used, and cast to an unsigned long.

I'm leaning towards fix number 1. The drawback to fix number 2 is that it just seems somewhat wrong to me to have the font be a negative number in ooDialog, when I know font handles are never negative. If you do:

font = self~createFont(fontName, fontSize)
say 'Font handle:' font

and see Font handle: -2094967297 you would think something is wrong.

The drawback to fix #1 is that some arguments to the SendMessage() API can be negative numbers.

Although I have looked and know that at this point in time all calls to SendWinMsg("ANY",...) in ooDialog do not use any negative arguments, it has the potential of causing a bug in the future if someone adds a call to SendWinMsg("ANY",...) in ooDialog that does use a negative arguement.

I guess the real problem is trying to have this very generic SendWinMsg() where you do not know what the arguements really are. Another fix would be to pull the font stuff out of the generic function and have very specific calls for the font stuff.

I'm set to commit the first fix, thought I see if you have any comment.

[Rick McGuire]

Logged In: YES
user_id=1125291
Originator: NO

Your fix #1 looks good to me.

[Mark Miesfeld]

Logged In: YES
user_id=191588
Originator: YES

Committed revision 770.

This commit fixes the one spot in ooDialog where the compiler differences was casuing a bug.