[Tcl-bugs] [ tcl-Bugs-2848176 ] Invalid results from "expr", v8.4.*, v8.5.*, v8.6.*

[SourceForge.net <no...@so...>]

Bugs item #2848176, was opened at 2009-08-31 23:31
Message generated for change (Settings changed) made by dgp
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=2848176&group_id=10894

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: 48. Number Handling
Group: current: 8.5.7
>Status: Closed
>Resolution: Invalid
Priority: 9
Private: No
Submitted By: Karl C. Hansen (kchansen)
Assigned to: Don Porter (dgp)
Summary: Invalid results from "expr", v8.4.*, v8.5.*, v8.6.*

Initial Comment:
Both crc16.tcl and crc32.tcl use a variable named "signmask", that is constructed from another variable named "signbit".

For 32-bit data, signbit is 0x8000000.

For 32-bit CRCs, the signmask appears to be designed to mask off the high-byte, i.e. it should end up 0x00ffffff.  Signmask is computed as follows:

set signmask [expr {~$signbit>>7}]

Printing $signbit shows the expected value of 0x80000000:

% puts $signbit
0x80000000

But if we print the bit-complement of this (which in 32-bit land we expect to be positive) we get:

% expr ~$signbit
-2147483649

When we format this using "format 0x%x" we oddly get:

% format 0x%x [expr ~$signbit]
0x7fffffff

Even worse, the above calculations for signmask now no longer give 0x00FFFFFF, but rather 0xFEFFFFFF, which is pretty much guaranteed to give the wrong result.

It looks like the number is kind of living in both "long-land" and 32-bit-land.  In long-land the 0x80000000 gets sign-extended from 32-bit-land, and after that the bit-complement would indeed be still negative.  But the format statement seems to think the number is 32-bit-land-only....

Help!

----------------------------------------------------------------------

>Comment By: Don Porter (dgp)
Date: 2009-09-01 13:14

Message:

This is not a bug.  It is an intentional feature change.

Starting with release Tcl 8.4.0 in 2002 (!) the numeric
literal value "0x80000000" is taken as the number 2147483648 .
This is an incompatible change from Tcl 8.3 and earlier
where the interpretation was platform dependent.  On 32-bit
systems it was the number -2147483648 and on 64-bit
systems it was the number 2147483648.  It was TIP 72
that started this change, and since then TIP 237 extended
the change so that integer literals are interpreted within
a system of unlimited range and no overflows at all.
If you want a negative number in hex notation, you must
lead with the minus sign.

Also, do not over look the "l" and "ll" modifers in your
[format] spec's:

% format %#x [expr ~0x80000000]
0x7fffffff
% format %#lx [expr ~0x80000000]
0xffffffff7fffffff
% format %#llx [expr ~0x80000000]
-0x80000001


Sounds like the crc*.tcl files you have are buggy.  Even
when they might have been used with Tcl 8.3, they were
still buggy, since they did not work on 64-bit systems.  Check
that you are still using some that are out of date.  If not,
report the bugs to the maintainer(s) of the package(s).

The possibly legitimate gripe against Tcl is that
Tcl does not provide any good tools in the built-in
command set to easily, efficiently, and with complete
cross-platform agreement operate on data known
to be "32-bit integers"  or "64-bit integers".  Tcl leaves
too many of the definitional details to the underlying
C implementation.  Tcl scripts that really want to operate
on 32-bit buffers find that they are forced to to a lot of
masking against 0xFFFFFFFF to get the job done right.
Whether the right answer to this problem is more 
operators and options in Tcl's built-ins or a good
extension or two, is a matter for someone motivated
to pursue.

----------------------------------------------------------------------

Comment By: Karl C. Hansen (kchansen)
Date: 2009-09-01 10:31

Message:
...first line of the description:

"...crc16.tcl and crc32.tcl...."

These are both in the crc portion of "tcllib".

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2009-09-01 04:27

Message:
It'd help a lot if you could let us know which crc*.tcl scripts have this
problem. (The issue is probably that the code therein is not 64-bit safe,
and so would have actually been a problem on some platforms even before
8.4. Though those platforms were much less common back then.)

----------------------------------------------------------------------

Comment By: Karl C. Hansen (kchansen)
Date: 2009-08-31 23:33

Message:
Because this deals with math and breaks some existing code (crc-32
calculations) I am hoping my assigning the highest priority will stick....

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=2848176&group_id=10894