#10 [PATCH] Fix for cast-256 on Alpha
Originally reported in Gentoo Bugzilla. (https://bugs.gentoo.org/show_bug.cgi?id=132356)
The KNOWN-BUGS file lists the following problem:
- cast-256 and rc6 do not work properly on Alpha (64 bit) machines
The problem was the rotl32 macro in lib/mcrypt_modules.h.
Consider the case where the number to be shifted is 1 and the shift argument is 32. The expected result is that the original number is the same as the final number, since it rotated the entire width of the field.
On Alpha (and others?), the 64-bit registers prevent it from rotating, since a shift of 32 actually rotates bits into the upper 32-bits, yielding a result of 0.
This did not affect AMD64 because gcc is able to generate 32-bit shift instructions, whereas all shifts on Alpha are 64-bit.
The fix is to mask the shift argument before shifting. The included patch masks the shift arguments in the rot{r,l}{16,32} macros and deletes the KNOWN-BUGS file as there are no other known bugs listed.
Please provide a new version of libmcrypt with this fix.
Patch to fix cast-256 on Alpha