Re: Buffer overrun in 64 bit build: problem found, solution lacking

[Sean M. <se...@ro...>]

On 4/14/09 11:20 PM, James W. Walker said:

>The code you're looking at is not in e3geom_trimesh_copydata, but in  
>e3geom_trimesh_copyattributes.

Ack, yes, sorry about that.

>I guess it's trying to get the size of  
>the data for any given attribute type, in a way that would be  
>compatible with custom attributes.

That was my impression too.

>It would definitely be a bad idea  
>to replace theClass->GetInstanceSize() by sizeof(TQ3Vector3D), because  
>it won't be right for other attributes than normal vectors, such as UV  
>coordinates.  If you only care about the built-in attribute types, you  
>could use an array lookup or switch statement to get the right size,  
>but it would be nice to figure out what's going wrong.

Agreed.

>The class in  
>question is E3NormalAttribute, a subclass of E3Attribute with one  
>additional member, a TQ3Vector3D.  It looks like GetInstanceSize  
>basically takes sizeof(E3NormalAttribute) - sizeof(E3Attribute).

I don't think this kind of subtraction is ever correct/reliable,
really.  The structure padding is not mandated by the C language, and so
the compiler is free to choose it.  Or am I missing something...?

>Apparently, in the 64-bit case, 4 pad bytes were added between the  
>base class and the instance data of the subclass. What exactly are the  
>rules about when structures are padded in the 64-bit world?

The Mac OS X ABI spells out the rules on OS X; other platforms have
other rules of course.  For example, for ppc64:

<http://developer.apple.com/documentation/DeveloperTools/Conceptual/
LowLevelABI/110-64-bit_PowerPC_Function_Calling_Conventions/
64bitPowerPC.html#//apple_ref/doc/uid/TP40002471>

I've just noticed Quesa.h has "#pragma options align=power".  Perhaps
the code really depends on that type of alignment?

-- 
____________________________________________________________
Sean McBride, B. Eng                 se...@ro...
Rogue Research                        www.rogue-research.com 
Mac Software Developer              Montréal, Québec, Canada