Re: [ctypes-users] feature request for setting _pack_ to a higher value

[Thomas H. <th...@ct...>]

Am 22.03.2010 23:06, schrieb Axel Seibert:
> 
> 
> 
> So, let me try to explain this maybe in some other words; I have this third party application library that supplies me with one or more header files. When I compile those with h2xml -> xml2py (using gccxml), I get the following resulting python code (and a coupleof other 1000 lines more):
> STRING = c_char_p
> ARLong32 = c_long
> ARULong32 = c_ulong
> ARTimestamp = ARLong32
> ARTime = ARLong32
> 
> class ARValueStruct(Structure):
>   pass
> class N13ARValueStruct3DOT_1E(Union):
>   pass
> N13ARValueStruct3DOT_1E._fields_ = [
>   ('noval_', size_t),
>   ('keyNum', c_uint),
>   ('intVal', ARLong32),
>   ('realVal', c_double),
>   ('charVal', STRING),
>   ('diaryVal', STRING),
>   ('enumVal', ARULong32),
>   ('timeVal', ARTimestamp),
>   ('maskVal', ARULong32),
>   ('timeOfDayVal', ARTime),
>   ('byteListVal', POINTER(ARByteList)),
>   ('decimalVal', STRING),
>   ('attachVal', POINTER(ARAttachStruct)),
>   ('ulongVal', ARULong32),
>   ('coordListVal', POINTER(ARCoordList)),
>   ('dateVal', c_int),
>   ('currencyVal', POINTER(ARCurrencyStruct)),
>   ('ptrVal', c_void_p),
> ]
> ARValueStruct._fields_ = [
>   ('dataType', c_uint),
>   ('u', N13ARValueStruct3DOT_1E),
> ]
> 
> When I use this in calling the library functions under Windows & Solaris everything works.
> When I use this in calling the library functions under Linux, I get completely unexpected results and crashes.
> 
> An older version of xml2py (from 2005) not only generated the python C types, but also two asserts for each of them: one to check the size of the struct, and one to check the alignment. Running those checks over the structures under Linux, revealed that for 44 of roughly 400 Structs, the default alignment under Linux did not match that of Windows or Solaris (please remember, everything using 32bit).
> Setting _pack_ = 8 at those 44 Structs resolved the issue -- no more unexpected results or crashes.
> The only thing is that I needed to patch file cfield.c in one single line of source code to enable setting _pack_ to a higher value than the platform default.
> 
> I hope I could describe it clearer this time.

Axel,

the semantics of the _pack_ attribute is (mainly) designed to match the behaviour
of the Microsoft C/C++ compiler.  Here is one document that explains it:

http://msdn.microsoft.com/en-us/library/xh3e3fd0%28v=VS.80%29.aspx

It may well be that there is a difference in the MSVC compiler and the Solaris or Linux
compilers, but I think think it is inappropriate to change the ctypes semantics at this point.

However, you should not underestimate the power of Python.  Here is a function that I just
wrote which will hopefully do what you need:  It changes the _fields_ list for a ctypes
Structure by inserting padding bytes.  You just have to replace all the

   ...._fields_ = [...]

definitions by this code:

   ...._fields_ = padded(8, [...])

where 8 is value for the padding that you need.  It may even be possible
to write some code that automatically changes the code generated by xml2py,
or even the code generator itself.  Here is the function with a little test:

<snip>
from ctypes import *

def padded(a, fields):
    """Patch the 'fields' sequence for a ctypes Structure by inserting
    dummy fields named '' so that the alignment of the original fields
    is a multiple of the 'a' value.
    """
    result = []
    size = 0
    for n, t in fields:
        size += sizeof(t)
        missing = a - size % a
        result.append((n, t))
        if missing:
            result.append(("", c_char * missing))
    return result

class X(Structure):
    _fields_ = padded(8,
                      [("a", c_char),
                       ("a1", c_long),
                       ("b", c_short),
                       ("b1", c_long),
                       ("c", c_long),
                       ("c1", c_long)])

for name, typ in X._fields_:
    if name:
        print "%3s: %2d %2d %2d" % (name, getattr(X, name).offset, sizeof(typ), alignment(typ))
</snip>

-- 
Thanks,
Thomas