Problem with alignment under Linux
"Axel Seibert" <[email protected]> Wed, 8 Feb 2012 14:42:32 +0100
| Newsgroups | gmane.comp.python.ctypes |
|---|---|
| Message-ID | <[email protected]> |
Dear list!
Please excuse the long posting
Nearly two years ago we had a discussion here on the list about an issue that
I have under Linux
(http://sourceforge.net/mailarchive/forum.php?thread_name=C3A6EB6D-E1DD-4D91-A579-9A88F5040ED5%40globonaut.org&forum_name=ctypes-users).
Unfortunately, we never resolved the issue, and I would like to take up the
discussion once more.
Problem:
Im wrapping a C library from an enterprise software environment (ARSystem
from BMC) with ctypes in python (I dont have access to the source code or any
details about the compiler used). This is working fine under Windows and SPARC
Solaris. Under Linux its not working with the standard ctypes setup. Does
not work means that I cannot use any of the library calls to talk to the
server, as it does not receive the parameters correctly.
Assumption
It seems that the memory alignment for this library under Linux does not match
ctypes expectations.
Workaround 1
By using _pack_ = 8, for some of the structures and unions, I can get this to
work reliably.
The problem is that ctypes' implementation limits the maximum size of the
_pack_ value. In other words, my workaround only works with a patched version
of ctypes (for a patch against python 2.7.2, please see below).
Workaround 2
During the discussion here on the mailinglist, Thomas developed two python
workarounds, one for structures
(http://sourceforge.net/mailarchive/message.php?msg_id=24837697), and one for
unions (http://sourceforge.net/mailarchive/message.php?msg_id=25006328), to
cope with my problem.
However, with those workarounds, it still does not work.
I have now researched this a bit more, and have come up with the following
test case:
-----------------------------------------
#!env python
from ctypes import *
import sys
def fill_structure(a, fields, override = False):
"""Patch the 'fields' sequence for a ctypes Structure by inserting
dummy fields named '' so that the alignment of the original fields
is a multiple of the 'a' value.
"""
if sys.platform.startswith('linux') or override:
result = []
size = 0
for n, t in fields:
size += sizeof(t)
missing = (a - sizeof(t)) % a
result.append((n, t))
if missing:
result.append(("", c_char * missing))
return result
else:
return fields
def fill_union(value, fields, override = False):
if sys.platform.startswith('linux') or override:
size = 0
for name, typ in fields:
size = max(size, sizeof(typ))
delta = 8 - size % 8
return fields + [("", c_char * (size + delta))]
else:
return fields
size_t = c_uint
ARLong32 = c_long
ARTimestamp = ARLong32
ARTime = ARLong32
ARULong32 = c_ulong
class N13ARValueStruct3DOLLAR_1E(Union):
_fields_ = fill_union(8, [
# ar.h 756
('noval_', size_t),
('keyNum', c_uint),
('intVal', ARLong32),
('realVal', c_double),
('charVal', c_char_p),
('diaryVal', c_char_p)
], override=True)
class ARValueStruct(Structure):
_fields_ = fill_structure(8, [
# ar.h 753
('dataType', c_uint),
('u', N13ARValueStruct3DOLLAR_1E),
], override=True)
print "This is on plattform: %s" % sys.platform
print "This is the alignment:"
print "%14s: %7s %5s %9s" % ('Name', 'Offset', 'Size', 'Alignment')
for name, typ in ARValueStruct._fields_:
if name:
print "%14s: %7d %5d %9d" % (name, getattr(ARValueStruct,
name).offset, sizeof(typ), alignment(typ))
for name, typ in N13ARValueStruct3DOLLAR_1E._fields_:
if name:
print "%14s: %7d %5d %9d" % (name, getattr(N13ARValueStruct3DOLLAR_1E,
name).offset, sizeof(typ), alignment(typ))
-----------------------------
If you run this under Linux, it will print out the following result:
This is on plattform: linux3
This is the alignment:
Name: Offset Size Alignment
dataType: 0 4 4
u: 8 16 4
noval_: 0 4 4
keyNum: 0 4 4
intVal: 0 4 4
realVal: 0 8 4
charVal: 0 4 4
diaryVal: 0 4 4
When I take the Structure and Union definition, add a _pack_ = 8 and use my
patched version of ctypes, I receive the following (please note the different
output for the field 'u'):
This is on plattform: linux3
This is the alignment:
Name: Offset Size Alignment
dataType: 0 4 4
u: 8 16 8
noval_: 0 4 4
keyNum: 0 4 4
intVal: 0 4 4
realVal: 0 8 4
charVal: 0 4 4
diaryVal: 0 4 4
Im asking the experts on this list to try to understand my problem. If you
are missing any details or additional information, please do not hesitate to
contact me. Im also willing to run any kinds of tests and stuff. I would very
much appreciate any python based solution. The current situation, that users
of the library need to patch Python cannot be the solution.
Thanx for your time & help!
Axel
Patch for python 2.7.2
*** Modules/_ctypes/cfield.c.orig 2012-02-05 12:56:21.902652940 +0100
--- Modules/_ctypes/cfield.c 2012-02-05 12:56:41.770652136 +0100
***************
*** 150,156 ****
/* fall through */
case NO_BITFIELD:
if (pack)
! align = min(pack, dict->align);
else
align = dict->align;
if (align && *poffset % align) {
--- 150,156 ----
/* fall through */
case NO_BITFIELD:
if (pack)
! align = pack; /*min(pack, dict->align);*/
else
align = dict->align;
if (align && *poffset % align) {
------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d