Guppy Python Programming Environment / Bugs / #2 Doesn't work on 64-bit architectures

Nobody/Anonymous - 2007-08-10

Logged In: NO

I have experienced the same issue on an AMD 64-bit system.

bingham at cenix-bioscience dot com

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Ragnar Kjřrstad - 2008-03-03

Logged In: YES
user_id=10550
Originator: NO

It segfaults for me on x86_64 too:
Core was generated by `python -c import guppy; guppy.hpy().heap()'.
Program terminated with signal 11, Segmentation fault.
#0 hv_is_obj_hidden (hv=0x2aaaae1b0210, obj=0xfffffffc9d502bc0) at src/heapy/hv.c:562
562 PyTypeObject *type = obj->ob_type;
(gdb) bt
#0 hv_is_obj_hidden (hv=0x2aaaae1b0210, obj=0xfffffffc9d502bc0) at src/heapy/hv.c:562
#1 0x00002aaaae1bd2ff in hv_cms_rec (obj=0xfffffffc9d502bc0, ta=0x7fff119cb680) at src/heapy/hv.c:588
#2 0x00002aaaae3d6d8d in bitfields_iterate (f=0xfeb2620, end_f=0xfeb54b0, visit=0x2aaaae3ddc20 <mutnodeset_iterate_visit>, arg=0x7fff119cb630)
at src/sets/bitset.c:2335
#3 0x00002aaaae3d8683 in NyAnyBitSet_iterate (v=<value optimized out>, visit=0x2aaaae3ddc20 <mutnodeset_iterate_visit>, arg=0x7fff119cb630)
at src/sets/bitset.c:2354
#4 0x00002aaaae3dd2a3 in NyNodeSet_iterate (ns=0x2aaaae1766c8, visit=0x2aaaae1bd2e0 <hv_cms_rec>, arg=0x7fff119cb680) at src/sets/nodeset.c:531
#5 0x00002aaaae1be1cd in hv_cleanup_mutset (hv=<value optimized out>, ns=<value optimized out>) at src/heapy/hv.c:607
#6 0x00002aaaae1be8ce in hv_heap (self=0x2aaaae1b0210, args=<value optimized out>, kwds=<value optimized out>) at src/heapy/hv.c:873
#7 0x0000003c9d2947a7 in PyEval_EvalFrame () from /usr/lib64/libpython2.4.so.1.0
#8 0x0000003c9d295905 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.4.so.1.0
#9 0x0000003c9d29405f in PyEval_EvalFrame () from /usr/lib64/libpython2.4.so.1.0
#10 0x0000003c9d294486 in PyEval_EvalFrame () from /usr/lib64/libpython2.4.so.1.0
#11 0x0000003c9d295905 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.4.so.1.0
#12 0x0000003c9d295952 in PyEval_EvalCode () from /usr/lib64/libpython2.4.so.1.0
#13 0x0000003c9d2b1ea9 in Py_CompileString () from /usr/lib64/libpython2.4.so.1.0
#14 0x0000003c9d2b3140 in PyRun_SimpleStringFlags () from /usr/lib64/libpython2.4.so.1.0
#15 0x0000003c9d2b9406 in Py_Main () from /usr/lib64/libpython2.4.so.1.0
#16 0x0000003c8a81d8a4 in __libc_start_main () from /lib64/libc.so.6
#17 0x0000000000400629 in _start ()

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Sverker Nilsson - 2008-03-14

Logged In: YES
user_id=356603
Originator: NO

Well, I am working on it, installed AMD64 Ubuntu yesterday:-)

It compiles cleanly now (svn rev 4) but still segfaults. I may be back in a couple days or so.

>>> from guppy import hpy
>>> hp=hpy()
>>> h=hp.heap()

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 47995668334304 (LWP 7571)]
hv_is_obj_hidden (hv=0x2ba6dd56e7c0, obj=0xfffffffedbe34050)
at src/heapy/hv.c:562
562 PyTypeObject *type = obj->ob_type;
(gdb) bt
#0 hv_is_obj_hidden (hv=0x2ba6dd56e7c0, obj=0xfffffffedbe34050)
at src/heapy/hv.c:562
#1 0x00002ba6dd58fbef in hv_cms_rec (obj=0xfffffffedbe34050,
ta=0x7fffcec93800) at src/heapy/hv.c:588
#2 0x00002ba6dd7aa67f in bitfields_iterate (f=0x8c90b0, end_f=0x8cc220,
visit=0x2ba6dd7b1780 <mutnodeset_iterate_visit>, arg=0x7fffcec937b0)
at src/sets/bitset.c:2335
#3 0x00002ba6dd7ac0d3 in NyAnyBitSet_iterate (v=<value optimized out>,
visit=0x2ba6dd7b1780 <mutnodeset_iterate_visit>, arg=0x7fffcec937b0)
at src/sets/bitset.c:2354
#4 0x00002ba6dd7b0da3 in NyNodeSet_iterate (ns=0x848870,
visit=0x2ba6dd58fbd0 <hv_cms_rec>, arg=0x7fffcec93800)
at src/sets/nodeset.c:531
#5 0x00002ba6dd590d1d in hv_cleanup_mutset (hv=<value optimized out>,
ns=<value optimized out>) at src/heapy/hv.c:607
#6 0x00002ba6dd59142e in hv_heap (self=0x2ba6dd56e7c0,
args=<value optimized out>, kwds=<value optimized out>)
at src/heapy/hv.c:873
#7 0x00000000004883fc in PyEval_EvalFrameEx ()
#8 0x0000000000489d60 in PyEval_EvalCodeEx ()
#9 0x0000000000487f32 in PyEval_EvalFrameEx ()
#10 0x0000000000488c3e in PyEval_EvalFrameEx ()
#11 0x0000000000489d60 in PyEval_EvalCodeEx ()
#12 0x0000000000489da2 in PyEval_EvalCode ()
#13 0x00000000004abbb7 in PyRun_InteractiveOneFlags ()
#14 0x00000000004abdc4 in PyRun_InteractiveLoopFlags ()
#15 0x00000000004abeca in PyRun_AnyFileExFlags ()
#16 0x0000000000414725 in Py_Main ()
#17 0x00002ba6dc8f2b44 in __libc_start_main () from /lib/libc.so.6
#18 0x0000000000413c69 in _start ()
(gdb) sverker@sverker-laptop:~/guppy-pe/trunk/guppy$

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Sverker Nilsson - 2008-03-16

Logged In: YES
user_id=356603
Originator: NO

Well, I will be back when I will be back. Probably next week, but no promises.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Sverker Nilsson - 2008-03-18

Logged In: YES
user_id=356603
Originator: NO

SVN Rev 10:

I have now compiled and tested successfully with AMD64 and Ubuntu 7.10, either 64 bit or 32 bit version.

Python 2.5 runs in either 64 or 32 bit.

But in 64 bit mode, Python 2.4 segfaults. Seems a little bit tricky and I suppose it is not so
common to use 2.4 with 64 bits?

>>> h=hp.heap()
*** glibc detected *** /usr/bin/python2.4: free(): invalid next size (normal): 0x00000000008641d0 ***
======= Backtrace: =========
/lib/libc.so.6[0x2b73d6d8eb0a]
/lib/libc.so.6[0x2b73d6d9073e]
/lib/libc.so.6(realloc+0x129)[0x2b73d6d92979]
guppy/sets/setsc.so[0x2b73d7c7615c]
guppy/sets/setsc.so[0x2b73d7c761a2]
guppy/sets/setsc.so[0x2b73d7c7664a]
guppy/sets/setsc.so[0x2b73d7c76a8f]
guppy/sets/setsc.so(NyNodeSet_setobj+0x39)[0x2b73d7c7ac89]
guppy/sets/setsc.so[0x2b73d7c7422f]
guppy/sets/setsc.so(NyAnyBitSet_iterate+0x83)[0x2b73d7c75c73]
guppy/sets/setsc.so(NyNodeSet_iterate+0x93)[0x2b73d7c7a8a3]
guppy/heapy/heapyc.so[0x2b73d7a5bf64]
/usr/bin/python2.4(PyEval_EvalFrame+0x4877)[0x476c07]
/usr/bin/python2.4(PyEval_EvalCodeEx+0x846)[0x478276]
/usr/bin/python2.4(PyEval_EvalFrame+0x412a)[0x4764ba]
/usr/bin/python2.4(PyEval_EvalFrame+0x47be)[0x476b4e]
/usr/bin/python2.4(PyEval_EvalCodeEx+0x846)[0x478276]
/usr/bin/python2.4(PyEval_EvalCode+0x32)[0x478322]
/usr/bin/python2.4(PyRun_InteractiveOneFlags+0x210)[0x49c210]
/usr/bin/python2.4(PyRun_InteractiveLoopFlags+0x44)[0x49c314]
/usr/bin/python2.4(PyRun_AnyFileExFlags+0x4a)[0x49cd9a]
/usr/bin/python2.4(Py_Main+0x99a)[0x41263a]
/lib/libc.so.6(__libc_start_main+0xf4)[0x2b73d6d3ab44]
/usr/bin/python2.4[0x411b19]
======= Memory map: ========
00400000-004fd000 r-xp 00000000 03:02 120048 /usr/bin/python2.4
006fc000-00727000 rw-p 000fc000 03:02 120048 /usr/bin/python2.4
00727000-0089b000 rw-p 00727000 00:00 0 [heap]
2b73d625b000-2b73d6278000 r-xp 00000000 03:02 480979 /lib/ld-2.6.1.so
2b73d6278000-2b73d62bc000 rw-p 2b73d6278000 00:00 0
2b73d62bd000-2b73d62fe000 rw-p 2b73d62bd000 00:00 0
2b73d62fe000-2b73d633d000 r--p 00000000 03:02 176955 /usr/lib/locale/en_US.utf8/LC_CTYPE
2b73d633d000-2b73d6344000 r--s 00000000 03:02 116167 /usr/lib/gconv/gconv-modules.cache
2b73d6344000-2b73d6375000 rw-p 2b73d6344000 00:00 0
2b73d6477000-2b73d6479000 rw-p 0001c000 03:02 480979 /lib/ld-2.6.1.so
2b73d6479000-2b73d648f000 r-xp 00000000 03:02 481075 /lib/libpthread-2.6.1.so
2b73d648f000-2b73d668e000 ---p 00016000 03:02 481075 /lib/libpthread-2.6.1.so
2b73d668e000-2b73d6690000 rw-p 00015000 03:02 481075 /lib/libpthread-2.6.1.so
2b73d6690000-2b73d6694000 rw-p 2b73d6690000 00:00 0
2b73d6694000-2b73d6696000 r-xp 00000000 03:02 481018 /lib/libdl-2.6.1.so
2b73d6696000-2b73d6896000 ---p 00002000 03:02 481018 /lib/libdl-2.6.1.so
2b73d6896000-2b73d6898000 rw-p 00002000 03:02 481018 /lib/libdl-2.6.1.so
2b73d6898000-2b73d689a000 r-xp 00000000 03:02 481098 /lib/libutil-2.6.1.so
2b73d689a000-2b73d6a99000 ---p 00002000 03:02 481098 /lib/libutil-2.6.1.so
2b73d6a99000-2b73d6a9b000 rw-p 00001000 03:02 481098 /lib/libutil-2.6.1.so
2b73d6a9b000-2b73d6b1b000 r-xp 00000000 03:02 481034 /lib/libm-2.6.1.so
2b73d6b1b000-2b73d6d1a000 ---p 00080000 03:02 481034 /lib/libm-2.6.1.so
2b73d6d1a000-2b73d6d1c000 rw-p 0007f000 03:02 481034 /lib/libm-2.6.1.so
2b73d6d1c000-2b73d6d1d000 rw-p 2b73d6d1c000 00:00 0
2b73d6d1d000-2b73d6e6f000 r-xp 00000000 03:02 480999 /lib/libc-2.6.1.so
2b73d6e6f000-2b73d706e000 ---p 00152000 03:02 480999 /lib/libc-2.6.1.so
2b73d706e000-2b73d7071000 r--p 00151000 03:02 480999 /lib/libc-2.6.1.so
2b73d7071000-2b73d7073000 rw-p 00154000 03:02 480999 /lib/libc-2.6.1.so
2b73d7073000-2b73d7079000 rw-p 2b73d7073000 00:00 0
2b73d7079000-2b73d707d000 r-xp 00000000 03:02 249912 /usr/lib/python2.4/lib-dynload/readline.so
2b73d707d000-2b73d727d000 ---p 00004000 03:02 249912 /usr/lib/python2.4/lib-dynload/readline.so
2b73d727d000-2b73d727f000 rw-p 00004000 03:02 249912 /usr/lib/python2.4/lib-dynload/readline.so
2b73d727f000-2b73d7280000 rw-p 2b73d727f000 00:00 0
2b73d728a000-2b73d7
Program received signal SIGABRT, Aborted.
[Switching to Thread 47776528828128 (LWP 6626)]
0x00002b73d6d4e765 in raise () from /lib/libc.so.6
(gdb) bt
#0 0x00002b73d6d4e765 in raise () from /lib/libc.so.6
#1 0x00002b73d6d501c0 in abort () from /lib/libc.so.6
#2 0x00002b73d6d8760b in ?? () from /lib/libc.so.6
#3 0x00002b73d6d8eb0a in ?? () from /lib/libc.so.6
#4 0x00002b73d6d9073e in ?? () from /lib/libc.so.6
#5 0x00002b73d6d92979 in realloc () from /lib/libc.so.6
#6 0x00002b73d7c7615c in immbitset_realloc (self=0x864030,
size=<value optimized out>) at src/sets/bitset.c:986
#7 0x00002b73d7c761a2 in sf_realloc (v=0x2b73d84361f0, size=6626)
at src/sets/bitset.c:1030
#8 0x00002b73d7c7664a in mutbitset_findpos_ins (v=0x2b73d84361b0, pos=14392)
at src/sets/bitset.c:1071
#9 0x00002b73d7c76a8f in mutbitset_set_or_clr (v=0x2b73d84361b0,
bitno=<value optimized out>, set_or_clr=1) at src/sets/bitset.c:2261
#10 0x00002b73d7c7ac89 in NyNodeSet_setobj (v=0x2b73d82b27a0, obj=0x707000)
at src/sets/nodeset.c:640
#11 0x00002b73d7c7422f in bitfields_iterate (f=0x8837e8, end_f=0x8858c8,
visit=0x2b73d7c7b220 <mutnodeset_iterate_visit>, arg=0x7fffd484b3a0)
at src/sets/bitset.c:2331
#12 0x00002b73d7c75c73 in NyAnyBitSet_iterate (v=<value optimized out>,
visit=0x2b73d7c7b220 <mutnodeset_iterate_visit>, arg=0x7fffd484b3a0)
at src/sets/bitset.c:2350
#13 0x00002b73d7c7a8a3 in NyNodeSet_iterate (ns=0x2b73d7a0e998,
visit=0x2b73d7a594c0 <hv_update_static_types_visitor>, arg=0x2b73d84383d0)
at src/sets/nodeset.c:531
#14 0x00002b73d7a5bf64 in hv_heap (self=0x2b73d84383d0,
args=<value optimized out>, kwds=<value optimized out>)
at src/heapy/hv.c:859
#15 0x0000000000476c07 in PyEval_EvalFrame ()
#16 0x0000000000478276 in PyEval_EvalCodeEx ()
#17 0x00000000004764ba in PyEval_EvalFrame ()
#18 0x0000000000476b4e in PyEval_EvalFrame ()
#19 0x0000000000478276 in PyEval_EvalCodeEx ()
#20 0x0000000000478322 in PyEval_EvalCode ()
#21 0x000000000049c210 in PyRun_InteractiveOneFlags ()
#22 0x000000000049c314 in PyRun_InteractiveLoopFlags ()
#23 0x000000000049cd9a in PyRun_AnyFileExFlags ()
#24 0x000000000041263a in Py_Main ()
#25 0x00002b73d6d3ab44 in __libc_start_main () from /lib/libc.so.6
#26 0x0000000000411b19 in _start ()
sverker@sverker-laptop:~$

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Ragnar Kjřrstad - 2008-03-25

Logged In: YES
user_id=10550
Originator: NO

Actually I think 64bit python 2.4 is probably the most common python version in use. All new systems are 64 bit now, and the newest enterprise linux distributions (such as Red Hat and SuSE) are still shipping python 2.4 (or older).

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Sverker Nilsson - 2008-04-05

Logged In: YES
user_id=356603
Originator: NO

SVN Rev 13:

Now works/is tested in Ubuntu 7.10 on AMD64 in either 64 or 32 bits mode,
with Python 2.3.7, 2.4.4 and 2.5.1.

One can observe that with 64 bits the memory usage is significantly higher.

-- This is a memory usage test in 32 bits mode:

Python 2.5.1 (r251:54863, Oct 5 2007, 13:36:32)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from guppy import hpy
>>> hp=hpy()
>>> hp.heap()
Partition of a set of 20430 objects. Total size = 1384120 bytes.
Index Count % Size % Cumulative % Kind (class / dict of class)
0 9553 47 572008 41 572008 41 str
1 4884 24 177792 13 749800 54 tuple
2 1413 7 96084 7 845884 61 types.CodeType
3 62 0 91376 7 937260 68 dict of module
4 152 1 83456 6 1020716 74 dict of type
5 1342 7 75152 5 1095868 79 function
6 168 1 72448 5 1168316 84 type
7 119 1 68408 5 1236724 89 dict of class
8 65 0 49928 4 1286652 93 dict (no owner)
9 960 5 34560 2 1321212 95 __builtin__.wrapper_descriptor
<41 more rows. Type e.g. '_.more' to view.>

-- This is when the AMD64 is used in 64 bit mode.
-- The total size of objects is 2419480 vs 1384120 bytes or 75 % more.

Python 2.5.1 (r251:54863, Oct 5 2007, 13:50:07)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from guppy import hpy
>>> hp=hpy()
>>> hp.heap()
Partition of a set of 20448 objects. Total size = 2419480 bytes.
Index Count % Size % Cumulative % Kind (class / dict of class)
0 9571 47 747032 31 747032 31 str
1 4884 24 394656 16 1141688 47 tuple
2 62 0 183248 8 1324936 55 dict of module
3 1413 7 169560 7 1494496 62 types.CodeType
4 152 1 168128 7 1662624 69 dict of type
5 1342 7 161040 7 1823664 75 function
6 168 1 145240 6 1968904 81 type
7 119 1 137768 6 2106672 87 dict of class
8 65 0 100376 4 2207048 91 dict (no owner)
9 960 5 76800 3 2283848 94 __builtin__.wrapper_descriptor
<41 more rows. Type e.g. '_.more' to view.>

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Sverker Nilsson - 2008-04-16

Logged In: YES
user_id=356603
Originator: NO

I will close this item in the near future...

I have not heard any failure report.

Somebody tried it with 64 bit successfully? A word would help somewhat...

:-)

Sverker

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Cameron Dale - 2008-04-16

Logged In: YES
user_id=1436551
Originator: YES

I just tried 0.1.8 in Debian unstable on amd64 using both python 2.4 and 2.5 without any problems. Thanks for the fix.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Sverker Nilsson - 2008-05-13

status: open --> closed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Sverker Nilsson - 2008-05-14

status: closed --> closed-fixed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Doesn't work on 64-bit architectures

Group

Searches

Help

#2 Doesn't work on 64-bit architectures

Discussion