#659 segmentation fault when unsubscribing from events

[Zbigniew Reszela]

I have found that some PyTango client applications, listening to Tango events, terminate with segmentation fault.

This crashes can be reproduced by repeating the process of subscribing for a relatively short period of time and unsubsribing.

A typical backtrace of this segmentation fault is as following:

Program terminated with signal 11, Segmentation fault.
#0  0x00007f81d22857ac in Tango::ZmqEventConsumer::push_zmq_event
(this=0x7f81c401f350, ev_name=...,
    endian=0 '\000', event_data=..., error=false, ds_ctr=@0x7f8100000001)
    at ../../../../tango.src/lib/cpp/client/zmqeventconsumer.cpp:2333
2333
callback->push_event(event_data);
(gdb) bt
#0  0x00007f81d22857ac in Tango::ZmqEventConsumer::push_zmq_event
(this=0x7f81c401f350, ev_name=...,
    endian=0 '\000', event_data=..., error=false, ds_ctr=@0x7f8100000001)
    at ../../../../tango.src/lib/cpp/client/zmqeventconsumer.cpp:2333
#1  0x00007f81d22881ea in Tango::ZmqEventConsumer::process_event
(this=0x6e2ce0,
    received_event_name=<optimized out>, received_endian=<optimized
out>, received_call=..., event_data=...)
    at ../../../../tango.src/lib/cpp/client/zmqeventconsumer.cpp:574
#2  0x00007f81d2288bfb in Tango::ZmqEventConsumer::run_undetached
(this=0x6e2ce0, arg=<optimized out>)
    at ../../../../tango.src/lib/cpp/client/zmqeventconsumer.cpp:305
#3  0x00007f81d0baf79e in omni_thread_wrapper () from
/homelocal/sicilia/lib/libomnithread.so.3
#4  0x00007f81d3a7af05 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f81d37be10d in clone () from /lib64/libc.so.6

However from time to time, a PyTango appears in the traces as well. See
the dump below:

Program terminated with signal 11, Segmentation fault.
#0  0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007fc43dd7b2a7 in ?? () from /usr/lib/x86_64-linux-
gnu/libboost_python-py27.so.1.54.0
#2  0x00000000004a83f2 in ?? ()
#3  0x00000000004907f8 in ?? ()
#4  0x00007fc43dd8c829 in
boost::python::detail::wrapper_base::get_override(char const*,
_typeobject*) const () from /usr/lib/x86_64-linux-gnu/libboost_python-
py27.so.1.54.0
#5  0x00007fc4403eb4f2 in
PyCallBackPushEvent::push_event(Tango::EventData*) () from
/usr/lib/python2.7/dist-packages/PyTango/_PyTango.so
#6  0x00007fc43fd5a869 in
Tango::ZmqEventConsumer::push_zmq_event(std::string&, unsigned char,
zmq::message_t&, bool, unsigned int const&) () from /usr/lib/x86_64-
linux-gnu/libtango.so.8
#7  0x00007fc43fd5d954 in
Tango::ZmqEventConsumer::process_event(zmq::message_t&, zmq::message_t&,
zmq::message_t&, zmq::message_t&) () from /usr/lib/x86_64-linux-
gnu/libtango.so.8
#8  0x00007fc43fd5ea3e in Tango::ZmqEventConsumer::run_undetached(void*)
() from /usr/lib/x86_64-linux-gnu/libtango.so.8
#9  0x00007fc43dfa675e in omni_thread_wrapper () from
/usr/lib/libomnithread.so.3
#10 0x00007fc441725062 in start_thread (arg=0x7fc43884b700) at
pthread_create.c:312
#11 0x00007fc440b37a3d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:111

I have prepared a naive script repeating subscribe/wait/unsubscribe infinitely. With a very demanding configuration of events one could reproduce the segmentation fault waiting ~ 1 minute. (by demanding I mean 10Hz frequency of event generation and 1 second of
subscription period).

The script is called test_python_cb:
usage: test_python_cb <dev_name> <attr_name> <event_type> <subscription_time>

I have also tested it with a similar c++ client program (test_cb.cpp) but here I was not able to reproduce the segmentation fault within several hours.

I repeated this tests both on our own compilation of Tango and PyTango, and on debian testing environment (thanks to Carlos Pascual pc): see log.txt. In both cases results were the same.
In case of debian testing the following versions were used:

• libtango8:amd64 8.1.2c+dfsg-4
• python-pytango 8.1.1-1

One can test it with TangoTest DS using an attribute configured to push periodic events with ~10 Hz frequency. The attached DummyEventGenerator.py DS could be used as well (after calling Start command with event period as an input argument).

[Zbigniew Reszela]

Previously posted on mailing list: link

[Tiago Coutinho]

Fixed in PyTango SVN trunk since revision #25919.
Will become visible in next PyTango release 8.1.3