#1544 STAFProc SIGSEGV crash in STAFService::SubmitRequest

[Sharon Lucas]

This problem was reported by Rich Scheller (Austin) while running STAX tests on a Ubuntu 15.04 Linux ppc64le system using a retail version of STAF V3.4.21 Beta 1.

STAFProc core dump information from 03/31/2015:

10:21:23 AM: [Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/staf/bin/STAFProc '.
Program terminated with signal SIGABRT, Aborted.
#0  0x00003fff8d79ef94 in __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:55
55  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) where
#0  0x00003fff8d79ef94 in __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:55
#1  0x00003fff8d7a16b4 in __GI_abort () at abort.c:118
#2  0x000000001003aae8 in generic_signal_handler(int) ()
#3  <signal handler called>
#4  0x00003fff8da08524 in ?? () from /usr/lib/powerpc64le-linux-gnu/libstdc++.so.6
#5  0x00003fff8da0897c in std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) () from /usr/lib/powerpc64le-linux-gnu/libstdc++.so.6
#6  0x0000000010075f5c in std::_Rb_tree<unsigned long, unsigned long, std::_Identity<unsigned long>, std::less<unsigned long>, std::allocator<unsigned long> >::_M_insert_unique(unsigned long const&) ()
#7  0x0000000010073978 in STAFService::submitRequest(STAFServiceRequest const&) ()
#8  0x0000000010018dd4 in submitLocalRequest(STAFRefPtr<STAFConnection>&, STAFServiceRequest&) ()
#9  0x0000000010022670 in handleLocalServiceRequestAPI(unsigned int, STAFConnectionProvider const*, STAFRefPtr<STAFConnection>&, unsigned int&) ()
#10 0x000000001001442c in HandleRequest(STAFConnectionProvider const*, STAFRefPtr<STAFConnection>&) ()
#11 0x00003fff8dba377c in STAFConnectionProvider::handleNewConnection(STAFConnectionProviderImpl*, STAFConnectionImpl*, STAFConnectionProviderFunctionTable const*, void*) () from /usr/local/staf/lib/libSTAF.so
#12 0x00003fff8cf52fa8 in STAFLocalIPCConnectionThread(void*) () from /usr/local/staf/lib/libSTAFLIPC.so
#13 0x00003fff8db6e58c in STAFThreadManager::workerThread() () from /usr/local/staf/lib/libSTAF.so
#14 0x00003fff8db6ea38 in STAFThreadManager::callWorkerThread(void*) () from /usr/local/staf/lib/libSTAF.so
#15 0x00003fff8db4f1f4 in RealSTAFThread(void*) () from /usr/local/staf/lib/libSTAF.so
#16 0x00003fff8dae833c in start_thread (arg=0x3fff6f7ff180) at pthread_create.c:335
#17 0x00003fff8d88b014 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
(gdb)

[Sharon Lucas]

• On 04/07/2015, Rich Scheller sent me the following note:

"Sometime late last week, I updated my guests to the latest Ubuntu 15.04 packages (which included some system library updates) and STAF has apparently stopped crashing. (I say "apparently" because this has always reproduced in a random interval of time and we'll have to wait a while to say for sure.)

For the time being, I am running our stress tests under version "3.4.21 Beta 1" of STAF. This version previously crashed quite often on flyg7 and dotg1, but it has not crashed since the latest Ubuntu updates. I reinstalled both dotg1 and flyg7 from scratch this afternoon and I've restarted our tests on both. I'll let you know how it goes."

• On 04/08/2015, Rich Scheller sent me the following note:

"I just wanted to document for my team that STAF crashed again today, in two different ways on dotg1 and flyg7, running on the latest level of Ubuntu 15.04. Sharon suggested that we disable event generation in the STAX service and restart our tests, which I have done.

Unfortunately, I don't have the debug version of STAF that Sharon built last week since dotg1 was reinstalled yesterday. I have restarted our "standard" STAF version (currently 3.4.21 Beta 1) for now."

• On 04/29/2015, Rich Scheller responded as follows to a note I sent asking how STAF was running on his Linux ppc64le Ubuntu systems:

"Right up until the end of our 15.04 test runs, it continued to dump core sporadically. On some guests, it hardly ever did, while others never could make it through a 72 hour test run. We're between releases at the moment, but I'll let you know what happens when we start test runs again."

• On 06/11/2015, I sent Rich Scheller a note asking how STAF has running on his Linux ppcp64le Ubuntu systems but I got no reply.

[Sharon Lucas]

Closing this bug as I haven't received a response from Rich Scheller so I'm assuming this is no longer a problem.