Menu
▾
▴
#4 Segfault (Possible race condition)
When I run a sample program (source in attached archive), lsf_drmaa spawns two threads and both access (through LSF) the lsf.conf file at the same time. Here's the backtrace from the two threads:
(gdb) thread 1
[Switching to thread 1 (process 20020)]#0 0x00002af5da1b31b2 in getNextLineD_ (fp=0x181a3c70, LineCount=0x40a31e5c, confFormat=1) at ../lib.words.c:785
785 ../lib.words.c: No such file or directory.
in ../lib.words.c
(gdb) bt
#0 0x00002af5da1b31b2 in getNextLineD_ (fp=0x181a3c70, LineCount=0x40a31e5c, confFormat=1) at ../lib.words.c:785
#1 0x00002af5da1b35de in getNextLineC_ (fp=0x181a3c70, LineCount=0x40a31e5c, confFormat=1) at ../lib.words.c:966
#2 0x00002af5da1b3632 in getNextLineC_ (fp=0x181a3c70, LineCount=0x40a31e5c, confFormat=1) at ../lib.words.c:978
#3 0x00002af5da1b3632 in getNextLineC_ (fp=0x181a3c70, LineCount=0x40a31e5c, confFormat=1) at ../lib.words.c:978
#4 0x00002af5da192e15 in readfileconfenv (pList1=0x2af5da2e3ce0, pList2=0x0, confPath=0x40a32100 "/wv/wvlsf/lsf/conf/lsf.conf", filter=0x0)
at ../lib.initenv.c:1270
#5 0x00002af5da1932fa in readconfenv_ (pList1=0x2af5da2e3ce0, pList2=0x0, confPath=0x0) at ../lib.initenv.c:1509
#6 0x00002af5da1920c6 in initenv_ (userEnv=0x0, pathname=0x0) at ../lib.initenv.c:645
#7 0x00002af5da18f959 in getname_ (limReqCode=LIM_GET_MASTINFO, name=0x2af5da38c700 "", namesize=64) at ../lib.info.c:463
#8 0x00002af5da18f911 in ls_getmastername () at ../lib.info.c:442
#9 0x00002af5da4d964b in getMasterName () at lsb.comm.c:1221
#10 0x00002af5da4d8d2c in callmbd (remoteCluster=0x0, request_buf=0x181a39a0 "", requestlen=68, reply_buf=0x40a34e28, replyHdr=0x40a34df0,
serverSock=0x2af5da6a52e0, postSndFunc=0, postSndFuncArg=0x0) at lsb.comm.c:922
#11 0x00002af5da4d8c9d in callmaster (remoteCluster=0x0, request_buf=0x181a39a0 "", requestlen=68, reply_buf=0x40a34e28, replyHdr=0x40a34df0,
serverSock=0x2af5da6a52e0, postSndFunc=0, postSndFuncArg=0x0) at lsb.comm.c:870
#12 0x00002af5da44cd38 in lsb_openjobinfo_req (req=0x40a34ec0) at lsb.jobs.c:929
#13 0x00002af5da44b9c6 in lsb_openjobinfo_a_ext (jobId=0, jobName=0x0, userName=0x0, queueName=0x0, hostName=0x0, options=1) at lsb.jobs.c:608
#14 0x00002af5da44b936 in lsb_openjobinfo_a (jobId=0, jobName=0x0, userName=0x0, queueName=0x0, hostName=0x0, options=1) at lsb.jobs.c:583
#15 0x00002af5da449821 in lsb_openjobinfo (jobId=0, jobName=0x0, userName=0x0, queueName=0x0, hostName=0x0, options=1) at lsb.jobs.c:51
#16 0x00002af5da00aa2c in lsfdrmaa_session_update_all_jobs_status (self=0x18190360) at session.c:269
#17 0x00002af5da0184c9 in fsd_drmaa_session_wait_thread (self=0x18190360) at fsd_session.c:735
#18 0x00000031bac064a7 in start_thread () from /lib64/libpthread.so.0
#19 0x00000031ba0d3c2d in clone () from /lib64/libc.so.6
(gdb) thread 2
[Switching to thread 2 (process 20019)]#0 setConfEnvOv (name=0x2af5da3b9ea0 "LSF_LOG_MASK", value=0x2af5da3baea0 "LOG_WARNING", paramList=0x2af5da2e4100,
overwrite=1) at ../lib.initenv.c:2097
2097 ../lib.initenv.c: No such file or directory.
in ../lib.initenv.c
(gdb) bt
#0 setConfEnvOv (name=0x2af5da3b9ea0 "LSF_LOG_MASK", value=0x2af5da3baea0 "LOG_WARNING", paramList=0x2af5da2e4100, overwrite=1) at ../lib.initenv.c:2097
#1 0x00002af5da193b99 in setConfEnv (name=0x2af5da3b9ea0 "LSF_LOG_MASK", value=0x2af5da3baea0 "LOG_WARNING", paramList=0x2af5da2e3ce0) at ../lib.initenv.c:2069
#2 0x00002af5da192ec6 in readfileconfenv (pList1=0x2af5da2e3ce0, pList2=0x2af5da6a5020, confPath=0x7fff88864ca0 "/wv/wvlsf/lsf/conf/lsf.conf", filter=0x0)
at ../lib.initenv.c:1288
#3 0x00002af5da1932fa in readconfenv_ (pList1=0x2af5da2e3ce0, pList2=0x2af5da6a5020, confPath=0x0) at ../lib.initenv.c:1509
#4 0x00002af5da1920c6 in initenv_ (userEnv=0x2af5da6a5020, pathname=0x0) at ../lib.initenv.c:645
#5 0x00002af5da44935c in lsb_init (appName=0x2af5da0207f5 "lsf_drmaa") at lsb.init.c:114
#6 0x00002af5da00a112 in lsfdrmaa_session_new (contact=0x0) at session.c:71
#7 0x00002af5da005cf5 in lsfdrmaa_new_session (self=0x2af5da127e00, contact=0x0) at drmaa.c:33
#8 0x00002af5da01c44c in drmaa_init (contact=0x0, error_diagnosis=0x7fff88866560 "h\006@", error_diag_len=1024) at drmaa_base.c:92
#9 0x0000000000400b12 in main (argc=1, argv=0x7fff88866a78) at main.cxx:48
Notice that both threads called readfileconfenv(). It appears to me as though LSF's "line" global is being manipulated by both threads (but I could be wrong). The attached tar file also includes a core dump.
Thanks,
Peter
Sample program and core dump