#2203 flightgear segfaults on simulation reset in generateRandomTileObjects()

[L29Ah]

I just went File -> Reset. 2019.1.1 here on Gentoo GNU/Linux. Backtrace:

#0  x86_64_fallback_frame_state (context=0x7fccdc7f5e30, context=0x7fccdc7f5e30, fs=0x7fccdc7f5f20) at ./md-unwind-support.h:63
#1  uw_frame_state_for (context=0x7fccdc7f5e30, fs=0x7fccdc7f5f20)
    at /var/tmp/portage/sys-devel/gcc-9.3.0/work/gcc-9.3.0/libgcc/unwind-dw2.c:1265
#2  0x00007fcd8d220259 in _Unwind_Backtrace (trace=0x7fcd8d15bc20 <backtrace_helper>, trace_argument=0x7fccdc7f60e0)
    at /var/tmp/portage/sys-devel/gcc-9.3.0/work/gcc-9.3.0/libgcc/unwind.inc:302
#3  0x00007fcd8d15bd98 in backtrace () from /lib64/libc.so.6
#4  0x000055b633e002c4 in segfault_handler(int) ()
#5  <signal handler called>
#6  0x00000000420de292 in ?? ()
#7  0x00007fcd8f854416 in osg::Group::insertChild(unsigned int, osg::Node*) () from /usr/lib64/libosg.so.160
#8  0x00007fcd90400f6c in SGTileDetailsCallback::generateRandomTileObjects(std::vector<SGTriangleInfo, std::allocator<SGTriangleInfo> >&, SGMaterialCache const*) () from /usr/lib64/libSimGearScene.so.2019.1.1
#9  0x00007fcd90401d01 in SGTileDetailsCallback::readNode(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, osgDB::Options const*) () from /usr/lib64/libSimGearScene.so.2019.1.1
#10 0x00007fcd8ff26e70 in osgDB::DatabasePager::DatabaseThread::run() () from /usr/lib64/libosgDB.so.160
#11 0x00007fcd8f69c3a5 in OpenThreads::ThreadPrivateActions::StartThread(void*) () from /usr/lib64/libOpenThreads.so.21
#12 0x00007fcd8f53904a in start_thread () from /lib64/libpthread.so.0
#13 0x00007fcd8d14ea2f in clone () from /lib64/libc.so.6

[L29Ah]

Got a different crash on another reset:

#0  0x00007f3e4a26a8db in raise () from /lib64/libc.so.6
#1  0x00007f3e4a25455d in abort () from /lib64/libc.so.6
#2  0x0000561f56520367 in segfault_handler(int) [clone .cold] ()
#3  <signal handler called>
#4  0x00007f3e4c5c0f1e in mark () from /usr/lib64/libSimGearCore.so.2019.1.1
#5  0x00007f3e4c5c150e in bottleneck () from /usr/lib64/libSimGearCore.so.2019.1.1
#6  0x00007f3e4c5c1813 in naGC_get () from /usr/lib64/libSimGearCore.so.2019.1.1
#7  0x00007f3e4c5c7cf5 in naNew () from /usr/lib64/libSimGearCore.so.2019.1.1
#8  0x00007f3e4c5c7d1b in naNewString () from /usr/lib64/libSimGearCore.so.2019.1.1
#9  0x00007f3e4c5bef50 in findConstantIndex () from /usr/lib64/libSimGearCore.so.2019.1.1
#10 0x00007f3e4c5c0ddd in genArgList () from /usr/lib64/libSimGearCore.so.2019.1.1
#11 0x00007f3e4c5bec37 in naCodeGen () from /usr/lib64/libSimGearCore.so.2019.1.1
#12 0x00007f3e4c5beef8 in newLambda () from /usr/lib64/libSimGearCore.so.2019.1.1
#13 0x00007f3e4c5bf45b in genExpr.part.5 () from /usr/lib64/libSimGearCore.so.2019.1.1
#14 0x00007f3e4c5c0218 in genHashElem () from /usr/lib64/libSimGearCore.so.2019.1.1
#15 0x00007f3e4c5c02b3 in genHash () from /usr/lib64/libSimGearCore.so.2019.1.1
#16 0x00007f3e4c5bfcb9 in genExpr.part.5 () from /usr/lib64/libSimGearCore.so.2019.1.1
#17 0x00007f3e4c5bfd4b in genExpr.part.5 () from /usr/lib64/libSimGearCore.so.2019.1.1
#18 0x00007f3e4c5c032c in genExprList () from /usr/lib64/libSimGearCore.so.2019.1.1
#19 0x00007f3e4c5bfbec in genExpr.part.5 () from /usr/lib64/libSimGearCore.so.2019.1.1
#20 0x00007f3e4c5beb74 in naCodeGen () from /usr/lib64/libSimGearCore.so.2019.1.1
#21 0x00007f3e4c5c95ae in naParseCode () from /usr/lib64/libSimGearCore.so.2019.1.1
#22 0x00007f3e4c5c5e70 in f_compile () from /usr/lib64/libSimGearCore.so.2019.1.1
#23 0x00007f3e4c5bb8e3 in setupFuncall () from /usr/lib64/libSimGearCore.so.2019.1.1
#24 0x00007f3e4c5bce9f in run () from /usr/lib64/libSimGearCore.so.2019.1.1
#25 0x00007f3e4c5be450 in naCall () from /usr/lib64/libSimGearCore.so.2019.1.1
#26 0x00007f3e4c5c59fd in f_call () from /usr/lib64/libSimGearCore.so.2019.1.1
#27 0x00007f3e4c5bb8e3 in setupFuncall () from /usr/lib64/libSimGearCore.so.2019.1.1
#28 0x00007f3e4c5bce9f in run () from /usr/lib64/libSimGearCore.so.2019.1.1
#29 0x00007f3e4c5be450 in naCall () from /usr/lib64/libSimGearCore.so.2019.1.1
#30 0x00007f3e4c5be6ed in naCallMethodCtx () from /usr/lib64/libSimGearCore.so.2019.1.1
#31 0x00007f3e4c5be7ae in naCallMethod () from /usr/lib64/libSimGearCore.so.2019.1.1
#32 0x0000561f56a31b73 in FGNasalSys::callMethod(naRef, naRef, int, naRef*, naRef) ()
#33 0x0000561f56a31ca1 in FGNasalListener::call(SGPropertyNode*, naRef) ()
#34 0x0000561f56a31ceb in FGNasalListener::valueChanged(SGPropertyNode*) ()
#35 0x00007f3e4c5db093 in SGPropertyNode::fireValueChanged(SGPropertyNode*) () from /usr/lib64/libSimGearCore.so.2019.1.1
#36 0x00007f3e4c5e100c in SGPropertyNode::set_bool(bool) () from /usr/lib64/libSimGearCore.so.2019.1.1
#37 0x00007f3e4c5ddd2d in SGPropertyNode::setBoolValue(bool) () from /usr/lib64/libSimGearCore.so.2019.1.1
#38 0x0000561f56a37ad9 in FGNasalSys::loadPropertyScripts(SGPropertyNode*) ()
#39 0x0000561f56a37e55 in FGNasalSys::loadPropertyScripts() ()
#40 0x0000561f56a399d1 in FGNasalSys::init() ()
#41 0x00007f3e4c61c6ce in SGSubsystemGroup::set_subsystem(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, SGSubsystem*, double) () from /usr/lib64/libSimGearCore.so.2019.1.1
#42 0x00007f3e4c620490 in SGSubsystemMgr::add(char const*, SGSubsystem*, SGSubsystemMgr::GroupType, double) () from /usr/lib64/libSimGearCore.so.2019.1.1
#43 0x0000561f56cfe8f9 in fgPostInitSubsystems() ()
#44 0x0000561f56d32b4f in fgIdleFunction() ()
#45 0x0000561f56cad599 in fgOSMainLoop() ()
#46 0x0000561f56d312af in fgMainInit(int, char**) ()
#47 0x0000561f565e4f4d in main ()

[James Turner]

Thanks for the report, there are unfortunartely a few different issues around reset. I see two basic kinds:

• OSG ref count is wrong (your first one, I think)
• Nasal Garbage collection ('GC') screws up.

Unfortunately both of these are hard to track down : at the point it screws u,p, the problem happened 'some time before' in some other code. I though I had fixed the Nasal ones a few months ago but evidently not :(

[Stuart Buchanan]

Is the scenery still loading when you attempt the simulation reset? I'm
not sure how that scenario is handled by the pageLOD system we use.

-Stuart

[James Turner]

ONe piece of good news : the Nasal fix was not in 2019.1.1 : can you test a development build? If not, 2020.1 will be out soon which includes the Nasal fix. Of course the OSG problem likely still exists unfortunatey.

[James Turner]

@stuartbuchanan scenery loaded is stopped during reset. We also tell osgDB to stop all its loader thread(s), and we wait for that to happen (i.e all pending ReaderWriter XYZ is done), before we continue with staritng things up again.

[Stuart Buchanan]

I've just raised a separate ticket
https://sourceforge.net/p/flightgear/codetickets/2204/, which I now realize
may be related. I'm just seeing this on the carrier though.

-Stuart

[Stuart Buchanan]

What aircraft were you running, and was it JSBSim or YASim?

[L29Ah]

GDT Hornet, YASim

[James Turner]

Please re-test on next with commits 297e5095762be2fc0fe7995f7115e2e58edb09ae (SimGear) and 06e6883396fef04caf8e6c54d1874de2b5581644 (flightgear) - will hopefully make a difference.