#2899 Segfault when Autogenerated Vegetation is enabled

[datsf]

When I enable Autogenerated Vegetation in the render settings and fly the UFO, I get a segfault within a couple of minutes. I'm on Debian 12 with manually build latest 'next' branch. I have many backtraces I can share.

#0  0x00007fe5ac0a794a in __dynamic_cast () from /lib/x86_64-linux-gnu/libstdc++.so.6
[Current thread is 1 (Thread 0x7fe540d496c0 (LWP 3082))]
(gdb) bt
#0  0x00007fe5ac0a794a in __dynamic_cast () from /lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x0000562ab6079cfd in simgear::QuadTreeCleaner::apply (this=<optimized out>, lod=...)
    at /home/daniel/FGFS/src/simgear.git/simgear/scene/tgdb/TreeBin.cxx:354
#2  0x00007fe5b05c7b77 in osg::LOD::accept(osg::NodeVisitor&) () from /lib/x86_64-linux-gnu/libosg.so.161
#3  0x00007fe5b059a693 in osg::Group::traverse(osg::NodeVisitor&) () from /lib/x86_64-linux-gnu/libosg.so.161
#4  0x0000562ab5a67658 in osg::Group::accept (nv=..., this=0x7fe4d6afc370) at /usr/include/osg/Group:38
#5  osg::Group::accept (this=0x7fe4d6afc370, nv=...) at /usr/include/osg/Group:38
#6  0x00007fe5b059a693 in osg::Group::traverse(osg::NodeVisitor&) () from /lib/x86_64-linux-gnu/libosg.so.161
#7  0x0000562ab5a67658 in osg::Group::accept (nv=..., this=0x7fe50233a100) at /usr/include/osg/Group:38
#8  osg::Group::accept (this=0x7fe50233a100, nv=...) at /usr/include/osg/Group:38
#9  0x00007fe5b059a693 in osg::Group::traverse(osg::NodeVisitor&) () from /lib/x86_64-linux-gnu/libosg.so.161
#10 0x0000562ab60796c7 in osg::MatrixTransform::accept (nv=..., this=0x7fe4e13c7730) at /usr/include/osg/MatrixTransform:37
#11 simgear::createForest (forestList=..., transform=..., options=options@entry=0x7fe5041cace0, depth=depth@entry=3)
    at /home/daniel/FGFS/src/simgear.git/simgear/scene/tgdb/TreeBin.cxx:456
#12 0x0000562ab620484d in SGTileDetailsCallback::generateRandomTileObjects (this=this@entry=0x7fe4e40bd4e0, matTris=...,
    matcache=0x7fe5025dfba0) at /usr/include/osg/Matrixd:448
#13 0x0000562ab620609c in SGTileDetailsCallback::readNode (this=0x7fe4e40bd4e0)
    at /home/daniel/FGFS/src/simgear.git/simgear/scene/tgdb/SGTileDetailsCallback.hxx:135
#14 0x00007fe5b0e39421 in osgDB::Registry::readNode(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, osgDB::Options const*, bool) () from /lib/x86_64-linux-gnu/libosgDB.so.161
#15 0x00007fe5b0e3497e in osgDB::DatabasePager::DatabaseThread::run() () from /lib/x86_64-linux-gnu/libosgDB.so.161
#16 0x00007fe5b0c1a67e in OpenThreads::ThreadPrivateActions::StartThread(void*) ()
   from /lib/x86_64-linux-gnu/libOpenThreads.so.21
#17 0x00007fe5ac2a8134 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#18 0x00007fe5ac3287dc in ?? () from /lib/x86_64-linux-gnu/libc.so.6

[xDraconian]

I believe this is resolved and I'm unable to reproduce the issue. Please retest to confirm.

[datsf]

It's still happening. Before there were two different faults, one segfault and one abort. This time I triggered the abort. I didn't test yet if the segfault still happens.

Last lines in the terminal:

  110.99 [WARN]:osg        Error VertexBufferObject::removeBufferData(47771) out of range.
  110.99 [WARN]:osg        Error VertexBufferObject::removeBufferData(47712) out of range.
  110.99 [WARN]:osg        Error VertexBufferObject::removeBufferData(47703) out of range.
Running FlightGear's terminate handler. The program is going to exit due to a fatal error condition, sorry.
Aborted (core dumped)

Backtrace:

#0  0x00007f9da96a9e2c in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f9da965afb2 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007f9da9645472 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00005653fbd90de9 in fg_terminate () at /home/daniel/FGFS/src/flightgear.git/src/Main/bootstrap.cxx:246
#4  0x00007f9da94a8e1a in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007f9da94a8e85 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007f9da94a912b in __cxa_rethrow () from /lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00005653fbd613da in SGBinObject::read_bin (this=this@entry=0x7f9d27ffb8d0, file=...)
    at /home/daniel/FGFS/src/simgear.git/simgear/io/sg_binobj.cxx:797
#8  0x00005653fcd2140c in SGLoadBTG (path=..., options=options@entry=0x7f9d1c424590)
    at /home/daniel/FGFS/src/simgear.git/simgear/scene/tgdb/obj.cxx:52
#9  0x00005653fcd0b134 in SGReaderWriterBTG::readNode (this=<optimized out>, fileName=..., options=<optimized out>)
    at /home/daniel/FGFS/src/simgear.git/simgear/scene/tgdb/SGReaderWriterBTG.cxx:70
#10 0x00005653fbe4e47d in simgear::ModelRegistryCallback<simgear::DefaultProcessPolicy, simgear::NoCachePolicy, simgear::NoOptimizePolicy, simgear::NoSubstitutePolicy, simgear::BuildGroupBVHPolicy>::loadUsingReaderWriter (fileName=..., 
    opt=opt@entry=0x7f9d1c424590) at /usr/local/include/simgear/scene/model/ModelRegistry.hxx:126
#11 0x00005653fbe4f389 in simgear::ModelRegistryCallback<simgear::DefaultProcessPolicy, simgear::NoCachePolicy, simgear::NoOptimizePolicy, simgear::NoSubstitutePolicy, simgear::BuildGroupBVHPolicy>::readNode (this=0x5653ff70d1d0, fileName=..., 
    opt=0x7f9d1c424590) at /usr/local/include/simgear/scene/model/ModelRegistry.hxx:98
#12 0x00005653fcb40720 in simgear::ModelRegistry::readNode (this=0x5653ff69dc40, fileName=..., opt=0x7f9d1c424590)
    at /home/daniel/FGFS/src/simgear.git/simgear/scene/model/ModelRegistry.cxx:806
#13 0x00007f9dae21a4e7 in osgDB::Registry::readNode(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, osgDB::Options const*, bool) () from /lib/x86_64-linux-gnu/libosgDB.so.161
#14 0x00007f9dae249578 in osgDB::readRefNodeFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, osgDB::Options const*) () from /lib/x86_64-linux-gnu/libosgDB.so.161
#15 0x00005653fccf1565 in simgear::ReaderWriterSTG::_ModelBin::load (this=this@entry=0x7f9d27ffd170, bucket=..., 
    opt=opt@entry=0x7f9d1ec3d440) at /usr/include/osg/ref_ptr:117
#16 0x00005653fccebf8a in simgear::ReaderWriterSTG::readNode (this=<optimized out>, fileName=..., options=0x7f9d1ec3d440)
    at /home/daniel/FGFS/src/simgear.git/simgear/scene/tgdb/ReaderWriterSTG.cxx:1047
#17 0x00005653fcb82b7a in simgear::ModelRegistryCallback<simgear::DefaultProcessPolicy, simgear::NoCachePolicy, simgear::NoOptimizePolicy, simgear::NoSubstitutePolicy, simgear::BuildLeafBVHPolicy>::loadUsingReaderWriter (fileName=..., 
    opt=opt@entry=0x7f9d1ec3d440) at /home/daniel/FGFS/src/simgear.git/simgear/scene/model/ModelRegistry.hxx:126
#18 0x00005653fcb83039 in simgear::ModelRegistryCallback<simgear::DefaultProcessPolicy, simgear::NoCachePolicy, simgear::NoOptimizePolicy, simgear::NoSubstitutePolicy, simgear::BuildLeafBVHPolicy>::readNode (this=0x5653ff70a240, fileName=..., 
    opt=0x7f9d1ec3d440) at /home/daniel/FGFS/src/simgear.git/simgear/scene/model/ModelRegistry.hxx:98
#19 0x00005653fcb40720 in simgear::ModelRegistry::readNode (this=0x5653ff69dc40, fileName=..., opt=0x7f9d1ec3d440)
    at /home/daniel/FGFS/src/simgear.git/simgear/scene/model/ModelRegistry.cxx:806
#20 0x00007f9dae21a4e7 in osgDB::Registry::readNode(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, osgDB::Options const*, bool) () from /lib/x86_64-linux-gnu/libosgDB.so.161
#21 0x00007f9dae21597e in osgDB::DatabasePager::DatabaseThread::run() () from /lib/x86_64-linux-gnu/libosgDB.so.161
#22 0x00007f9dae0e067e in OpenThreads::ThreadPrivateActions::StartThread(void*) ()
--Type <RET> for more, q to quit, c to continue without paging--
   from /lib/x86_64-linux-gnu/libOpenThreads.so.21
#23 0x00007f9da96a8134 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#24 0x00007f9da97287dc in ?? () from /lib/x86_64-linux-gnu/libc.so.6

[datsf]

Problem confirmed by two more users (rooman, southside) in the FG forum:
https://forum.flightgear.org/viewtopic.php?f=45&p=422670#p422647

We are all on Debian/Ubuntu, if that matters.

[Huntley Palmer]

Also on Fedora 39, Ryzen 16GBy Nvidia 750Ti on a build fetched less than 5 days ago. Running with a debugger shows various endpoints on backtraces but often during object deleting. Running fgfs with Auto Generated Scenery with 'top' in a separate terminal shows fgfs memory consumption steadily increasing until swapping starts. When the swapfile is approximately half full the kswap0d daemon bcomes the dominant cpu load and fgfs stops with a segfault .
Departing westward from KGRR, Grand Rapids at low altitude demonstrates memory consumption increases over rural terrain; memory use does not decrease as the scenery changes to plain water, over the great lake.

[Huntley Palmer]

I'm running with no multithreading selected and when I've tried setting database-pager-threads to 1 : the sim runs slower but for much longer without error. Fgfs log shows many errors viz:
/fgfs/simgear/simgear/scene/tgdb/TreeBin.cxx:507: Error parsing tree entry in: Path "/fgScen/tsyn/Trees/w090n40/w086n45/w090n40_w086n45_1548792_SubUrban_trees_shader.txt" line: ""

Attached is a screen grab of the backtrace:

[Huntley Palmer]

After fetch & pull 26Jun with changes to treebin: