Menu
▾
▴
#15 Postprocessing / JIT / crash on large meshes
The following (very) high resolution .ply file crashes r754 in JoinVerticesProcess::ProcessMesh at line 128:
http://www-graphics.stanford.edu/data/3Dscanrep/xyzrgb/xyzrgb_dragon.ply.gz
What fails is the following sequence:
std::vector<Vertex> uniqueVertices;
uniqueVertices.reserve( pMesh->mNumVertices);
mNumVertices is over 21 million in this model; even on a 4GB machine with plenty of free RAM this is susceptible of failing.
Import information is provided below; please note that requesting the generation of vertex instead of face normals results in Assimp hanging (both Windows and Mac OS X), notwithstanding the available RAM at the time of import.
Import options:
aiProcess_RemoveComponent
aiProcess_LimitBoneWeights
aiProcess_TransformUVCoords
aiProcess_Triangulate
aiProcess_JoinIdenticalVertices
aiProcess_RemoveRedundantMaterials
aiProcess_SortByPType
aiProcess_GenUVCoords
aiProcess_GenNormals // requesting smooth normals hangs the system
aiProcess_FindInvalidData
aiProcess_ValidateDataStructure
aiProcess_FindDegenerates
aiProcess_OptimizeMeshes
Import settings:
ms_pkImporter->SetPropertyFloat(AI_CONFIG_PP_GSN_MAX_SMOOTHING_ANGLE,
45.0f);
ms_pkImporter->SetPropertyInteger(AI_CONFIG_PP_FD_REMOVE,1);
const int iRemoveComponents =
aiComponent_COLORS | aiComponent_TANGENTS_AND_BITANGENTS;
ms_pkImporter->SetPropertyInteger(AI_CONFIG_PP_RVC_FLAGS,
iRemoveComponents);
This occurs with any model, not just PLY (obviously, since JoinVerticesProcess is format-agnostic). In fact I encountered this exact error last night with a 1.6 million vertex mesh in a COLLADA model.
The reason it breaks even though you have enough RAM would be due to fragmentation. The solution would be for Assimp to use a custom allocator for the vector (Boost has some). I tried using Boost's pool_allocator (ie std::vector<Vertex, Boost::pool_allocator<Vertex> > uniqueVertices; (or something like that)), but I got compile time errors, and I'm not very good with templates or allocators, nor familiar with Boost, so I gave up very quickly :) Also contributing to my giving up is that I realised I didn't need/want to do JoinIdenticalVertices for that model :P
The solution seems to be to let JoinVertices process large meshes in small batches (depending on the available memory, but 100kverts could be a reasonable limit for most real world scenarios).
Three problems:
-- JoinVertices guarantees having no duplicate vertices after it is done. This will be much more difficult to achieve.
-- making the limit a dynamic one violates the idea of maintaining stable output across all platforms and systems.
-- choosing a fixed limit is not really a solution - what happens if we're really on a low-low-low memory machine? Apart from that, most Assimp loaders/postprocessing steps don't care of memory consumption and allocate what they need.
(Similar restrictions apply to the other memory-intensive postprocessing steps).
Bye, Alex
Btw: it shouldn't crash -- Assimp catches std::bad_alloc& and returns failure (it can crash, however, if another exception kicks in during stack unwinding. This would be a bug in the respective loader then).
(Splitting this issue into two).
https://sourceforge.net/tracker/?func=detail&aid=3081314&group_id=226462&atid=1067632
Yeah, there's no "crash", for me at least. Assimp does indeed catch the bad alloc.
And yeah, the splitting solution would be pretty messy for the reasons you said, but it would be nice to have given the temporary memory penalty as it stands.
And if this method is going to be overhauled, then it's a good time to implement multithreading ;)
When I initially reported this issue, it crashed for me in a noboost target -- it might very well work fine with Boost, haven't tried it since I don't use it.
I am going to implement it the aforementioned ideas into the JoinIdenticalVertices-step - I'll report back. Could take a few days (you know, the RL thing ...).
Bye, Alex
I think this needs further consideration, delaying post 3.0.