#279 CMemoryFile allocation issues

[Theo]

I ran into memory allocation issues with Lime and it appears mostly related to odd memory allocation/management strategies in Irrlicht base library. Specifically, expectation that an external library allocates and Irrlicht frees the memory which is arguably bad behavior. While there are workarounds it makes reference counting difficult if wrapper has to do the memory management for this file.

I've worked around my crashes by introducing a flag in FileSystem which will make a local buffer which can be freed when dropped and since both are from same allocator there are no issues rather than expecting calling library to allocate correctly which may not be possible. Patch is attached.

[Michael Zeilfelder]

I don't think it's needed. Irrlicht already has the option to drop or not drop the memory - which is all the engine needs to know. If you need to copy the memory first for some reason then just do that and pass the copied memory to Irrlicht.

[Theo]

I think perhaps you do not understand how I am interacting with your system. I am interacting indirectly to the engine loaded as a dll. I do not have access to your memory allocator for allocating/freeing generic memory to my knowledge.

So I have 3 choices:
1. Allocate memory locally and save with the filesystem object and free both at same time from myside (hard to do especially when reference counting is in play, this is what you seem to advocate).
2. Allocate memory from Irrlicht and allow you to free. (If you dont like this then provide an API for allocating and freeing generic memory that I can give to filesystem so that it can free when references are gone.)
3. fork your engine and patch it myself (what I'm currently doing but would like to avoid).

I think that you copying the memory as apart of the object creation is reasonable and less involved than creating a generic memory allocator/free system on another interface. If you want to do the latter then ok with me as well but seems less elegant.

[Michael Zeilfelder]

Ok, I think I start to understand. You can't allocate a general block of memory on the heap in your application at all? So you can't pass it to Irrlicht and let Irrlicht release it? Does Lime (or .NET I guess) not allow you allocating memory on the heap with some memcpy alike function? Sorry, I've never worked with .NET, so I didn't think of that - but I guess it might make sense with managed memory. I just expected those languages also have some way of passing memory to native libraries and let them release it. If that's not the case at all then I guess we will need this indeed.

Then again it might also be something which could be handled in the Lime wrapper maybe?

[Christian Stehno]

This memory handling probelm is basically always there, which is why we use irrAllocator in many places. Which would be a solution here. Otherwise only option 1 seems valid - unless we also have here the two boolean flags for 'Irrlicht frees memory' and 'Irrlicht copies data'. Could be that we use those flags only for images, though.