gamedevlists-windows

Re: [GD-Windows] new/delete across DLL's

[Rich <leg...@xm...>]

In article <E18...@sc...>,
    Ben Hawes <cr...@ca...>  writes:

> in the game exe, I call wibble =3D new CFooBar;
> Later on, within the exe, I call delete wibble; and get an Assert=
>  warning because I'm deallocating from the wrong heap.

Yep.  That's the way Windows DLLs work.  Allocations inside a DLL are
done on the heap belonging to the DLL (because the DLL can live in
memory longer than your process, it must use its own heap for
allocations).

You can deal with it in one of several ways:
- use a static library, not a DLL
- use a helper function in the DLL that does the deallocation for you

I'd use a static library unless there really is some overriding
reason to use a DLL.  "It just works" that way.
-- 
"The Direct3D Graphics Pipeline"-- code samples, sample chapter, FAQ:
          <http://www.xmission.com/~legalize/book/>
        izfree: Open source tools for Windows Installer
                <http://izfree.sourceforge.net>

RE: [GD-Windows] new/delete across DLL's

[Grills, J. <jg...@so...>]

Assuming C++, shouldn't a virtual destructor be sufficient to solve this
problem?  At the very most, you might have to have the virtual destructor
and overload new and delete per class, but those could trivially turn around
and call the global versions.

I load our graphics interface from a DLL, and it does allocate memory that
is released from the client.  I've hooked the memory management functions in
the DLL to call back into the main game executable so that I really only
have one heap.  My intent wasn't to work around this issue, but rather to
make sure that I had all memory in a single heap so that I could monitor the
game's memory usage and fragmentation.

j

-----Original Message-----
From: Rich [mailto:leg...@xm...]
Sent: Monday, March 24, 2003 12:08 PM
To: gam...@li...
Subject: Re: [GD-Windows] new/delete across DLL's 



In article <E18...@sc...>,
    Ben Hawes <cr...@ca...>  writes:

> in the game exe, I call wibble =3D new CFooBar;
> Later on, within the exe, I call delete wibble; and get an Assert=
>  warning because I'm deallocating from the wrong heap.

Yep.  That's the way Windows DLLs work.  Allocations inside a DLL are
done on the heap belonging to the DLL (because the DLL can live in
memory longer than your process, it must use its own heap for
allocations).

You can deal with it in one of several ways:
- use a static library, not a DLL
- use a helper function in the DLL that does the deallocation for you

I'd use a static library unless there really is some overriding
reason to use a DLL.  "It just works" that way.

RE: [GD-Windows] new/delete across DLL's

[Jon W. <hp...@mi...>]

Virtual destructor has nothing to do with memory allocation.

Virtual destructor and override of operator new and delete may=20
help a little bit, but you still get into trouble with templates=20
and other inline functions.

Repeat after me: Factories. Interfaces. Bliss. :-)

Un-balanced allocations/deletions really are a bad idea for a=20
variety of reasons; this being just one.

If you link your DLLs and your EXEs against the same runtime DLL=20
then they will share a heap, and things will appear to work,=20
until such time as someone wants to write a plug-in for your app=20
using Borland, or some other version of MSVC, or you want to run=20
a debug plug-in with the release EXE (debug and release MSVCRT=20
libs are not compatible).

And things get worse with MSVCPRT, which is needed for the MS=20
implementation of STL. It suffers all the problems of unbalanced=20
allocations already -- if you implement your own operator new,=20
then some template instances inside MSVCPRT will use the old=20
operator delete anyway, 'cause that's what was in effect when=20
that DLL was built.

Cheers,

			/ h+


> -----Original Message-----
> From: gam...@li...
> [mailto:gam...@li...]On Behalf Of
> Grills, Jeff
> Sent: Monday, March 24, 2003 10:49 AM
> To: 'gam...@li...'
> Subject: RE: [GD-Windows] new/delete across DLL's=20
>=20
>=20
>=20
> Assuming C++, shouldn't a virtual destructor be sufficient to solve =
this
> problem?  At the very most, you might have to have the virtual =
destructor
> and overload new and delete per class, but those could trivially=20
> turn around
> and call the global versions.
>=20
> I load our graphics interface from a DLL, and it does allocate memory =
that
> is released from the client.  I've hooked the memory management=20
> functions in
> the DLL to call back into the main game executable so that I really =
only
> have one heap.  My intent wasn't to work around this issue, but rather =
to
> make sure that I had all memory in a single heap so that I could=20
> monitor the
> game's memory usage and fragmentation.
>=20
> j
>=20
> -----Original Message-----
> From: Rich [mailto:leg...@xm...]
> Sent: Monday, March 24, 2003 12:08 PM
> To: gam...@li...
> Subject: Re: [GD-Windows] new/delete across DLL's=20
>=20
>=20
>=20
> In article <E18...@sc...>,
>     Ben Hawes <cr...@ca...>  writes:
>=20
> > in the game exe, I call wibble =3D3D new CFooBar;
> > Later on, within the exe, I call delete wibble; and get an Assert=3D
> >  warning because I'm deallocating from the wrong heap.
>=20
> Yep.  That's the way Windows DLLs work.  Allocations inside a DLL are
> done on the heap belonging to the DLL (because the DLL can live in
> memory longer than your process, it must use its own heap for
> allocations).
>=20
> You can deal with it in one of several ways:
> - use a static library, not a DLL
> - use a helper function in the DLL that does the deallocation for you
>=20
> I'd use a static library unless there really is some overriding
> reason to use a DLL.  "It just works" that way.
>=20
>=20
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Gamedevlists-windows mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-windows
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=3D555
>=20

RE: [GD-Windows] new/delete across DLL's

[Brian H. <bri...@py...>]

>Repeat after me: Factories. Interfaces. Bliss. :-)

Just to be clear, you're saying this is bad:

foo *f =3D Foo::newFoo();
delete f;

but that this is good:

foo *f =3D Foo::newFoo();
Foo::deleteFoo(f);

?

>Un-balanced allocations/deletions really are a bad idea for a
>variety of reasons; this being just one.

Well, a big part of me pretty much blames the language, but=
 that's a 
bit too convenient given that Microsoft's DLL idiosyncracies seem=
 to 
be rather unique in the world of operating systems (although you=
 do 
point out some interesting cases on Unix).

This is one reason why overloading global operator new seems like=
 a 
pipe dream in everything but the most specific cases (e.g. I can=
 get 
away with it because I don't use DLLs for anything on Windows --=
 but 
I can't get away with it at all using GCC for various reasons I=
 never 
bothered to figure out).

Brian

Re: [GD-Windows] new/delete across DLL's

[Pierre T. <p.t...@wa...>]

>Just to be clear, you're saying this is bad:
>
>foo *f = Foo::newFoo();
>delete f;
>
>but that this is good:
>
>foo *f = Foo::newFoo();
>Foo::deleteFoo(f);

Definitely.

Exactly what Javier said.

I ran into a ridiculously high amount of nasty bugs in Flexporter, because
of this issue. (MAX + MFC + my own engine in multiple DLLs, inlining all the
time, overring new/delete, mixing various versions of MSVCRT => blam ! blam!
more blam!)

Factories, factories, and some more factories. Don't even try fighting it,
you'll only regret it later ........ :)

Note that BoundsChecker usually detects the new/delete mismatch (and blame
you rightfully).

- Pierre
www.codercorner.com

RE: [GD-Windows] new/delete across DLL's

[Jon W. <hp...@mi...>]

> Just to be clear, you're saying this is bad:
>=20
> foo *f =3D Foo::newFoo();
> delete f;

Yes. Unless operator new/delete are overridden in Foo, but=20
evenso, there's other problems you'll run into.

> but that this is good:
>=20
> foo *f =3D Foo::newFoo();
> Foo::deleteFoo(f);

Yes.

I'd prefer something like:

Foo * f =3D gFooFactory->newFoo();
f->release();

> >Un-balanced allocations/deletions really are a bad idea for a
> >variety of reasons; this being just one.
>=20
> Well, a big part of me pretty much blames the language, but that's a=20
> bit too convenient given that Microsoft's DLL idiosyncracies seem to=20
> be rather unique in the world of operating systems (although you do=20
> point out some interesting cases on Unix).

Actually, AIX, MacOS and Windows all have the "sealed container"=20
linkage model, and most other UNIX-es have the "merged global symbol=20
table" model. I like "sealed container" because it makes it possible=20
for me to prove that I tested in the same environment that I will=20
run. (Don't know about MacOS X though -- maybe they went over to the=20
dark side with their new linkage model)

Cheers,

				/ h+

RE: [GD-Windows] new/delete across DLL's

[Brian H. <bri...@py...>]

>Foo * f =3D gFooFactory->newFoo();
>f->release();

is there a "delete this" somewhere in there?  And I assume the 
destructor and constructor are made private, etc.?

Brian

RE: [GD-Windows] new/delete across DLL's

[Jon W. <hp...@mi...>]

Absolutely.

The "delete this" is last in the release() code path, by=20
the way, not touching members after it :-)

Cheers,

				/ h+


> -----Original Message-----
> From: gam...@li...
> [mailto:gam...@li...]On Behalf Of
> Brian Hook
> Sent: Monday, March 24, 2003 12:18 PM
> To: gam...@li...
> Subject: RE: [GD-Windows] new/delete across DLL's
>=20
>=20
> >Foo * f =3D gFooFactory->newFoo();
> >f->release();
>=20
> is there a "delete this" somewhere in there?  And I assume the=20
> destructor and constructor are made private, etc.?
>=20
> Brian
>=20
>=20
>=20
>=20
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Gamedevlists-windows mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-windows
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_idU5
>=20

Re: [GD-Windows] new/delete across DLL's

[Rich <leg...@xm...>]

In article <HEE...@mi...>,
    "Jon Watte" <hp...@mi...>  writes:

> You may note that COM does a lot of these things, btw :-)

Oh, that's a very good point.  You can encapsulate your objects as COM
objects.  If you're careful in designing your interfaces, you can use
them from VB, C++, or a .NET language as well.
-- 
"The Direct3D Graphics Pipeline"-- code samples, sample chapter, FAQ:
          <http://www.xmission.com/~legalize/book/>
        izfree: Open source tools for Windows Installer
                <http://izfree.sourceforge.net>

Re: [GD-Windows] new/delete across DLL's

[Rich <leg...@xm...>]

In article <005b01c2f233$ede559c0$7b02000a@mcaf.local>,
    "Javier Arevalo" <ja...@py...>  writes:

> If both the EXE and your DLL link with the MSVC DLL runtime, it will work
> correctly because all allocations are physically performed inside the
> (single instance of the) runtime DLL.

I don't think this is true.  Its not a matter of what runtime you use,
its a matter of where you perform the allocation.  We used the DLL
runtime and had the exact same sort of issue reported by the original
poster.  Putting the deallocation inside the DLL via a helper function
fixed the problem.  However, the specifics are a fuzzy memory at this
point.  The one thing I remembered was that allocations performed by
DLL code are allocated from a distinct heap.  It has to be this way
for the memory dynamically allocated by a DLL to survive the death of
your process.
-- 
"The Direct3D Graphics Pipeline"-- code samples, sample chapter, FAQ:
          <http://www.xmission.com/~legalize/book/>
        izfree: Open source tools for Windows Installer
                <http://izfree.sourceforge.net>

Re: [GD-Windows] new/delete across DLL's

[Javier A. <ja...@py...>]

I never had to use this myself since we always have virtualized and/or
strong-ownership interfaces in our DLLs. IIRC the idea was like this: every
module has its own Windows heap. EXE and DLL files are distinct modules,
thus have different heaps. If you use the runtime in its DLL form, your
new/malloc calls (and delete/free) end up being processed by the runtime
DLL, thus only one heap is ever used for this purpose.

Rich wrote:
> In article <005b01c2f233$ede559c0$7b02000a@mcaf.local>,
>     "Javier Arevalo" <ja...@py...>  writes:
>
>> If both the EXE and your DLL link with the MSVC DLL runtime, it will
>> work correctly because all allocations are physically performed
>> inside the (single instance of the) runtime DLL.
>
> I don't think this is true.  Its not a matter of what runtime you use,
> its a matter of where you perform the allocation.  We used the DLL
> runtime and had the exact same sort of issue reported by the original
> poster.  Putting the deallocation inside the DLL via a helper function
> fixed the problem.  However, the specifics are a fuzzy memory at this
> point.  The one thing I remembered was that allocations performed by
> DLL code are allocated from a distinct heap.  It has to be this way
> for the memory dynamically allocated by a DLL to survive the death of
> your process.

  Javier Arevalo
  Pyro Studios

Re: [GD-Windows] new/delete across DLL's

[Ben H. <cr...@ca...>]

>...The one thing I remembered was that
>allocations performed by DLL code are allocated from a distinct
>heap.  It has to be this way for the memory dynamically=
 allocated by
>a DLL to survive the death of your process.

Now, if I was trying to deallocate memory I'd allocated in the=
 DLL, 
then I could understand it (I've already encountered that one).

But I'm deallocating memory (I thought) I'd allocated in the 
exe...just happens to be a class declared in the DLL...or does=
 that 
end up amounting to the same thing?

And, wrt Jon Watte's comments...I'm already heavily doing an 
Interface/Factory structure...this is one of my Factories I'm=
 trying 
to create within Game::Game(), and then deallocate in ~Game()

Or do I now need a FactoryFactory ?

[ cruise / casual-tempest.net / transference.org ]

RE: [GD-Windows] new/delete across DLL's

[Jon W. <hp...@mi...>]

> Or do I now need a FactoryFactory ?

Factories are singletons, right?
They can be static members of themselves, and live somewhere=20
in globals space, rather than being dynamically allocated.

Cheers,

				/ h+

Re: [GD-Windows] new/delete across DLL's

[brian s. <pud...@po...>]

Whoa, nellie! 

I think people need to be careful when they use the word "heap" in this 
discussion.  The heap that can't be shared is the statically linked C 
runtime heap - those data structures are allocated per DLL/EXE and 
memory allocated from one C runtime heap can't be freed by another.  
Imagine that you passed a pointer from a linked list into the remove 
routine of another linked list and you should get the picture.  
Confusion (at best) results.

The dynamically-linked C runtime solves some of these problems, assuming 
you can be sure that all your code uses the same version of the 
MSVCRT*.DLL - but as Jon points out there's more than one flavor of that 
DLL, so you'll probably end up in hell anyway once you alloc from one 
flavor and free from another.

A Windows heap doesn't have these issues.  The data structures for the 
Windows heap are stored once *per process* in the single instance of 
Kernel32.dll that's mapped in your process.  So you could implement 
operator new globally by just inlining it as GlobalAlloc or 
HeapAlloc(GetProcessHeap()...).  It might not be the most 
space-efficient allocator for small blocks, but it's perfectly 
functional...and why should you PC guys and gals care about running out 
of memory, you've got VM. :) 

Finally, DLLs (and memory allocated in them by your process) do NOT 
survive the death of your process.  How can they?  The DLL is a part of 
your process.  The code might still be mapped into memory if another 
process is using it, but when the process goes down, the data segment of 
the DLL that your process was using goes bye-bye.  Remember, the real 
problem with mixing allocations is because a separate C runtime heap is 
maintained in each module that statically links the C runtime.

--brian

p.s. If I wanted to avoid this problem, I would do exactly what Jon said.

Rich wrote:

>In article <005b01c2f233$ede559c0$7b02000a@mcaf.local>,
>    "Javier Arevalo" <ja...@py...>  writes:
>
>  
>
>>If both the EXE and your DLL link with the MSVC DLL runtime, it will work
>>correctly because all allocations are physically performed inside the
>>(single instance of the) runtime DLL.
>>    
>>
>
>I don't think this is true.  Its not a matter of what runtime you use,
>its a matter of where you perform the allocation.  We used the DLL
>runtime and had the exact same sort of issue reported by the original
>poster.  Putting the deallocation inside the DLL via a helper function
>fixed the problem.  However, the specifics are a fuzzy memory at this
>point.  The one thing I remembered was that allocations performed by
>DLL code are allocated from a distinct heap.  It has to be this way
>for the memory dynamically allocated by a DLL to survive the death of
>your process.
>  
>

Re: [GD-Windows] new/delete across DLL's

[Rich <leg...@xm...>]

In article <BD6...@ma...>,
    "Grills, Jeff" <jg...@so...>  writes:

> Assuming C++, shouldn't a virtual destructor be sufficient to solve this
> problem?

That's an interesting point.  It might work, but I'm not sure, I
haven't tried it.
-- 
"The Direct3D Graphics Pipeline"-- code samples, sample chapter, FAQ:
          <http://www.xmission.com/~legalize/book/>
        izfree: Open source tools for Windows Installer
                <http://izfree.sourceforge.net>

Re: [GD-Windows] new/delete across DLL's

[Rich <leg...@xm...>]

In article <3E7...@po...>,
    brian sharon <pud...@po...>  writes:

> Finally, DLLs (and memory allocated in them by your process) do NOT 
> survive the death of your process.  How can they?

OK, I misspoke slightly.  DLL code can survive your process, but data
allocated to your process by the DLL won't survive.  At any rate, the
factory approach is a good one if you're using DLLs.
-- 
"The Direct3D Graphics Pipeline"-- code samples, sample chapter, FAQ:
          <http://www.xmission.com/~legalize/book/>
        izfree: Open source tools for Windows Installer
                <http://izfree.sourceforge.net>

Re: [GD-Windows] new/delete across DLL's

[Javier A. <ja...@py...>]

If both the EXE and your DLL link with the MSVC DLL runtime, it will work
correctly because all allocations are physically performed inside the
(single instance of the) runtime DLL.

Rich wrote:
>
>> in the game exe, I call wibble =3D new CFooBar;
>> Later on, within the exe, I call delete wibble; and get an Assert=
>>  warning because I'm deallocating from the wrong heap.
>
> Yep.  That's the way Windows DLLs work.  Allocations inside a DLL are
> done on the heap belonging to the DLL (because the DLL can live in
> memory longer than your process, it must use its own heap for
> allocations).
>
> You can deal with it in one of several ways:
> - use a static library, not a DLL
> - use a helper function in the DLL that does the deallocation for you
>
> I'd use a static library unless there really is some overriding
> reason to use a DLL.  "It just works" that way.

  Javier Arevalo
  Pyro Studios