gamedevlists-general — List for discussing general game development issues

RE: [GD-General] Compile times

[Mick W. <mi...@ne...>]

> -----Original Message-----
> From: gam...@li... 
> [mailto:gam...@li...] On 
> Behalf Of Josiah Manson
> Sent: Tuesday, December 17, 2002 6:11 PM
> To: gam...@li...
> Subject: Re: [GD-General] Compile times
> 
> 
> Maybe I am totally off base here, but it seems to me that if 
> you really want to prevent fragmentation, for example in a 
> server that has a really long up time, the best thing to do 
> would be to use smart pointers of some sort.
> 
> By not having your pointers directly point to a memory 
> location, this allows you to change what the pointer points 
> to. It seems to me that having this additional abstraction 
> would allow you to periodically defragment memory, and the 
> programmer wouldn't have to worry about the details at all. 
> It just works.
>

Sounds good in theory, and I'm sure it has been tried many times.

Smart pointer are not free.  For it to to help with fragmentation you
have to apply it to EVERYTHING, which starts to add up.  There is the
additional memory required to store it (8-16+ bytes, depending on
implementation). Then there is the code bloat (in your executable size)
due to all the extra code generated for the double dereferencing, which
takes up memory.  Then there is the extra CPU time taken up by this
code, and the extra meory access (not cheap on the PS2, trashing the
cache and all that).

For that, you need to do a cost-benefit analysis, is it worth it?  What
do you gain?

And even though, in theory, you should simply be able to replace
"regular" pointers with smart pointers; in practice there will be
numerous problems related to this changeover.

Also, not every block of memory can be made movable.  DMA packets on the
PS2 can contain absolute addresses.  3rd party libraries like Renderware
might not be malleable enough to use your smart system. 

I'd like to hear from anyone who uses a fully smart pointer based memory
allocation scheme with movable blocks.  I don't think you can do it, and
still ship a game.

Mick

Re: [GD-General] Compile times

[Josiah M. <jm...@ma...>]

Maybe I am totally off base here, but it seems to me that if you really want
to prevent fragmentation, for example in a server that has a really long up
time, the best thing to do would be to use smart pointers of some sort.

By not having your pointers directly point to a memory location, this allows
you to change what the pointer points to. It seems to me that having this
additional abstraction would allow you to periodically defragment memory,
and the programmer wouldn't have to worry about the details at all. It just
works.

This is essentially what happens with your file system on the hard drive.
There is no easy way to know where exactly a file is on disk, and it could
change any time with no one being the wiser.

Josiah Manson

RE: [GD-General] Compile times

[Kyle W. <ky...@ga...>]

Brian Hook wrote:
> The problem I see is that a lot of coders today are trying very hard to
> hide the details because this OOP dogma of "encapsulation, look at
> interfaces only, DON'T THINK ABOUT THE IMPLEMENTATION!" has been shoved
> down everyone's throats so much that you almost feel guilty knowing
> whether an operation might be expensive or not.

Well, in all fairness I think that on a large team (e.g., Bioware's
Neverwinter Nights team, which had *22* programmers) it's just impossible to
keep track of all the details of other engineers implementations.  The teams
I've worked on have averaged 10-12 programmers and generally haven't had
what I consider great internal communication.  Under those circumstances,
separation of interface from implementation and avoidance of memory
ownership issues are Good Things, despite the costs.

Kyle

Re: Fragmentation was RE: [GD-General] Compile times

[<phi...@pl...>]

> So realistically speaking, you have a little less than 2GB.

Which is several orders of magnitude more than 32MB, or 2MB if you want to
drop back a year or two.

I mean, it's really easy to fragment an address space when you're using
~99% of it.

Cheers,
Phil

RE: Fragmentation was RE: [GD-General] Compile times

[Brian H. <bri...@py...>]

> Still, your 2GB of address space will hide fragmentation a lot better
> than the PS2s 32MB of address space.

Right, but as a matter of discipline (*cough*) I try to code avoiding
fragmentation because I never know where one of my, say, puzzle games
might be ported (handhelds aren't exactly super highpowered).  And on
the completely opposite end, I've worked on extremely big games where
datasts are routinely several hundred MB chunks.  If you're not paying
attention in these cases, you may find that you hit some pathological
combination of allocs/free that ends up fragmenting things so badly that
you're in trouble.  And debugging this can often be a huge pain in the
ass because if you have a memory tracker it will report a fairly
innocent situation (unless your memory tracker also tracks
fragmentation).

> On the PC, it does not matter so
> much if you loose 100k for fragmentation (or even leaks...) when
> changing levels.

True, you have to start working with VERY large data sets for this to
become a serious problem.  Alternatively, if you're dealing with a
persistent world, this is also a problem with much smaller memory
chunks.

If you write a small scale persistent game with a long enough up time,
you may find that even with no leaks you can't allocate the same stuff
you could when the game first started up.  That is a ROYAL bitch to
find.

> But on the PS2, you are going to notice this very
> quickly.  I've always felt that this area is one of the fundamental
> differences between PC programmers and Console programmers. 

And this is also an area (again) that I feel shows a flaw in C++.  When
programmers start adopting the mindset of "I don't care about memory, it
just works", the cost of finding out that it DOESN'T just work is
MONUMENTAL.  I've worked on projects like this, where 1GB+ datasets were
routinely being considered, along with month+ up times, but ZERO thought
was put into how to handle fragmentation.  Disaster waiting to happen.

Fragmentation is prevalent enough a problem that Objective-C supports
"zones" inherently.  And any long time programmers will have been bitten
by this enough that they probably firmly understand the concept of
managing multiple heaps/memory arenas to avoid fragmentation.  

Of course, you can overload operator new to "fix" this (for particle
systems, for example).  Until you need multiple arenas for a certain
object type.  Then, of course, you can work around that too.  But after
enough work arounds, I just go "You know, I spend more time working
around solutions than I do fixing the problems the solutions 'fix'".

Crap, when did I get so old that I feel like prefacing everything with
"Back in my day..."? =)

-Hook

RE: Fragmentation was RE: [GD-General] Compile times

[Mick W. <mi...@ne...>]

> -----Original Message-----
> From: gam...@li... 
> [mailto:gam...@li...] On 
> Behalf Of Brian Hook
> Sent: Tuesday, December 17, 2002 11:56 AM
> To: gam...@li...
> Subject: Fragmentation was RE: [GD-General] Compile times
> 
> 
> > Virtual memory hides a multitude of sins.
> > 
> > Fragmentation being the big one.
> 
> Unfortunately, no it doesn't.  Because the problem isn't 
> fragmentation of physical memory, it's fragmentation of 
> virtual addresses.
> 
> There's a common misconception that you get 32-bits of 
> address space on, say, Windows.  You don't.  You get 31-bits, 
> since half the address space is reserved for the OS.  Out of 
> this 2GB you have to share between heap, static data, and 
> code.  So realistically speaking, you have a little less than 2GB.
> 
> If you allocate and delete large chunks of memory, 
> interspersed with small ones, you will fragment that address 
> space such that you may have enough physical memory for an 
> operation, but you won't have enough contiguous address space 
> to allocate it.
> 

Still, your 2GB of address space will hide fragmentation a lot better
than the PS2s 32MB of address space.  On the PC, it does not matter so
much if you loose 100k for fragmentation (or even leaks...) when
changing levels.  But on the PS2, you are going to notice this very
quickly.  I've always felt that this area is one of the fundamental
differences between PC programmers and Console programmers. 

Mick.

RE: [GD-General] Compile times

[Brian H. <bri...@py...>]

> So really it's a way to automate some 
> of the discipline that Brian is advocating.

And just to be clear, it's not like I'm advocating that we all eschew
any form of compile time safeguards.  I'm not quite ready to ditch
function prototypes just yet.

I believe in doing as much at compile time as possible up until the
point where you start incurring greater costs than the benefit.  Where
that crossover occurs depends on the individual, which is why I don't
mean to imply that "STL always sucks", but more like "the benefits of
STL don't outweight the penalties for me".

Much like scripting languages, etc.  I guess it all wraps up
philosophically to me as "What practical advantages am I gaining for
theoretical benefits, and can I achieve these advantages in other ways?"

Brian

RE: [GD-General] Compile times

[Brian H. <bri...@py...>]

> Behalf Of Kyle Wilson
> 
> This general problem -- hiding of potentially expensive 
> operations -- is really a problem with C++ code in general, 
> not just with smart pointers.

Not to unfairly dog C++ -- because I've dogged it plenty in my life =)
-- but this is generally a problem between "very high level languages"
and "high level languages".  It's a philosophical difference between
"hide the details, concentrate on the overall" and "I need to know the
details".

The problem I see is that a lot of coders today are trying very hard to
hide the details because this OOP dogma of "encapsulation, look at
interfaces only, DON'T THINK ABOUT THE IMPLEMENTATION!" has been shoved
down everyone's throats so much that you almost feel guilty knowing
whether an operation might be expensive or not.

> After all, it's not immediately 
> obvious whether a variable going out of scope will disappear 
> off the stack or call an expensive destructor when a function 
> exits.  It's not obvious whether object destruction will free 
> resources or recursively call other destructors or spin-lock 
> waiting for another thread.  And the C++ emphasis on 
> interface over implementation makes it harder for users of a 
> class to keep up with the cost of creating/deleting/using 
> instances of that class.

Well stated.

Brian

Fragmentation was RE: [GD-General] Compile times

[Brian H. <bri...@py...>]

> Virtual memory hides a multitude of sins.
> 
> Fragmentation being the big one.

Unfortunately, no it doesn't.  Because the problem isn't fragmentation
of physical memory, it's fragmentation of virtual addresses.

There's a common misconception that you get 32-bits of address space on,
say, Windows.  You don't.  You get 31-bits, since half the address space
is reserved for the OS.  Out of this 2GB you have to share between heap,
static data, and code.  So realistically speaking, you have a little
less than 2GB.

If you allocate and delete large chunks of memory, interspersed with
small ones, you will fragment that address space such that you may have
enough physical memory for an operation, but you won't have enough
contiguous address space to allocate it.

Brian

Re: [GD-General] Compile times

[Thatcher U. <tu...@tu...>]

On Dec 17, 2002 at 10:21 -0800, Mick West wrote:
> > Behalf Of Thatcher Ulrich
> > 
> > On Dec 16, 2002 at 11:46 -0600, brian hook wrote:
> > > introducing a degree of non-determinism in my code (smart pointers),
> > 
> > Note: the behavior of ref-counted smart pointers is 
> > completely deterministic.  (That's one of the nice things 
> > about ref-counting, compared to other GC approaches.)
> 
> Well, all code is completely deterministic, if you want to get picky.

When you compare reference counting to mark/sweep GC or multithreaded
code, there's definitely a meaningful and practical distinction, even
if you're not picky.

Anyway, I personally think "smart pointers as an implementation of
handles" is pretty worthwhile and doesn't violate Brian's original
objection, which is that he doesn't know for sure when an object will
be freed.  I.e. you still have a manager somewhere that keeps a list
of smart pointers, so nothing can go away until the manager says so.
Then you put some checks in the manager to find and punish people who
hold references when they shouldn't be (e.g. at level-shutdown time).
So really it's a way to automate some of the discipline that Brian is
advocating.

-- 
Thatcher Ulrich
http://tulrich.com

Re: [GD-General] Compile times

[<phi...@pl...>]

>> And honestly, dynamic memory allocation is so rare (at least in my code
base, because I abhor fragmentation) that it's not like I run into these
problems multiple times a day.

> That's a big difference. I, at least, use dynamic allocation a lot and
I'd expect (naively?) that a lot of PC games do as well.

Virtual memory hides a multitude of sins.

Fragmentation being the big one.

Cheers,
Phil

RE: [GD-General] Compile times

[Paul B. <pa...@mi...>]

> -----Original Message-----
> From: Mick West [mailto:mi...@ne...]=20
> Subject: RE: [GD-General] Compile times
>=20
> So, while you could (maybe) say that the behavior of a=20
> ref-counted smart pointer is completely deterministic,=20
> I think it's a fair comment to say that replacing the=20
> usage of a regular pointer with a smart pointer will
> decrease the amount of determinism in a piece of code=20
> (Brian's code, not the smart pointer code), by making=20
> it reliant on the actions of other loosely coupled=20
> objects about which it knows nothing.

Right.  The one thing that I question is whether it is
"easier" overall (in a large codebase) to mentally manage
the "who-should-delete-this" problem or the similar=20
"is-this-release-going-to-delete-this" problem.

We use refcounted objects a lot (we also use passively
tracked pointers with explicit ownership for other things)
and if there is a place in the code where one *expects*
the release to delete the object we insert something like:

	ASSERT_LAST_REFERENCE(ref_counted_object_pointer);
	RELEASE(ref_counted_object_pointer);

Both of these are macros for the non-smart pointer case.
The smart pointer could support a similar method that could
be asserted.

The key here is that we have an compiled piece of code that
verifies an assumption (that the code in question in fact
the last remaining reference at this point).  It is difficult=20
to have a similar *compiled* artifact for verifying that I=20
should be the one deleting a pointer.  I think this has some=20
benefit.

Like I said, we use both refcounting and explicit lifetime
control (for different systems) as both have their benefits.
We try to maintain a consistent scheme within major systems
so people expend as little mental effort as possible when=20
writing or debugging code.

Paul

RE: [GD-General] Compile times

[Kyle W. <ky...@ga...>]

Mick West wrote:
> When you speak of the determinism of code, I understand that as a
> measure of the extent to which you can predict the behaviour of a piece
> of code.  For example "'delete p' will call the destructor of object *p"
> vs "'delete p' might call the destructor of the object *p"

This general problem -- hiding of potentially expensive operations -- is
really a problem with C++ code in general, not just with smart pointers.
After all, it's not immediately obvious whether a variable going out of
scope will disappear off the stack or call an expensive destructor when a
function exits.  It's not obvious whether object destruction will free
resources or recursively call other destructors or spin-lock waiting for
another thread.  And the C++ emphasis on interface over implementation makes
it harder for users of a class to keep up with the cost of
creating/deleting/using instances of that class.

The uncertainty is, of course, the price we pay for the higher level of
abstraction at which C++ allows us to operate.  I like C++, and I like smart
pointers, and I find them both quite useful for the applications on which I
work.  I'm working on complex games on PCs and newer consoles, though.  If I
were writing for the GBA, or PalmOS, or cell phone applications, I'd be a
lot more likely to give up smart pointers and classes in favor of the
tighter control of straight C.

Kyle

RE: [GD-General] Compile times

[Mick W. <mi...@ne...>]

> -----Original Message-----
> From: gam...@li... 
> [mailto:gam...@li...] On 
> Behalf Of Thatcher Ulrich
> Sent: Tuesday, December 17, 2002 9:32 AM
> To: gam...@li...
> Subject: Re: [GD-General] Compile times
> 
> 
> On Dec 16, 2002 at 11:46 -0600, brian hook wrote:
> > introducing a degree of non-determinism in my code (smart pointers),
> 
> Note: the behavior of ref-counted smart pointers is 
> completely deterministic.  (That's one of the nice things 
> about ref-counting, compared to other GC approaches.)
> 

Well, all code is completely deterministic, if you want to get picky.

I think what Brian was referring to, is that with re-counted smart
pointer, if you "delete" it, then you don't know if the object it points
to is going to be actually deleted.  So you are writing code where you
don't know what it is going to do.

Of course, you could actual determine what it is going to do, by adding
some more code to check the reference count.  But does this make your
code deterministic?  And is that even something you want the smart
pointer to supply an interface to?

I fear getting pedantic here.  But when you speak of determinism of a
thing, then you are referring to the lack of dependency on external
factors in contributing to the future state of an object. 

When you speak of the determinism of code, I understand that as a
measure of the extent to which you can predict the behaviour of a piece
of code.  For example "'delete p' will call the destructor of object *p"
vs "'delete p' might call the destructor of the object *p"   

So, while you could (maybe) say that the behavior of a ref-counted smart
pointer is completely deterministic, I think it's a fair comment to say
that replacing the usage of a regular pointer with a smart pointer will
decrease the amount of determinism in a piece of code (Brian's code, not
the smart pointer code), by making it reliant on the actions of other
loosely coupled objects about which it knows nothing.

Okay, I'll stop now.  Sorry.

Mick.

Re: [GD-General] Compile times

[Thatcher U. <tu...@tu...>]

On Dec 16, 2002 at 11:46 -0600, brian hook wrote:
> introducing a degree of non-determinism in my code (smart pointers), 

Note: the behavior of ref-counted smart pointers is completely
deterministic.  (That's one of the nice things about ref-counting,
compared to other GC approaches.)

-- 
Thatcher Ulrich
http://tulrich.com

Re: Smart pointers was RE: [GD-General] Compile times

[brian h. <bri...@py...>]

> So, you've actually used manual ref counting Brian?

Yes, and it kinda sucks.  I don't even ref-count in my code.  I just 
know when and where stuff is allocated or freed.  
 
> >Where is it difficult to rely on convention and discipline?
> 
> The biggest problem is that it doesn't scale well. It's yet another 
> thing for new coders to learn and another thing for busy people to 
> remember to do. All other things being equal it's much better to 
> design the system so that you don't have to rely on a convention.

I sympathize and philosophically agree with this point of view, but at 
the same time in my experience there are usually enough trade offs with 
going with the "automated" approach that it simply ends up not being 
worth it.

I strongly feel that relying on discipline and solid interfaces is the 
right way to go, but I do understand it doesn't scale well because as 
teams grow you start getting fewer disciplined programmers and more 
programmers intent on "getting stuff done" instead of *gasp* thinking 
about what they're doing before they do it.

And as you pointed out elsewhere, this also depends a lot on how much 
dynamic memory allocationn you do.  Since I avoid it like the plague to 
minimize memory related errors (counter-argument: if I used a proper 
smart pointer class or GC system I wouldn't have to be so paranoid 
about such errors), it's much less of an issue for me.

In fact, for all my code, nearly all dynamic memory allocation is done 
at startup, all freeing is done at shut down, and there is almost no 
run-time allocation at all.  I try to use the stack as much as possible 
for objects that don't have a long life time.  I end up restricting 
dynamic allocation to when I don't know the size or number of objects I 
need, or if I'm hiding a derived class implementation (i.e. I'm using 
base class pointers and factory methods).

-Hook

Re: Smart pointers was RE: [GD-General] Compile times

[Jesse J. <jes...@mi...>]

At 11:51 AM -0600 12/16/02, brian hook wrote:
>  > >- non-deterministic destruction (a problem with general GC as well)
>>
>>  How so? With a real GC objects aren't deleted until the collector
>>  gets around to freeing them, but with all the ref-counted smart
>>  pointer classes I've seen they are deleted when the last reference is
>>  released.
>
>Non-deterministic was a poor choice of phrase.  More
>like "unpredictable" because, by definition, if you're using smart
>pointers you've opted to stop thinking about de-allocation of memory. 
>So you're never quite sure when the "last reference" is released, since
>that can change as the code evolves.

Yep, but I haven't found this to be an issue.

>  > >- cascading destruction if you deref a root node
>>
>>  Why is this a problem?
>
>Because along with "unpredictable de-allocation", it can lead to
>cascades that may cause a hiccup in your system.  If you have
>heavyweight destructors, having a bunch of them called instantly
>because you did this:
>
>foo->pBar = NULL;
>
>is both non-obvious (if you haven't been able to tell, I also abhor
>code that looks straightforward but may actually do a bunch of non-
>obvious stuff) and annoying.

I will grant you non-obvious (especially if you haven't used a 
garbage collected language much), but it seems to me any hiccups will 
come about at pretty much the same place in the game whether or not 
you use reference counting.

>  > Well, what are the options? In my experience automating reference
>>  counting with smart pointers is a hell of a lot better than doing it
>>  manually.
>
>My experience has been the opposite.  My anecdote can kick your
>anecdotes ass! =)

So, you've actually used manual ref counting Brian? InDesign used ref 
counting like crazy and the newer code that used smart pointers was 
noticeably cleaner than the old code.

>  > Doing all the memory management manually with naked calls
>>  to new and delete is feasible with discipline and some programming
>>  conventions, but there are usually a few instances where this becomes
>>  difficult.
>
>Where is it difficult to rely on convention and discipline?

The biggest problem is that it doesn't scale well. It's yet another 
thing for new coders to learn and another thing for busy people to 
remember to do. All other things being equal it's much better to 
design the system so that you don't have to rely on a convention.

   -- Jesse

Re: [GD-General] Compile times

[Jesse J. <jes...@mi...>]

At 11:46 AM -0600 12/16/02, brian hook wrote:
>Every time you call "new" or "delete" or return a pointer or accept a
>pointer, it should set off alarm bells immediately.  It should mean
>that you need to stop and document what you're doing.  It should mean
>that you should take the time to make a Doxygen or similar style header
>that defines exactly what you're doing.  You should decide what
>preconditions, postconditions and invariants need to be defined and
>adhered to for your particular class or function.

Pretty much. And that's the problem with what you're suggesting: it 
requires careful analysis of ownership issues and accompanying 
documentation. Whereas a smart pointer side steps all of these issues 
and AFAICT has no real drawbacks.

>I hesitate to call my coding style "XP" since that's so overused, but
>basically one reason I'm drifting away from large C++ frameworks is
>that you almost never, ever get it right the first time.  Or the second
>time.  Or the third time.  The ability to abstract and refactor
>iteratively is immensely powerful, but C++ code bases have a tendency
>to build up resistance to fundamentaly change, making these changes
>difficult to do.  Part of this is the fault of tools, obviously.

Frameworks don't have to have deep hierarchies or a zillion classes. 
But they do require some experience and knowledge of OOD. I'm 
confident that a skilled OO designer would have an easier time 
iterating classes than an equivalent C programmer evolving C code.

>And honestly, dynamic memory allocation is so rare (at least in my code
>base, because I abhor fragmentation) that it's not like I run into
>these problems multiple times a day.

That's a big difference. I, at least, use dynamic allocation a lot 
and I'd expect (naively?) that a lot of PC games do as well.

   -- Jesse

Re: Smart pointers was RE: [GD-General] Compile times

[Jesse J. <jes...@mi...>]

At 11:42 AM -0500 12/16/02, Thatcher Ulrich wrote:
>On Dec 16, 2002 at 04:05 -0800, Jesse Jones wrote:
>>  At 8:16 PM -0600 12/15/02, brian hook wrote:
>>  >Assuming that we're defining "smart pointer" as ref-counted,
>>  >templatized pointer access, then here are the typical bugaboos:
>
>[snip]
>
>>  Well, what are the options?
>
>Brian and Ignacio mentioned handles.  It seems to me that handles are
>equivalent to smart pointers, with a few differences in details.  The
>similarities:
>
>* If your handle manager keeps a ref-count (or something similar) to
>   avoid deleting an object that has an active handle, then handles
>   essentially behave like ref-counted smart pointers.
>
>* If your handle manager is allowed to delete objects when it feels
>   like it, then handles are essentially the same as weak pointers,
>   with a strong pointer in the handle manager.

This seems right to me and handles seem like a good option if you 
don't have enough memory to hold all of your objects and you can 
re-construct them from a resource file or some sort of save file.

   -- Jesse

Re: [GD-General] Compile times

[Jesse J. <jes...@mi...>]

At 5:58 PM +0100 12/16/02, Ignacio Casta=F1o wrote:
>Jesse Jones wrote:
>>  I have some thoughts on how this can be done, but I'd like to hear
>>  some more details on how you've implemented this. :-)
>
>Well, that was mentioned some time ago in the Algorithm mailing list by Tom
>Forsyth, I think... However, I saw that for the first time in the fmod soun=
d
>library to handle channel references.

Thanks for the summary. How is this better than simply ref counting 
objects though?

>I don't think that handles are the solution to all the problems and that
>smart pointers are bad. However, I think that there are many ways of using
>smart pointers wrong, while the handle method is simple and errors are easy
>to catch.

Well, this is a separate issue: you don't have to use a smart pointer 
if you ref count your objects. It just makes the code clearer and 
eliminates a source of errors since the ref counting is automated.

Having said the smart pointers are normally very simple. As a rule 
they don't use any template meta-programming wackiness and they 
contain very little code (look at the boost smart pointer 
implementations for example). It's hard to go wrong if you have a 
solid understanding of how constructors and assignment operators work.

   -- Jesse

Re: Smart pointers was RE: [GD-General] Compile times

[Ivan-Assen I. <as...@ha...>]

> Where is it difficult to rely on convention and discipline?

In the real world.

We never leave to convention what we can enforce via the interface.

RE: Smart pointers was RE: [GD-General] Compile times

[Wayne C. <wc...@re...>]

> Non-deterministic was a poor choice of phrase.  More
> like "unpredictable" because, by definition, if you're using smart
> pointers you've opted to stop thinking about de-allocation of memory.
> So you're never quite sure when the "last reference" is released, since
> that can change as the code evolves.

I know this a GD list rather than an 'any app list' but do you have a
problem with smart pointer\ref counting per se. or just when it relates to
game development? :)

> > Doing all the memory management manually with naked calls
> > to new and delete is feasible with discipline and some programming
> > conventions, but there are usually a few instances where this becomes
> > difficult.
> 
> Where is it difficult to rely on convention and discipline?

COM or a similar object model provides one source where the creator may not
necessarily be the one who destroys a particular object. The object may be
passed into a completely different part of the system which wants it to
stick around. A game environment is quite a controlled environment, but I've
heard of a few game engines that use COM (or similar) for their object
model.

Wayne
-Virus scanned and cleared ok

Re: Smart pointers was RE: [GD-General] Compile times

[brian h. <bri...@py...>]

> >- non-deterministic destruction (a problem with general GC as well)
> 
> How so? With a real GC objects aren't deleted until the collector 
> gets around to freeing them, but with all the ref-counted smart 
> pointer classes I've seen they are deleted when the last reference is 
> released.

Non-deterministic was a poor choice of phrase.  More 
like "unpredictable" because, by definition, if you're using smart 
pointers you've opted to stop thinking about de-allocation of memory.  
So you're never quite sure when the "last reference" is released, since 
that can change as the code evolves.

> >- cascading destruction if you deref a root node
> 
> Why is this a problem?

Because along with "unpredictable de-allocation", it can lead to 
cascades that may cause a hiccup in your system.  If you have 
heavyweight destructors, having a bunch of them called instantly 
because you did this:

foo->pBar = NULL;

is both non-obvious (if you haven't been able to tell, I also abhor 
code that looks straightforward but may actually do a bunch of non-
obvious stuff) and annoying.
 
> Well, what are the options? In my experience automating reference 
> counting with smart pointers is a hell of a lot better than doing it 
> manually. 

My experience has been the opposite.  My anecdote can kick your 
anecdotes ass! =)

> Doing all the memory management manually with naked calls 
> to new and delete is feasible with discipline and some programming 
> conventions, but there are usually a few instances where this becomes 
> difficult.

Where is it difficult to rely on convention and discipline?

-Hook

Re: [GD-General] Compile times

[brian h. <bri...@py...>]

> You've said something similar to the "proper discipline" comment 
> several times now, and I think you've got a blind spot going on here. 

It's not a blind spot, it's called wishful idealism =)

> If you're coding by yourself, personal discipline is all you need. 
> But if you're coding on a team, you have to have discipline that 
> applies to everyone on the team.

Yes.

> And not everyone is a coding machine.

Don't have to be a machine, just have to be careful and thoughtful.

> Not everyone works on all the code all the time.

They don't have to.

Every time you call "new" or "delete" or return a pointer or accept a 
pointer, it should set off alarm bells immediately.  It should mean 
that you need to stop and document what you're doing.  It should mean 
that you should take the time to make a Doxygen or similar style header 
that defines exactly what you're doing.  You should decide what 
preconditions, postconditions and invariants need to be defined and 
adhered to for your particular class or function.

I don't think this is too much to ask.  No one is in such a hurry that 
they can't take a minute to think about and write down the assumptions 
about allocation, ownership and referencing for memory they're dealing 
with.

Now, don't get me wrong, my code isn't perfect in this regard, but I'm 
generally aware of where things are sloppy and when/how I need to fix 
them.  I write code in an iterative fashion, and I comment the hell out 
of it when I need to fix something later.  In fact, my code has 
probably gotten an order of magnitude better now that I'm going back to 
more ANSI C style of coding.

I hesitate to call my coding style "XP" since that's so overused, but 
basically one reason I'm drifting away from large C++ frameworks is 
that you almost never, ever get it right the first time.  Or the second 
time.  Or the third time.  The ability to abstract and refactor 
iteratively is immensely powerful, but C++ code bases have a tendency 
to build up resistance to fundamentaly change, making these changes 
difficult to do.  Part of this is the fault of tools, obviously.

> The old "He allocated it but *I* need to free it" is one example of 
> the problem.

But that's a trivial problem.  Ten years ago Taligent defined a set of 
naming conventions to indicate responsibility for creation, 
destruction, referencing, etc.

And honestly, dynamic memory allocation is so rare (at least in my code 
base, because I abhor fragmentation) that it's not like I run into 
these problems multiple times a day.
 
> And some of the stuff you've dissed lately -- wrapping allocations in 
> smart pointers, typesafe containers, and so forth -- are useful in 
> almost direct proportion to the number of people working on the code. 

Maybe vast team size is a mitigating factor, but I still can't help but 
feel that when this is used as an excuse it's because:

- there is poor communication

- the individuals that make up the team aren't as good as they should 
be because of the required size of the team

I don't believe in trying to solve the above problems (poorly) by 
introducing a degree of non-determinism in my code (smart pointers), 
encouraging lazy and lack of thought (smart pointers), and increasing 
my build times by a factor of 100 (STL).

> It lets me think at a higher level.

Thinking about low level details is not mutually exclusive with 
thinking at a higher level.  In fact, I would argue that routinely 
discounting the former affects the latter.

Brian

RE: [GD-General] Redundant Include Guards

[Grills, J. <jg...@so...>]

I once wrote a perl script to implement Lakos' include guard techniques on
all of our source, and it had no significant impact on our build times in
MSVC 6.

j
Jeff Grills
Technical Director
Star Wars Galaxies
Sony Online Entertainment

-----Original Message-----
From: Arjen Beij [mailto:arj...@ga...]
Sent: Monday, December 16, 2002 3:03 AM
To: gam...@li...
Subject: RE: [GD-General] Redundant Include Guards



Not sure about that. It does have "#pragma once" though, which "specifies
that the file, in which the pragma resides, will be included (opened) only
once by the compiler in a build."

Arjen Beij | AI Programmer

Lost Boys Games | Prins Hendrikkade 139 | 1011AS Amsterdam The Netherlands
Tel: +31 (0)20 4272277 | Fax: +31 (0)20 4274040 |
arj...@ga...
www.games.lostboys.com

> -----Original Message-----
> From: gam...@li...
> [mailto:gam...@li...]On Behalf Of
> Dirk Ringe
> Sent: Monday, December 16, 2002 09:27
> To: gam...@li...
> Subject: RE: [GD-General] Redundant Include Guards
>
>
> At least GCC is smart enough to prevent opening header files if they are
> guarded by include guard defines and when they are already parsed. Whether
> this applies to MSVC i do not know.
>
> Dirk