RE: [GD-General] Compile times

[Brian H. <bri...@py...>]

> Does anyone out there have a game project of 10-20MB in size 
> (source & header files), that builds in 5 minutes or less?  I 
> would love to hear about it.

I think that part of the problem is having a game project that large.
With size comes almost unavoidable complexity, which is what I was
talking about -- what is exactly causing them to get that huge?

I know the games I'm personally working on are not that dramatically
complex, they're cheezy puzzle games, but at the same time nearly all
games (especially cross platform ones) end up having to do a minimal set
of operations regardless of overall complexity.

For example, no matter what type of game, you're looking at having some
kind of unarchiver (unless you just dump all your files straight onto
disk, which I suppose some people still do); some kind of
TGA/JPG/PNG/BMP/whatever loader; some kind of streaming audio support
(MP3, Miles, Ogg Vorbis); some kind of graphics/math stuff; if
appropriate, some kind of networking layer built on UDP; some kind of
audio wave file loader/playback; possibly some GUI stuff; memory
management and file format loaders; cross-platform file system stuff;
cross-platform endianess mucking about; cross-platform
registry/preferences manipulation; etc. etc.

My total source base (including third party open source stuff),
including all cross platform stuff, is about 5.5MB (of which MySQL is
2MB).  So at any given time, a full rebuild is probably around 4.5 -
5MB.  Nothing huge, but even so, a full rebuild for me is around one
minute or less.  Take my code, multiply by 4 for a 20MB code base, and
we're still talking about 5 minutes or so, tops.  It's possible that
there's some magic project size that causes MSVC to choke as well.

Hmmm, actually, one thing I should probably mention is that my projects
consist of tons of subprojects.  I don't have a single monolithic DSP
that has like 300 files or anything like that, and it's possible that
this prevents symbol tables or other internal compiler data from getting
out of control.  

> I wonder if in some ways you cross an invisible line where 
> complexity and inter-dependency increases dramatically as 
> your project size grows.

My guess is that this is true.  At some point you lose the ability to
have a gestalt understanding of your code base as a whole, and once
you're at that point, programmers just start chucking things into
directories as they see fit.

> Large-scale projects are not a new beast -- there are whole 
> books written on the subject (J. Lakos. Large-Scale C++ 
> Software Design, for example).

The problem is that many of the books are either written with a late 80s
mindset towards software engineering, which is often at odds with the
new and trendy extreme programming we're seeing, or they're long, gory
discussions about how things exploded on some large project, thinly
disguised post-mortems.

One of my former managers expressed discomfort about a project, and he
said something like "I feel like we're trying to build a Boeing 777,
where everyone puts together all their pieces at the very end and it
fires up and works" and I was like "Yeah, that's exactly what's going
on, and it'll work just fine, this is how modern software engineering
works".  I had complete faith that with well defined interfaces and unit
tests that assembling disparate subsystems 6 months into a project would
not be that tough.

Holy crap was I wrong.  These days, I'm the complete opposite.  To
paraphrase a friend of a friend, the most important thing you can do is
to have the game running ALL THE TIME.  Like, from day one, you should
have SOMETHING that compiles, builds and runs.  

Every day there should be a running, functional executable.  If you
can't build one, or can't guarantee this, then this implies that there
is something seriously wrong with your process.  I know that sounds
rather extreme, but from the projects I've observed, this is more true
than not.  When you hear phrases like "We'll put those pieces together
in a few weeks", alarm bells should be going off.

> Brian 
> has a strong suspicion much of it is accidental and could be 
> removed if the right choices were made during the course of 
> development.

That's a fairly accurate assessment of my feelings, coupled with a
strong feeling that a lot of the complexity is also designed in from the
outset because of a belief that it's necessary.  The case in point I use
are smart pointers, which almost always end up costing more than they're
worth in projects I've seen.  Those are typically designed in from day
one because they seem like a good idea, but then in practice enough ugly
things rear their heads that everyone ends up regretting them at the
end.  And, as I've lamented earlier, I do feel that STL is a contributor
to this as well, but that's borderline religious.

> I would love to see some larger projects that 
> build quickly so that I could gain more confidence in this theory.

My only addition to this is that I'm somewhat suspicious that projects
actually need to be that large, but that's neither here nor there.  I
doubt anyone with a gigantic project feels that it's a bunch of bloated
crap, but I know that I feel a good chunk of my code could just be
tossed or at least refactored into something more manageable, and I only
have a few megabytes of code.

Brian