gamedevlists-general — List for discussing general game development issues

RE: [GD-General] remove unused functions

[Daniel V. <vo...@ep...>]

> > In what version? MSVC++ 6 certainly doesn't expose it.
> 
> 7 and above. __FUNCSIG__ is even better.

Learning something new every day - thanks! Time to upgrade some macros :)

-- Daniel, Epic Games Inc.

re: [GD-General] remove unused functions

[George W. <ge...@ap...>]

On Mon, 19 May 2003 13:57:48 -0500, "Corrinne Yu" <cor...@sp...>
wrote:
> One thing I would like to do to it is to start removing un-used
> functions. 

A "low tech" solution would be to "#if 0" disable all the routines in a
selected sub-set of source file and then try to build your project. When the
linker complains go back and re-enable (un-disable) the "missing" routines.
-- 
Enjoy,
George Warner,
Schizophrenic Optimization Scientists
Apple Developer Technical Support (DTS)

RE: [GD-General] remove unused functions

[Jamie F. <ja...@qu...>]

have you tried a profiler? take a profiler which will instrument the
functions for you, then use the counts to find which functions are never
really used?

Jamie


-----Original Message-----
From: gam...@li...
[mailto:gam...@li...]On Behalf Of
Corrinne Yu
Sent: 19 May 2003 19:58
To: gam...@li...
Subject: [GD-General] remove unused functions


The game engine and game code base has gotten very very big. And it is
about time to clean things up and improve its structure and
documentation.

One thing I would like to do to it is to start removing un-used
functions. I tried to use MSDev to remove un-referenced functions as a
start but didn't find any. It turns out quite a few of the "un-used"
functions are "referenced" (and thus can't be that easily automatically
determined) but I know from how the game and engine code is supposed to
run that they won't ever be called.

They are not exactly unreachable code either, so doing the unreachable
code thing also doesn't work.

I truly "dread" :D the idea to have to manually step through all the
code and manually cull out every single un-used function by hand (it is
a lot of code to wade through!). I wonder what tools exist inside of
MSDev (or simple easy to use small external tools) that can help me in
my highly un fun task of cleaning up my code base, specifically in the
task of removing "un-used" functions.

Thank you for your experience in this task, and your suggestions.



-------------------------------------------------------
This SF.net email is sponsored by: ObjectStore.
If flattening out C++ or Java code to make your application fit in a
relational database is painful, don't do it! Check out ObjectStore.
Now part of Progress Software. http://www.objectstore.net/sourceforge
_______________________________________________
Gamedevlists-general mailing list
Gam...@li...
https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_id=557

Re: [GD-General] remove unused functions

[Ken P. <ke...@dr...>]

One more thing to nudge me towards an upgrade I suppose...

At 05:00 AM 5/20/2003, you wrote:

>Ken Paulson wrote:
> > At 08:07 PM 5/19/2003, you wrote:
> >
> >>> maybe there's a MS #define that lists the function name?
> >>
> >> yep.  __FUNCTION__
> >
> > In what version? MSVC++ 6 certainly doesn't expose it.
>
>7 and above. __FUNCSIG__ is even better.
>
>
>-------------------------------------------------------
>This SF.net email is sponsored by: ObjectStore.
>If flattening out C++ or Java code to make your application fit in a
>relational database is painful, don't do it! Check out ObjectStore.
>Now part of Progress Software. http://www.objectstore.net/sourceforge
>_______________________________________________
>Gamedevlists-general mailing list
>Gam...@li...
>https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
>Archives:
>http://sourceforge.net/mailarchive/forum.php?forum_id=557

Re: [GD-General] remove unused functions

[Peter D. <pd...@mm...>]

Ken Paulson wrote:
> At 08:07 PM 5/19/2003, you wrote:
> 
>>> maybe there's a MS #define that lists the function name?
>> 
>> yep.  __FUNCTION__
> 
> In what version? MSVC++ 6 certainly doesn't expose it.

7 and above. __FUNCSIG__ is even better.

Re: [GD-General] remove unused functions

[<Lag...@wa...>]

Not totaly correct : __FUNCTION__ work with MSVC 7

---------------------
Lagarde Sébastien
Neko entertainment

----- Original Message -----
From: "Daniel Vogel" <vo...@ep...>
To: <gam...@li...>
Sent: Tuesday, May 20, 2003 7:50 AM
Subject: RE: [GD-General] remove unused functions


> Sorry, meant to be a bit more elaborate - AFAIK __FUNCTION__ only works
with
> gcc and not with VC.
>
> -- Daniel, Epic Games Inc.
>
> > -----Original Message-----
> > From: gam...@li...
> > [mailto:gam...@li...]On Behalf Of
> > Daniel Vogel
> > Sent: Monday, May 19, 2003 10:54 PM
> > To: gam...@li...
> > Subject: RE: [GD-General] remove unused functions
> >
> >
> > You're stuck with __FILE__ and __LINE__ then.
> >
> > -- Daniel, Epic Games Inc.
> >
> > > -----Original Message-----
> > > From: gam...@li...
> > > [mailto:gam...@li...]On Behalf Of
> > > Ken Paulson
> > > Sent: Monday, May 19, 2003 10:24 PM
> > > To: gam...@li...
> > > Subject: RE: [GD-General] remove unused functions
> > >
> > >
> > > At 08:07 PM 5/19/2003, you wrote:
> > >
> > > > > maybe there's a MS #define that lists the function name?
> > > >
> > > >yep.  __FUNCTION__
> > >
> > > In what version? MSVC++ 6 certainly doesn't expose it.
> > >
> > >
> > >
> > >
> > > -------------------------------------------------------
> > > This SF.net email is sponsored by: ObjectStore.
> > > If flattening out C++ or Java code to make your application fit in a
> > > relational database is painful, don't do it! Check out ObjectStore.
> > > Now part of Progress Software. http://www.objectstore.net/sourceforge
> > > _______________________________________________
> > > Gamedevlists-general mailing list
> > > Gam...@li...
> > > https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> > > Archives:
> > > http://sourceforge.net/mailarchive/forum.php?forum_id=557
> > >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: ObjectStore.
> > If flattening out C++ or Java code to make your application fit in a
> > relational database is painful, don't do it! Check out ObjectStore.
> > Now part of Progress Software. http://www.objectstore.net/sourceforge
> > _______________________________________________
> > Gamedevlists-general mailing list
> > Gam...@li...
> > https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> > Archives:
> > http://sourceforge.net/mailarchive/forum.php?forum_id=557
> >
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: ObjectStore.
> If flattening out C++ or Java code to make your application fit in a
> relational database is painful, don't do it! Check out ObjectStore.
> Now part of Progress Software. http://www.objectstore.net/sourceforge
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557

Re: [GD-General] Re: asset & document management

[J C L. <cl...@ka...>]

On Mon, 19 May 2003 13:42:39 +0100 
Stefan Boberg <ste...@te...> wrote:

> Hmm... I don't think I would really like to use a revision control
> system where rollback took place locally, using local data. 

Why?

My preferred approach (which uses some properties of BitKeeper and
similar systems) is to have a bit-image copy of the master repository on
the local system.  I then perform operations locally until I'm happy
with the results before sending the changesets up to the master, to the
staging box, departmental master, or whatever.

> Grabbing the latest version is the most common action anyway so this
> might not be such a big issue in practice but in the presence of
> branches there's not always just one "latest" version which
> complicates things ;)

Having the local repository be a first class node (ie other repositories
can be checked out from it etc) removes most of these complications, tho
at the cost of adding some human/organisational requirements.

-- 
J C Lawrence                
---------(*)                Satan, oscillate my metallic sonatas. 
cl...@ka...               He lived as a devil, eh?		  
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.

Re: [GD-General] Re: asset & document management

[J C L. <cl...@ka...>]

On Sun, 18 May 2003 00:26:40 +0200 
Anders Nilsson <br...@ho...> wrote:

> Why aren't they storing the most recent file and then doing the binary
> diffs backwards? So they store the diff of how to get from the newest
> file to the next newest file. That way the newest file would be fast
> to use, you can save full files every time you branch etc. Might be
> slower to get old versions but that's mostly for backup/safety anyway.

The reverse diffs are one of the many reasons I generally won't use CVS
or any of the other RCS based repositories.  Very large chunks of the
historical versions of the IRIX and HP-UX source trees are no longer
accessible in any form due to undetected corruption of the ,v files in
the backups.  In one way or another the ,v file was corrupted, the
corrupted file was backed up, time passed, and no uncorrupted versions
exist.

Forward diff based systems, like BitKeeper and SCCS are necessarily more
expensive, but have the advantage of checking the logical consistency of
the files on every operation.  Ergo, corruption and other problems are
revealed essentially instantly.  In practice the added expense when
compared to the overhead of open(2)/close(2) is generally minimal, even
for files with >2^16 changesets (which I have).

-- 
J C Lawrence                
---------(*)                Satan, oscillate my metallic sonatas. 
cl...@ka...               He lived as a devil, eh?		  
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.

Re: [GD-General] Re: asset & document management

[J C L. <cl...@ka...>]

On Sat, 17 May 2003 23:10:05 +0100 
Neil Stewart <ne...@r0...> wrote:

> I asked Perforce about this a while ago (when we first started using
> it, in fact) and IIRC they said that although storing binary deltas
> would save disk space, it would hurt performance quite badly, as they
> would have to construct enormous files from lots of tiny changes. They
> went for the performance option, which is fair enough.

BitKeeper (of which I'm inordinately fond) takes the default approach of
checking in UUencoded versions of binary files.  As a one-size-fits-all
a[approach it works reasonably.  You can of course also add your own
diff/delta/etc tools with appropriate configs and it will willingly use
those methods instead for the files types you specify.

-- 
J C Lawrence                
---------(*)                Satan, oscillate my metallic sonatas. 
cl...@ka...               He lived as a devil, eh?		  
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.

RE: [GD-General] remove unused functions

[Daniel V. <vo...@ep...>]

Sorry, meant to be a bit more elaborate - AFAIK __FUNCTION__ only works with
gcc and not with VC.

-- Daniel, Epic Games Inc.

> -----Original Message-----
> From: gam...@li...
> [mailto:gam...@li...]On Behalf Of
> Daniel Vogel
> Sent: Monday, May 19, 2003 10:54 PM
> To: gam...@li...
> Subject: RE: [GD-General] remove unused functions
>
>
> You're stuck with __FILE__ and __LINE__ then.
>
> -- Daniel, Epic Games Inc.
>
> > -----Original Message-----
> > From: gam...@li...
> > [mailto:gam...@li...]On Behalf Of
> > Ken Paulson
> > Sent: Monday, May 19, 2003 10:24 PM
> > To: gam...@li...
> > Subject: RE: [GD-General] remove unused functions
> >
> >
> > At 08:07 PM 5/19/2003, you wrote:
> >
> > > > maybe there's a MS #define that lists the function name?
> > >
> > >yep.  __FUNCTION__
> >
> > In what version? MSVC++ 6 certainly doesn't expose it.
> >
> >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: ObjectStore.
> > If flattening out C++ or Java code to make your application fit in a
> > relational database is painful, don't do it! Check out ObjectStore.
> > Now part of Progress Software. http://www.objectstore.net/sourceforge
> > _______________________________________________
> > Gamedevlists-general mailing list
> > Gam...@li...
> > https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> > Archives:
> > http://sourceforge.net/mailarchive/forum.php?forum_id=557
> >
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: ObjectStore.
> If flattening out C++ or Java code to make your application fit in a
> relational database is painful, don't do it! Check out ObjectStore.
> Now part of Progress Software. http://www.objectstore.net/sourceforge
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557
>

RE: [GD-General] remove unused functions

[Daniel V. <vo...@ep...>]

You're stuck with __FILE__ and __LINE__ then.

-- Daniel, Epic Games Inc. 

> -----Original Message-----
> From: gam...@li...
> [mailto:gam...@li...]On Behalf Of
> Ken Paulson
> Sent: Monday, May 19, 2003 10:24 PM
> To: gam...@li...
> Subject: RE: [GD-General] remove unused functions
> 
> 
> At 08:07 PM 5/19/2003, you wrote:
> 
> > > maybe there's a MS #define that lists the function name?
> >
> >yep.  __FUNCTION__
> 
> In what version? MSVC++ 6 certainly doesn't expose it.
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: ObjectStore.
> If flattening out C++ or Java code to make your application fit in a
> relational database is painful, don't do it! Check out ObjectStore.
> Now part of Progress Software. http://www.objectstore.net/sourceforge
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557
> 

RE: [GD-General] remove unused functions

[Ken P. <ke...@dr...>]

At 08:07 PM 5/19/2003, you wrote:

> > maybe there's a MS #define that lists the function name?
>
>yep.  __FUNCTION__

In what version? MSVC++ 6 certainly doesn't expose it.

RE: [GD-General] remove unused functions

[Jim S. <ja...@mi...>]

> maybe there's a MS #define that lists the function name?

yep.  __FUNCTION__

RE: [GD-General] remove unused functions

[Corrinne Y. <cor...@sp...>]

Thank you for all these suggestions. Quite a few of them sound do-able.

Using a formal analysis tool would had been good if it were done from
the beginning, but at this point I am probably stuck with several of the
global "#define" search and replace and text dump suggestions, as I am
trying to do this with as little new code writing, and as little
"learning how to use a new tool" as possible.

I may consider looking at code analysis tools when I start a new project
(or using the existing engine to start a new project, or let other
companies or teams use the existing engine with a new project) to see if
it is a lot of work and effort (or high learning curve). I wish I had
looked more into "big code project type tools" earlier on; kept
convincing myself my project probably would be "small" and it does turn
into one of those "software project management size" type of code base;
it got even bigger than source code projects for when I worked for other
companies and teams. Engines and games must have gotten highly complex
in the last year or so and had increased greatly in function complexity.
Games and engines after this one probably would become so huge that
"formal software documentation and structuring" would become almost a
necessity for the next round. (Days of winging it un-documented appear
to be diminishing. :) )

Again, thank you for all the pointers. They have all been very helpful.

Re: [GD-General] remove unused functions

[Neil S. <ne...@r0...>]

> One thing I would like to do to it is to start removing un-used
> functions. I tried to use MSDev to remove un-referenced functions as a
> start but didn't find any. It turns out quite a few of the "un-used"
> functions are "referenced" (and thus can't be that easily automatically
> determined) but I know from how the game and engine code is supposed to
> run that they won't ever be called.
>
> They are not exactly unreachable code either, so doing the unreachable
> code thing also doesn't work.

What about using a code coverage tool, like Compuware's DevPartner or
something similar? They are really designed to show what code is (and isn't)
covered by unit tests, but they might be able to work with an entire run of
your code. If not, you could create some unit tests for each of your
components, such as rendering, AI, etc., and then run them through such a
tool.

- Neil.

Re: [GD-General] remove unused functions

[Colin F. <cp...@ea...>]

Perhaps you could profile your code (Linux: gprof?  Windows: VC++ 6.0
profile option), and dump the output to a text file.  Report the function
entry counts, not timings or line numbers.

Strip out functions with non-zero total entries (if list is sorted by
counts, just chop off the non-zero part of the list).

What you do next depends on what you want to do, and how much work
you are willing to put in to this.

For me it would be fairly easy to write a program to use the list of
functions to search through all source code files, identify both
calling instances and function implementations, and prompt the user
to approve proposed changes (e.g., removal or commenting out).

It's tricky for situations like:

  if (never_true)
    {
    if (unused_function( x, y, z ))
      {
      // ...
      }
    }

and:

  f = unused_function( x, y, z );

since you have to do more than strip out the function call itself!
But maybe you CAN just strip out the function calling instance,
and replace with a blank, or junk, and wait for compile errors
to alert people to required fixes.  That sounds risky.

If you go with the prompting approach, you can have your utility
propose a variety of substitutions for the block and vicinity.
Based on the context you can come up with one or more likely
"fixes" that the user can choose from.

I'm sure hitting single keys for "SKIP", "PROPOSED MODIFICATION #1",
"PROPOSED MODIFICATION #2", etc, per fix, would be a very fast process
for a human.

If you have methods with identical names in several C++ classes,
things get a little trickier, since you are starting to do the
work of the compiler to analyze code!

Actually, this whole thing sounds like a fun utility to write, like
"lint", but with the added information from a profiler.  Maybe such
a thing already exists...

--- Colin




----- Original Message -----
From: "Tom Forsyth" <to...@mu...>
To: <gam...@li...>
Sent: Monday, May 19, 2003 12:38 PM
Subject: RE: [GD-General] remove unused functions


> Maybe you can add a prologue to every function that does two things:
>
> (a) declares a static instance of a class whose constructor adds the
string
> "__FILE__ __LINE__" to a table.
>
> (b) when it is called, finds and removes the string from the table.
>
>
> Play the game for a while, go through as many different codepaths and
> behaviours as you can think of, then dump the table to disk or whatever.
And
> there you go - a list of all the functions (well, where they are - maybe
> there's a MS #define that lists the function name?) that haven't been
called
> this time around.
>
> Then add ASSERT (!"Nobody ever calls me or they would have removed this
> assert") to each file, leave it a couple of weeks, and then kill all the
> functions that still have that assert.
>
>
> Tom Forsyth - Muckyfoot bloke and Microsoft MVP.
>
> This email is the product of your deranged imagination,
> and does not in any way imply existence of the author.
>
> > -----Original Message-----
> > From: Corrinne Yu [mailto:cor...@sp...]
> > Sent: 19 May 2003 19:58
> > To: gam...@li...
> > Subject: [GD-General] remove unused functions
> >
> >
> > The game engine and game code base has gotten very very big. And it is
> > about time to clean things up and improve its structure and
> > documentation.
> >
> > One thing I would like to do to it is to start removing un-used
> > functions. I tried to use MSDev to remove un-referenced functions as a
> > start but didn't find any. It turns out quite a few of the "un-used"
> > functions are "referenced" (and thus can't be that easily
> > automatically
> > determined) but I know from how the game and engine code is
> > supposed to
> > run that they won't ever be called.
> >
> > They are not exactly unreachable code either, so doing the unreachable
> > code thing also doesn't work.
> >
> > I truly "dread" :D the idea to have to manually step through all the
> > code and manually cull out every single un-used function by
> > hand (it is
> > a lot of code to wade through!). I wonder what tools exist inside of
> > MSDev (or simple easy to use small external tools) that can help me in
> > my highly un fun task of cleaning up my code base, specifically in the
> > task of removing "un-used" functions.
> >
> > Thank you for your experience in this task, and your suggestions.
> >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: ObjectStore.
> > If flattening out C++ or Java code to make your application fit in a
> > relational database is painful, don't do it! Check out ObjectStore.
> > Now part of Progress Software. http://www.objectstore.net/sourceforge
> > _______________________________________________
> > Gamedevlists-general mailing list
> > Gam...@li...
> > https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> > Archives:
> > http://sourceforge.net/mailarchive/forum.php?forum_id=557
> >
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: ObjectStore.
> If flattening out C++ or Java code to make your application fit in a
> relational database is painful, don't do it! Check out ObjectStore.
> Now part of Progress Software. http://www.objectstore.net/sourceforge
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557

RE: [GD-General] remove unused functions

[Stefan B. <ste...@te...>]

> I truly "dread" :D the idea to have to manually step through all the
> code and manually cull out every single un-used function by hand (it =
is
> a lot of code to wade through!). I wonder what tools exist inside of
> MSDev (or simple easy to use small external tools) that can help me in
> my highly un fun task of cleaning up my code base, specifically in the
> task of removing "un-used" functions.

   There are coverage analysis tools that you could use to do this. =
Normally
these tools are used in automated testing, to verify that all code paths =
are
exercised by the test cases. We have some Rational PurifyPlus licenses =
and
as part of their package they bundle Quantify and PureCoverage. I'm sure
there are lots of other packages available but I have no idea what's the
best solution.=20

   Anyway, just Google for "code coverage analysis" or something like =
that
and you should get plenty of hits.

Cheers,
Stef! :)
--
Stefan Boberg, R&D Manager - Team17 Software Ltd.
bo...@te...

RE: [GD-General] remove unused functions

[Tom F. <to...@mu...>]

Maybe you can add a prologue to every function that does two things:

(a) declares a static instance of a class whose constructor adds the string
"__FILE__ __LINE__" to a table.

(b) when it is called, finds and removes the string from the table.


Play the game for a while, go through as many different codepaths and
behaviours as you can think of, then dump the table to disk or whatever. And
there you go - a list of all the functions (well, where they are - maybe
there's a MS #define that lists the function name?) that haven't been called
this time around.

Then add ASSERT (!"Nobody ever calls me or they would have removed this
assert") to each file, leave it a couple of weeks, and then kill all the
functions that still have that assert.


Tom Forsyth - Muckyfoot bloke and Microsoft MVP.

This email is the product of your deranged imagination,
and does not in any way imply existence of the author.

> -----Original Message-----
> From: Corrinne Yu [mailto:cor...@sp...]
> Sent: 19 May 2003 19:58
> To: gam...@li...
> Subject: [GD-General] remove unused functions
> 
> 
> The game engine and game code base has gotten very very big. And it is
> about time to clean things up and improve its structure and
> documentation.
> 
> One thing I would like to do to it is to start removing un-used
> functions. I tried to use MSDev to remove un-referenced functions as a
> start but didn't find any. It turns out quite a few of the "un-used"
> functions are "referenced" (and thus can't be that easily 
> automatically
> determined) but I know from how the game and engine code is 
> supposed to
> run that they won't ever be called.
> 
> They are not exactly unreachable code either, so doing the unreachable
> code thing also doesn't work.
> 
> I truly "dread" :D the idea to have to manually step through all the
> code and manually cull out every single un-used function by 
> hand (it is
> a lot of code to wade through!). I wonder what tools exist inside of
> MSDev (or simple easy to use small external tools) that can help me in
> my highly un fun task of cleaning up my code base, specifically in the
> task of removing "un-used" functions.
> 
> Thank you for your experience in this task, and your suggestions.
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: ObjectStore.
> If flattening out C++ or Java code to make your application fit in a
> relational database is painful, don't do it! Check out ObjectStore.
> Now part of Progress Software. http://www.objectstore.net/sourceforge
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557
> 

[GD-General] remove unused functions

[Corrinne Y. <cor...@sp...>]

The game engine and game code base has gotten very very big. And it is
about time to clean things up and improve its structure and
documentation.

One thing I would like to do to it is to start removing un-used
functions. I tried to use MSDev to remove un-referenced functions as a
start but didn't find any. It turns out quite a few of the "un-used"
functions are "referenced" (and thus can't be that easily automatically
determined) but I know from how the game and engine code is supposed to
run that they won't ever be called.

They are not exactly unreachable code either, so doing the unreachable
code thing also doesn't work.

I truly "dread" :D the idea to have to manually step through all the
code and manually cull out every single un-used function by hand (it is
a lot of code to wade through!). I wonder what tools exist inside of
MSDev (or simple easy to use small external tools) that can help me in
my highly un fun task of cleaning up my code base, specifically in the
task of removing "un-used" functions.

Thank you for your experience in this task, and your suggestions.

Re: [GD-General] Re: asset & document management

[Neil S. <ne...@r0...>]

> I'd think the time required to grab huge files from a server is comparable
> to the time required to get the much smaller diffs and apply them to the
> existing file. We've found that even complex stuff like LZ-like
compression
> makes disk accesses faster, not slower, because it's so fast to decompress
> and still gives good space savings. So applying a diff is a doddle,
surely?

Loading compressed files is faster because the hard disk access is the
limiting factor, not the cost of decompressing the data. Writing compressed
files can also be faster, depending on the cost of the dictionary search and
the resultant compression factor. Perforce does compress its files,
suggesting that they found both reading and writing to be fast enough.

When applying diffs, you are actually reading more from disk than the size
of the ultimate file, not less, and the amount of extra disk access is
dependent on how many diffs you will have to apply to get the required
version of the file, so going further back will take more and more hard disk
access and therfore be slower.

Applying a few diffs onto an almost complete file should be pretty quick
though, which is why I was suggesting having intermittent, complete copies
of each file. It should be easy to find a tradeoff where both space savings
and performance are good.

- Neil.

RE: [GD-General] Re: asset & document management

[Stefan B. <ste...@te...>]

> > One of the things they seem to pride themselves in is
> > Perforce's ability to
> > reliably roll-back to any changelist or label very quickly,
>=20
> I'd think the time required to grab huge files from a server is =
comparable
> to the time required to get the much smaller diffs and apply them to =
the
> existing file. We've found that even complex stuff like LZ-like
> compression makes disk accesses faster, not slower, because it's so =
fast
> to decompress and still gives good space savings. So applying a diff =
is a
> doddle, surely?

   Hmm... I don't think I would really like to use a revision control =
system
where rollback took place locally, using local data. The diffs are =
always
applied on the server and the resulting file is sent to the client.
Therefore grabbing a big "key" file (i.e. a standalone copy much like a
key-frame in video files) and applying a set of deltas would have to be
performed on the server and then sent to the client. Unless you can come =
up
with a very clever on-the-fly delta application algorithm, this could
consume a fair amount of server resources, which is something P4 tries =
very
hard not to do (and that's why it scales to a very large number of users
without requiring a hefty server).

   Grabbing the latest version is the most common action anyway so this
might not be such a big issue in practice but in the presence of =
branches
there's not always just one "latest" version which complicates things ;)

   I believe the Subversion team originally stated that binaries would =
be
stored using (xdelta? rsync?) deltas but that's not been implemented yet =
so
I guess it's not as easy as they anticipated.

Cheers,
Stef!
--
Stefan Boberg, R&D Manager - Team17 Software Ltd.
bo...@te...

Re: [GD-General] Re: asset & document management

[Mickael P. <mpo...@ed...>]

>> One of the things they seem to pride themselves in is
>> Perforce's ability to
>> reliably roll-back to any changelist or label very quickly,
>
> I'd think the time required to grab huge files from a server is
> comparable to the time required to get the much smaller diffs and
> apply them to the existing file. We've found that even complex stuff
> like LZ-like compression makes disk accesses faster, not slower,
> because it's so fast to decompress and still gives good space
> savings. So applying a diff is a doddle, surely?

Hum, of course the whole get operation will eventually be faster, but the PC
resources will totaly go down, and the machine will crawl.

If you don't have any particular treatment to perform on the data coming
from the server, you can afford to use functions like "CopyFileEx", and
basicaly the whole operation will not even be noticeable (except for disk
usage) by the user.

    Mickael Pointier

RE: [GD-General] Re: asset & document management

[Tom F. <to...@mu...>]

> One of the things they seem to pride themselves in is 
> Perforce's ability to
> reliably roll-back to any changelist or label very quickly, 

I'd think the time required to grab huge files from a server is comparable
to the time required to get the much smaller diffs and apply them to the
existing file. We've found that even complex stuff like LZ-like compression
makes disk accesses faster, not slower, because it's so fast to decompress
and still gives good space savings. So applying a diff is a doddle, surely?


Tom Forsyth - Muckyfoot bloke and Microsoft MVP.

This email is the product of your deranged imagination,
and does not in any way imply existence of the author.

> -----Original Message-----
> From: Neil Stewart [mailto:ne...@r0...]
> Sent: 18 May 2003 00:10
> To: gam...@li...
> Subject: Re: [GD-General] Re: asset & document management
> 
> 
> > Why aren't they storing the most recent file and then doing 
> the binary
> diffs
> > backwards? So they store the diff of how to get from the 
> newest file to
> the
> > next newest file. That way the newest file would be fast to 
> use, you can
> > save full files every time you branch etc. Might be slower 
> to get old
> > versions but that's mostly for backup/safety anyway.
> 
> Well, they aren't storing any binary diffs at the moment, never mind
> backwards. ;)
> 
> I get your point, but I don't think they were willing to compromise
> performance on _any_ revision of a file, including the most 
> common usage of
> getting the latest version.
> 
> One of the things they seem to pride themselves in is 
> Perforce's ability to
> reliably roll-back to any changelist or label very quickly, 
> specifically as
> a bug-finding tool, i.e. you can roll back to a version where 
> a nasty bug
> does not exist and then roll forward to see what broke. This 
> would not be
> possible if they only optimised for the latest version of a 
> file, so they
> chose not to store binary diffs at all.
> 
> What I was suggesting was a halfway-house, where you could 
> tradeoff overall
> performance (on all files) against disk usage.
> 
> - Neil.
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: If flattening out C++ or Java
> code to make your application fit in a relational database is 
> painful, 
> don't do it! Check out ObjectStore. Now part of Progress Software.
> http://www.objectstore.net/sourceforge
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557
> 

Re: [GD-General] Re: asset & document management

[Neil S. <ne...@r0...>]

> Why aren't they storing the most recent file and then doing the binary
diffs
> backwards? So they store the diff of how to get from the newest file to
the
> next newest file. That way the newest file would be fast to use, you can
> save full files every time you branch etc. Might be slower to get old
> versions but that's mostly for backup/safety anyway.

Well, they aren't storing any binary diffs at the moment, never mind
backwards. ;)

I get your point, but I don't think they were willing to compromise
performance on _any_ revision of a file, including the most common usage of
getting the latest version.

One of the things they seem to pride themselves in is Perforce's ability to
reliably roll-back to any changelist or label very quickly, specifically as
a bug-finding tool, i.e. you can roll back to a version where a nasty bug
does not exist and then roll forward to see what broke. This would not be
possible if they only optimised for the latest version of a file, so they
chose not to store binary diffs at all.

What I was suggesting was a halfway-house, where you could tradeoff overall
performance (on all files) against disk usage.

- Neil.

RE: [GD-General] Re: asset & document management

[Anders N. <br...@ho...>]

Why aren't they storing the most recent file and then doing the binary diffs
backwards? So they store the diff of how to get from the newest file to the
next newest file. That way the newest file would be fast to use, you can
save full files every time you branch etc. Might be slower to get old
versions but that's mostly for backup/safety anyway.

Anders Nilsson

>> It would be nice if they supported some sort of xdelta-like diff storage
>> instead, but it seems like it's not a massive issue nowadays when huge
>> drives are cheap.

Neil:
>I asked Perforce about this a while ago (when we first started using it, in
>fact) and IIRC they said that although storing binary deltas would save
disk
>space, it would hurt performance quite badly, as they would have to
>construct enormous files from lots of tiny changes. They went for the
>performance option, which is fair enough.