gamedevlists-general — List for discussing general game development issues

RE: [GD-General] C++ analyzers?

[Brian H. <bri...@py...>]

Quite true, and this illustrates my earlier comments of how bad most
software development tools.  Something as conceptually simple as
renaming a variable can be incredibly difficult to manage.  Even using a
standard automated tool like perl or sed/awk you run into the problem
that there's no context.

We have tools to automatically generate code and do all other kinds of
fluff crap, but if you actually want something that can go through and
rename all instances of "m_foo" for all instances of class "CBar", it
doesn't seem possible.  If we don't even have tools that can tell you
something as trivial as what variables are unused, then the above
doesn't seem likely to happen any time soon =)

MSVC doesn't warm that:

int i = 0;

is an unused variable even if you don't use it anywhere.  And any class
with a constructor also won't trigger the warning, even if the
constructor doesn't do anything.  So I find myself with lots of
temporaries, etc. that are laying around that I didn't realize that are
there or are no longer using.  A common example is when you create a
temporary solely to make debugging easier:

foo f;

f = blah();

callSomefunction( f );

But then later you fix the bug, restructure the code, etc. and do:

callSomefunction( blah() );

But you forget that "foo f" is still there.  And it's gonna stay there
forever now =)  One of the nicer aspects of ANSI C is that since there
aren't hidden operations like constructors, destructors, etc. then it
can be a lot more stringent about finding things like that.

That ebrowser thing might be nice.

Brian

> -----Original Message-----
> From: gam...@li... 
> [mailto:gam...@li...] On 
> Behalf Of Brian Sharon
> Sent: Thursday, July 18, 2002 12:28 PM
> To: gam...@li...
> Subject: RE: [GD-General] C++ analyzers?
> 
> 
> I hope I'm not guilty of stating the blatantly obvious, but 
> renaming variables and checking in has to be one of the 
> classic "I couldn't have broken the build, I didn't change 
> anything" moves.  I wouldn't think renamed symbols are a good 
> thing to ignore, if you're trying figure out why things that 
> used to work no longer do.

RE: [GD-General] C++ analyzers?

[Brian S. <bs...@mi...>]

I hope I'm not guilty of stating the blatantly obvious, but renaming
variables and checking in has to be one of the classic "I couldn't have
broken the build, I didn't change anything" moves.  I wouldn't think
renamed symbols are a good thing to ignore, if you're trying figure out
why things that used to work no longer do.

--brian

> -----Original Message-----
> From: Peter Lipson [mailto:pe...@to...]=20
> Sent: Thursday, July 18, 2002 11:20 AM
> To: gam...@li...
> Subject: RE: [GD-General] C++ analyzers?
>=20
>=20
> coincidentally enough, today I was searching for (and not=20
> finding..) a file
> differencing program that understood C++. Most differencing=20
> programs can be
> configured to ignore whitespace; it would be nice to ignore=20
> comments, too.
> Even more powerful would be one that compared the parse-tree, ignoring
> symbol name changes. This may sound frivolous, but it's not.=20
> Typically, when
> I'm merging some code into an existing project, I'll add a=20
> ton of comments,
> rename things to match a common set of naming conventions,=20
> maybe add or
> remove a few parentheses or curly braces, and maybe move a=20
> few variable
> declarations or definitions.
>   If the code breaks after that, I've totally lost the=20
> ability to (in an
> automated way) compare it to the original source. But none of=20
> the above
> operations -should- have changed the way the parsed code=20
> should look. (OK,
> maybe moving variable definitions could..)
>=20
>   I've hunted around the net a bit, and haven't found a code=20
> repository of
> helpful little utilities like that. I suspect it's out there=20
> somewhere,
> though.
>=20
> Peter
>=20
> -----Original Message-----
> From: gam...@li...
> [mailto:gam...@li...]On Behalf Of
> Brian Hook
> Sent: Wednesday, July 17, 2002 11:35 AM
> To: gam...@li...
> Subject: [GD-General] C++ analyzers?
>=20
>=20
> One of my bitches about software development are that the tools suck.
> Many things that can and should be automated simply aren't. =20
> Tools like
> Emacs and make provide a lot of helpers when you have to do=20
> this manual
> stuff, but they don't actually solve problems that are=20
> language specific
> (since those tools aren't language specific).
>=20
> When working on a large codebase, invariably it acquires=20
> several layers
> of cruft and detritus.  Several Java IDEs provide high level=20
> tools to do
> things like refactor, automatically name types and objects, etc.  This
> can be immensely useful.
>=20
> Now that I'm parked on about 120K of source code that I've written,
> there are times when I want to issue a "query" about some type or
> variable.
>=20
> Of course, that query ends up being "grep", which completely=20
> and utterly
> sucks when you have lots of name collisions.  There are=20
> certain variable
> and function names that are so ubiquitous that grepping for=20
> them returns
> a thousand hits.  init/create/alloc and shutdown/destroy/free; flags,
> type, name, next, prev, id; etc.
>=20
> One way I've gotten around this is defensive coding, whereby I prefix
> members with their parent type name:
>=20
> struct HSprite
> {
>    int iSpriteWidth; //instead of iWidth
> };
>=20
> Anyone that has used an old K&R style compiler that didn't=20
> consider the
> type and global namespaces differently will probably have=20
> done something
> similar out of necessity.
>=20
> It should be entirely possible for a code analyzer to go through and
> build a dependency and call graph, with knowledge of the framework's
> inheritance, friendship and polymorphism, and answer queries=20
> or generate
> warnings.
>=20
> When doing spring cleaning on my code base, some of the examples of
> things that I know the compiler/linker could do but simply don't (and
> which requires lots of searching for me) are:
>=20
> - warn when including header files that aren't needed.  A typical
> example is when you're doing some debugging and you need sprintf() or
> FILE so you #include <stdio.h>, but forget to remove it when you're
> done.
>=20
> - warn when forward declarations aren't used or aren't relevant, e.g.
> you do "class Foo" which you've later renamed to "class=20
> FooBar", but you
> still have forward declarations to a non-existent class (which don't
> cause problems)
>=20
> - warn when friendships aren't necessary.  This happens when you
> establish a friendship from A to B because you don't want
> B::privateThing accessible by the world, just by A.  But then later on
> you change some stuff, and now A doesn't need=20
> B::privateThing, but it's
> still a friend of the class.
>=20
> - warn about types, classes, variables, and functions that are never
> referenced.  This is pretty open ended, and can mean everything from
> abstract base classes that are never derived from to concrete classes
> that are never instantiated, but it helps to find vestigial=20
> cruft.  This
> has a lot of problems if all you're doing is building a=20
> library because
> you can't tell what the client program is calling, but if you're
> building test programs, it's a good way of making sure you have
> reasonable coverage analysis without actually running a coverage
> analysis program.
>=20
> And, I'm sure, there are a million other things it could=20
> check for that
> currently require manual intervention.  Anyone know if such a tool
> exists?
>=20
> -Hook

RE: [GD-General] C++ analyzers?

[Mick W. <mi...@ne...>]

Araxis Merge (www.araxis.com) can be configured to ignore lines based on
regular expressions.  It comes with default expressions for (optionally)
ignoring C++ // style comments, and VSS auto-inserted data.   It's also a
brilliant merge program.

Mick.

-----Original Message-----
From: gam...@li...
[mailto:gam...@li...]On Behalf Of
Peter Lipson
Sent: Thursday, July 18, 2002 11:20 AM
To: gam...@li...
Subject: RE: [GD-General] C++ analyzers?


coincidentally enough, today I was searching for (and not finding..) a file
differencing program that understood C++. Most differencing programs can be
configured to ignore whitespace; it would be nice to ignore comments, too.

Re[2]: [GD-General] C++ analyzers?

[Kerim B. <wa...@st...>]

Hello Peter,

Thursday, July 18, 2002, 10:19:42 PM, you wrote:

PL> coincidentally enough, today I was searching for (and not finding..) a file
PL> differencing program that understood C++. Most differencing programs can be
PL> configured to ignore whitespace; it would be nice to ignore comments, too.
PL> Even more powerful would be one that compared the parse-tree, ignoring
PL> symbol name changes. This may sound frivolous, but it's not. Typically, when
PL> I'm merging some code into an existing project, I'll add a ton of comments,
PL> rename things to match a common set of naming conventions, maybe add or
PL> remove a few parentheses or curly braces, and maybe move a few variable
PL> declarations or definitions.
PL>   If the code breaks after that, I've totally lost the ability to (in an
PL> automated way) compare it to the original source. But none of the above
PL> operations -should- have changed the way the parsed code should look. (OK,
PL> maybe moving variable definitions could..)

PL>   I've hunted around the net a bit, and haven't found a code repository of
PL> helpful little utilities like that. I suspect it's out there somewhere,
PL> though.

It's not precisely what you're asking for but using unit tests and
integrating your code often helps with this kind of problems alot.

(In case someone missed it read excellent Martin Fowler's "Continuous
Integration" where he explays why and when it is good for a man to
check-in often: 
http://www.martinfowler.com/articles/continuousIntegration.html )

Best regards,
 Kerim                            mailto:wa...@st...


PL> Peter

PL> -----Original Message-----
PL> From: gam...@li...
PL> [mailto:gam...@li...]On Behalf Of
PL> Brian Hook
PL> Sent: Wednesday, July 17, 2002 11:35 AM
PL> To: gam...@li...
PL> Subject: [GD-General] C++ analyzers?


PL> One of my bitches about software development are that the tools suck.
PL> Many things that can and should be automated simply aren't.  Tools like
PL> Emacs and make provide a lot of helpers when you have to do this manual
PL> stuff, but they don't actually solve problems that are language specific
PL> (since those tools aren't language specific).

PL> When working on a large codebase, invariably it acquires several layers
PL> of cruft and detritus.  Several Java IDEs provide high level tools to do
PL> things like refactor, automatically name types and objects, etc.  This
PL> can be immensely useful.

PL> Now that I'm parked on about 120K of source code that I've written,
PL> there are times when I want to issue a "query" about some type or
PL> variable.

PL> Of course, that query ends up being "grep", which completely and utterly
PL> sucks when you have lots of name collisions.  There are certain variable
PL> and function names that are so ubiquitous that grepping for them returns
PL> a thousand hits.  init/create/alloc and shutdown/destroy/free; flags,
PL> type, name, next, prev, id; etc.

PL> One way I've gotten around this is defensive coding, whereby I prefix
PL> members with their parent type name:

PL> struct HSprite
PL> {
PL>    int iSpriteWidth; //instead of iWidth
PL> };

PL> Anyone that has used an old K&R style compiler that didn't consider the
PL> type and global namespaces differently will probably have done something
PL> similar out of necessity.

PL> It should be entirely possible for a code analyzer to go through and
PL> build a dependency and call graph, with knowledge of the framework's
PL> inheritance, friendship and polymorphism, and answer queries or generate
PL> warnings.

PL> When doing spring cleaning on my code base, some of the examples of
PL> things that I know the compiler/linker could do but simply don't (and
PL> which requires lots of searching for me) are:

PL> - warn when including header files that aren't needed.  A typical
PL> example is when you're doing some debugging and you need sprintf() or
PL> FILE so you #include <stdio.h>, but forget to remove it when you're
PL> done.

PL> - warn when forward declarations aren't used or aren't relevant, e.g.
PL> you do "class Foo" which you've later renamed to "class FooBar", but you
PL> still have forward declarations to a non-existent class (which don't
PL> cause problems)

PL> - warn when friendships aren't necessary.  This happens when you
PL> establish a friendship from A to B because you don't want
PL> B::privateThing accessible by the world, just by A.  But then later on
PL> you change some stuff, and now A doesn't need B::privateThing, but it's
PL> still a friend of the class.

PL> - warn about types, classes, variables, and functions that are never
PL> referenced.  This is pretty open ended, and can mean everything from
PL> abstract base classes that are never derived from to concrete classes
PL> that are never instantiated, but it helps to find vestigial cruft.  This
PL> has a lot of problems if all you're doing is building a library because
PL> you can't tell what the client program is calling, but if you're
PL> building test programs, it's a good way of making sure you have
PL> reasonable coverage analysis without actually running a coverage
PL> analysis program.

PL> And, I'm sure, there are a million other things it could check for that
PL> currently require manual intervention.  Anyone know if such a tool
PL> exists?

PL> -Hook



PL> -------------------------------------------------------
PL> This sf.net email is sponsored by:ThinkGeek
PL> Welcome to geek heaven.
PL> http://thinkgeek.com/sf
PL> _______________________________________________
PL> Gamedevlists-general mailing list
PL> Gam...@li...
PL> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
PL> Archives:
PL> http://sourceforge.net/mailarchive/forum.php?forum_id=557




PL> -------------------------------------------------------
PL> This sf.net email is sponsored by:ThinkGeek
PL> Welcome to geek heaven.
PL> http://thinkgeek.com/sf
PL> _______________________________________________
PL> Gamedevlists-general mailing list
PL> Gam...@li...
PL> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
PL> Archives:
PL> http://sourceforge.net/mailarchive/forum.php?forum_id=557

RE: [GD-General] C++ analyzers?

[Peter L. <pe...@to...>]

coincidentally enough, today I was searching for (and not finding..) a file
differencing program that understood C++. Most differencing programs can be
configured to ignore whitespace; it would be nice to ignore comments, too.
Even more powerful would be one that compared the parse-tree, ignoring
symbol name changes. This may sound frivolous, but it's not. Typically, when
I'm merging some code into an existing project, I'll add a ton of comments,
rename things to match a common set of naming conventions, maybe add or
remove a few parentheses or curly braces, and maybe move a few variable
declarations or definitions.
  If the code breaks after that, I've totally lost the ability to (in an
automated way) compare it to the original source. But none of the above
operations -should- have changed the way the parsed code should look. (OK,
maybe moving variable definitions could..)

  I've hunted around the net a bit, and haven't found a code repository of
helpful little utilities like that. I suspect it's out there somewhere,
though.

Peter

-----Original Message-----
From: gam...@li...
[mailto:gam...@li...]On Behalf Of
Brian Hook
Sent: Wednesday, July 17, 2002 11:35 AM
To: gam...@li...
Subject: [GD-General] C++ analyzers?


One of my bitches about software development are that the tools suck.
Many things that can and should be automated simply aren't.  Tools like
Emacs and make provide a lot of helpers when you have to do this manual
stuff, but they don't actually solve problems that are language specific
(since those tools aren't language specific).

When working on a large codebase, invariably it acquires several layers
of cruft and detritus.  Several Java IDEs provide high level tools to do
things like refactor, automatically name types and objects, etc.  This
can be immensely useful.

Now that I'm parked on about 120K of source code that I've written,
there are times when I want to issue a "query" about some type or
variable.

Of course, that query ends up being "grep", which completely and utterly
sucks when you have lots of name collisions.  There are certain variable
and function names that are so ubiquitous that grepping for them returns
a thousand hits.  init/create/alloc and shutdown/destroy/free; flags,
type, name, next, prev, id; etc.

One way I've gotten around this is defensive coding, whereby I prefix
members with their parent type name:

struct HSprite
{
   int iSpriteWidth; //instead of iWidth
};

Anyone that has used an old K&R style compiler that didn't consider the
type and global namespaces differently will probably have done something
similar out of necessity.

It should be entirely possible for a code analyzer to go through and
build a dependency and call graph, with knowledge of the framework's
inheritance, friendship and polymorphism, and answer queries or generate
warnings.

When doing spring cleaning on my code base, some of the examples of
things that I know the compiler/linker could do but simply don't (and
which requires lots of searching for me) are:

- warn when including header files that aren't needed.  A typical
example is when you're doing some debugging and you need sprintf() or
FILE so you #include <stdio.h>, but forget to remove it when you're
done.

- warn when forward declarations aren't used or aren't relevant, e.g.
you do "class Foo" which you've later renamed to "class FooBar", but you
still have forward declarations to a non-existent class (which don't
cause problems)

- warn when friendships aren't necessary.  This happens when you
establish a friendship from A to B because you don't want
B::privateThing accessible by the world, just by A.  But then later on
you change some stuff, and now A doesn't need B::privateThing, but it's
still a friend of the class.

- warn about types, classes, variables, and functions that are never
referenced.  This is pretty open ended, and can mean everything from
abstract base classes that are never derived from to concrete classes
that are never instantiated, but it helps to find vestigial cruft.  This
has a lot of problems if all you're doing is building a library because
you can't tell what the client program is calling, but if you're
building test programs, it's a good way of making sure you have
reasonable coverage analysis without actually running a coverage
analysis program.

And, I'm sure, there are a million other things it could check for that
currently require manual intervention.  Anyone know if such a tool
exists?

-Hook



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Gamedevlists-general mailing list
Gam...@li...
https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_id=557

Re: [GD-General] C++ analyzers?

[Parveen K. <pk...@sf...>]

Brian Hook wrote:
> One of my bitches about software development are that the tools suck.
> Many things that can and should be automated simply aren't.  Tools like
> Emacs and make provide a lot of helpers when you have to do this manual
> stuff, but they don't actually solve problems that are language specific
> (since those tools aren't language specific).
> 
[snip]

If your using emacs, etags might be a little more helpful than grep in certain 
cases.
http://www.gnu.org/manual/emacs-21.2/html_chapter/emacs_26.html#SEC334

Or ebrowse, which is specifically for C++, but I haven't ever used it.
http://www-es.fernuni-hagen.de/cgi-bin/info2html?(ebrowse)Top

If your using emacs, seems you might be using gcc as well.  It's always a good 
idea to turn on all warnings (and most people do) with -Wall.

There are a bunch of compilation switches that aren't covered by -Wall that I 
like to use:
-Weffc++ -Wold-style-cast -pedantic -Wunreachable-code -ansi

-Weffc++ catches a few of items (but not nearly enough) from Scott Meyer's 
Effective C++, which is a pretty cool concept.

I've never used gcov.
http://gcc.gnu.org/onlinedocs/gcc-3.1/gcc/Gcov.html#Gcov
Does anyone have any experience with it?  Seems like it might be able to find 
some of that "vestigial cruft".
I guess gprof, or any other good profiler, can be used to get the same effect.

Don't know how useful these primarily *nix tools will be since a lot of 
developers are probably sitting in Win32 land.

Parveen

RE: [GD-General] C++ analyzers?

[Tom F. <to...@mu...>]

PC-Lint is slightly eccentric (setup, manual, etc), but it's heart is in the
right place, and it does work. As it's motto says "noisy at first, quietens
down after a while". Very very simple to tell it to ignore stuff you don't
care about. Don't worry too much about it filling a few gig of disk space
with warnings the first time you run it :-)


Tom Forsyth - purely hypothetical Muckyfoot bloke.

This email is the product of your deranged imagination,
and does not in any way imply existence of the author.

> -----Original Message-----
> From: Brian Hook [mailto:bri...@py...]
> Sent: 17 July 2002 20:00
> To: gam...@li...
> Subject: RE: [GD-General] C++ analyzers?
> 
> 
> I've used Doxygen and like it quite a bit, although getting it
> configured takes a bit of work (and a friend pointed out to me that
> apparently it doesn't work with ANSI C).
> 
> However, AFAIK it only documents, it doesn't actually generate many
> useful warnings unless it finds slightly mismatched prototypes.  You
> have to go through and examine the class graph to see 
> discrepancies, but
> unfortunately there are a lot of root nodes since all base classes are
> siblings (so if you have a lot of structs, it gets cluttered, 
> quickly).
> It also doesn't differentiate between locally defined (file 
> scope) data
> structures.
> 
> I'll check out PC-Lint though.
> 
> Thanks,
> 
> Brian
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557
> 

RE: [GD-General] C++ analyzers?

[Brian H. <bri...@py...>]

I've used Doxygen and like it quite a bit, although getting it
configured takes a bit of work (and a friend pointed out to me that
apparently it doesn't work with ANSI C).

However, AFAIK it only documents, it doesn't actually generate many
useful warnings unless it finds slightly mismatched prototypes.  You
have to go through and examine the class graph to see discrepancies, but
unfortunately there are a lot of root nodes since all base classes are
siblings (so if you have a lot of structs, it gets cluttered, quickly).
It also doesn't differentiate between locally defined (file scope) data
structures.

I'll check out PC-Lint though.

Thanks,

Brian

Re: [GD-General] C++ analyzers?

[Thatcher U. <tu...@tu...>]

On Jul 17, 2002 at 11:34 -0700, Brian Hook wrote:
>
[snip]
> 
> And, I'm sure, there are a million other things it could check for that
> currently require manual intervention.  Anyone know if such a tool
> exists?

PC-Lint exists; claims to do lots of the stuff you mentioned, and more:

http://www.gimpel.com/html/lintinfo.htm

I've never used it.  Also I think people sometimes use Doxygen to do
some of what you mentioned.

-- 
Thatcher Ulrich
http://tulrich.com

[GD-General] C++ analyzers?

[Brian H. <bri...@py...>]

One of my bitches about software development are that the tools suck.
Many things that can and should be automated simply aren't.  Tools like
Emacs and make provide a lot of helpers when you have to do this manual
stuff, but they don't actually solve problems that are language specific
(since those tools aren't language specific).

When working on a large codebase, invariably it acquires several layers
of cruft and detritus.  Several Java IDEs provide high level tools to do
things like refactor, automatically name types and objects, etc.  This
can be immensely useful.

Now that I'm parked on about 120K of source code that I've written,
there are times when I want to issue a "query" about some type or
variable.

Of course, that query ends up being "grep", which completely and utterly
sucks when you have lots of name collisions.  There are certain variable
and function names that are so ubiquitous that grepping for them returns
a thousand hits.  init/create/alloc and shutdown/destroy/free; flags,
type, name, next, prev, id; etc.

One way I've gotten around this is defensive coding, whereby I prefix
members with their parent type name:

struct HSprite
{
   int iSpriteWidth; //instead of iWidth
};

Anyone that has used an old K&R style compiler that didn't consider the
type and global namespaces differently will probably have done something
similar out of necessity.

It should be entirely possible for a code analyzer to go through and
build a dependency and call graph, with knowledge of the framework's
inheritance, friendship and polymorphism, and answer queries or generate
warnings.

When doing spring cleaning on my code base, some of the examples of
things that I know the compiler/linker could do but simply don't (and
which requires lots of searching for me) are:

- warn when including header files that aren't needed.  A typical
example is when you're doing some debugging and you need sprintf() or
FILE so you #include <stdio.h>, but forget to remove it when you're
done.

- warn when forward declarations aren't used or aren't relevant, e.g.
you do "class Foo" which you've later renamed to "class FooBar", but you
still have forward declarations to a non-existent class (which don't
cause problems)

- warn when friendships aren't necessary.  This happens when you
establish a friendship from A to B because you don't want
B::privateThing accessible by the world, just by A.  But then later on
you change some stuff, and now A doesn't need B::privateThing, but it's
still a friend of the class.

- warn about types, classes, variables, and functions that are never
referenced.  This is pretty open ended, and can mean everything from
abstract base classes that are never derived from to concrete classes
that are never instantiated, but it helps to find vestigial cruft.  This
has a lot of problems if all you're doing is building a library because
you can't tell what the client program is calling, but if you're
building test programs, it's a good way of making sure you have
reasonable coverage analysis without actually running a coverage
analysis program.

And, I'm sure, there are a million other things it could check for that
currently require manual intervention.  Anyone know if such a tool
exists?

-Hook

RE: [GD-General] UDP questions

[Jaek S. <smi...@cs...>]

> >   - It may be corrupt
> 
> This I'm not sure about -- isn't a CRC check part of the UDP and TCP
> protocols, or is this something that every application is expected to
> implement?  I know that some application protocols do CRC (e.g. Quake)
> but I was under the impression this was more to stop cheating, not to
> verify that the data wasn't corrupted.

TCP definately is guarenteed data-wise, order-wise, etc.

UDP, however, uses only a quick CheckSum - which, I believe, can be turned off.

With a quick search, here's a quote:

  "the Internet checksum is not foolproof ... it is still
  possible that there are undetected errors in the segment.
  For this reason, a number of protocols use more
  sophisticated error detection techniques than simple
  checksumming."
  ( http://www-net.cs.umass.edu/kurose/transport/UDP.html )

Also, another quote (from the same page):

  "Although UDP provides error checking, it does not
  do anything to recover from an error. Some implementations
  of UDP simply discard the damaged segment; others pass the
  damaged segment to the application with a warning. "
  ( http://www-net.cs.umass.edu/kurose/transport/UDP.html )

I can't guarentee the quality of the above information.  However, it matches my understanding from when I studied the TCP/IP protocol several years ago.  (If only my memory weren't volatile I could guarentee my answers also, heh).

About a year ago I found out about a protocol being developed (that I think was being build on top of IP - same layer as TCP and UDP) that did guarenteed messaging.  I can't remember the name - should see if I can find some info / status.  Streams are just so wrong in the world of software data communications. :P

Later for now
--Jaek

RE: [GD-General] RE: Gamedevlists-general digest, Vol 1 #79 - 3 msgs

[Jaek S. <smi...@cs...>]

> Thats the whole point of the article, to my eye.  Establishing a protocol
> such that the sender includes enough information that the reciever knows
> where the packet boundaries are, once their udp packets have been
> converted into a meaningless byte stream (which is what the buffering
> inside recv() does for you)

Datagrams are never really converted to a 'meaningless byte stream'.

Here's part of the man-page on recv():

  "All  three  routines return the length of the message on
  successful completion.  If a message is too long to fit in
  the supplied buffer, excess bytes may be discarded depending
  on the  type  of socket the message is received from..."

Thus, for a datagram (UDP), you would generally lose the excess amount.  Generally, you want to pass a buffer the size of your maximum sized message.  (Note: 64K should be the max possible size for UDP - if I recall correctly).

--Jaek

[GD-General] RE: Gamedevlists-general digest, Vol 1 #79 - 3 msgs

[Jeff L. <je...@SP...>]

> Given the feedback from you and others, however, it looks 
> either Beej's 
> comments are wrong, or what he defines as a "packet" is 
> something other 
> than a UDP packet (e.g. his own packet structure, etc.) and thus 
> possibly something that can become fragmented.  From looking at it, 
> though, it doesn't seem to be the case (i.e. the example he 
> gives of a 
> simple chat protocol shouldn't have a case where it fragments at the 
> application level).

The key here is that you are blurring UDP packets with "amount of data sent
to send()" or "amount of data that comes back from recv()".

Both of these routines buffer (which he does allude to in 6.3).

recv() might return an incomplete packet, because you pass it a length of
the amount of data you are prepared to read.  It doesn't return when it
reaches the end of a UDP packet, or when it reaches the end of what the user
passed to send().

Thats the whole point of the article, to my eye.  Establishing a protocol
such that the sender includes enough information that the reciever knows
where the packet boundaries are, once their udp packets have been converted
into a meaningless byte stream (which is what the buffering inside recv()
does for you)

RE: [GD-General] UDP questions

[Brian H. <bri...@py...>]

> > Define "legitimate"?
> 
> Legitimate is defined as a packet that was really generated 
> by the client or server, as opposed to spoofed or otherwise 
> crafted by a third party program sitting in the middle.

Okay, that's easy enough to deal with -- all my network traffic is
encrypted with a randomly generated session key that's delivered with
RSA encryption to the server.  On top of that I also have
username/password authentication, so doing a man-in-the-middle attack
should be pretty much impossible.

I was more worried that I would need to do another checksum just to make
sure that the data wasn't inadvertently corrupted.

Brian

RE: [GD-General] UDP questions

[Brian H. <bri...@py...>]

> calling recv for tcp will give you a certain amount of data.  
> it doesn't necessarily correspond to any sort of packet.  
> Remember, tcp is a stream protocol, so it doesn't understand 
> datagrams.

Ah-ha!  Okay, duh -- I've never actually even considered using TCP, so
that's where the confusion comes from.  So his tutorial was, in fact,
correct, it was just the context (TCP) that was confusing me.

> the protocol verifies that CRCs match for the packet.  That's 
> about it. Application protocols should verify their own CRCs 
> to make sure the packet is legitimate.

Define "legitimate"?

Brian

RE: [GD-General] UDP questions

[Brian H. <bri...@py...>]

> Taking an extremely quick look at the link you sent, it looks 
> like the methods he is showing are for a stream (TCP) socket, 
> and not a datagram (UDP) socket.

Yeah, he bounces around a bit between the two.

I'm leery of saying the guy is wrong, because the guide is extremely
comprehensive -- I don't see how you could write something that big and
in-depth and screw up something as fundamental as UDP packet delivery
constraints.  So I was assuming I didn't understand what he was saying
(entirely likely).

> Calling recv will receive a single, complete datagram.  

Which his comments contradict.  This would seem _especially_ true of a
TCP implementation.

> However, the following may be true (does someone want to 
> verify these):
> 
>   - It may not be unique

True.

>   - It may be corrupt

This I'm not sure about -- isn't a CRC check part of the UDP and TCP
protocols, or is this something that every application is expected to
implement?  I know that some application protocols do CRC (e.g. Quake)
but I was under the impression this was more to stop cheating, not to
verify that the data wasn't corrupted.

>   - It may be out of order

Definitely.

Brian

RE: [GD-General] UDP questions

[Jaek S. <smi...@cs...>]

> Given the feedback from you and others, however, it looks either 
> Beej's
> comments are wrong, or what he defines as a "packet" is something other 
> than a UDP packet (e.g. his own packet structure, etc.) and thus 
> possibly something that can become fragmented.  From looking at it, 
> though, it doesn't seem to be the case (i.e. the example he gives of a 
> simple chat protocol shouldn't have a case where it fragments at the 
> application level).

Taking an extremely quick look at the link you sent, it looks like the methods he is showing are for a stream (TCP) socket, and not a datagram (UDP) socket.  Ex:

   if ((listener = socket(AF_INET, SOCK_STREAM, 0)) == -1) {

Thus... he is probably doing a message-over stream based protocol.

Calling recv will receive a single, complete datagram.  However, the following may be true (does someone want to verify these):

  - It may not be unique
  - It may be corrupt
  - It may be out of order

Later for now
--Jaek

Re: [GD-General] UDP questions

[Mads B. D. <ma...@ch...>]

On Sun, 30 Jun 2002, brian hook wrote:

> > What are your sources?
> 
> Well, my two sources are the following (via gamedev.net) 
> and "everything else I've ever read" (i.e. Stevens, WinSock references, 
> etc.).
> 
> http://www.ecst.csuchico.edu/~beej/guide/net/html/advanced.html#sendall
> 
> In the above, which is written in a fairly authoritative tone, there 
> are a couple pieces that made it sound like you could receive packet 
> fragments instead of entire packets.  That's the part that confused me.

Thankfully, I read it as Stevens agress with what I wrote. I would trust 
him. I would even trust the winsock docs in this case.... :-)

Maybe someone else have another 2 cent?

Mads

-- 
Mads Bondo Dydensborg.                               ma...@ch...
There has never been a good war or a bad peace.
                           - Benjamin Franklin

Re: [GD-General] UDP questions

[brian h. <bri...@py...>]

> What are your sources?

Well, my two sources are the following (via gamedev.net) 
and "everything else I've ever read" (i.e. Stevens, WinSock references, 
etc.).

http://www.ecst.csuchico.edu/~beej/guide/net/html/advanced.html#sendall

In the above, which is written in a fairly authoritative tone, there 
are a couple pieces that made it sound like you could receive packet 
fragments instead of entire packets.  That's the part that confused me.

Specific excerpts:

"Likewise, when you're receiving this data, you need to do a bit of 
extra work. To be safe, you should assume that you might receive a 
partial packet....We need to call recv() over and over again until the 
packet is completely received.
.
.
.
Every time you recv() data, you'll feed it into the work buffer and 
check to see if the packet is complete. 
.
.
.
Well, here's the second of the one-two punch: you might have read past 
the end of one packet and onto the next in a single recv() call. That 
is, you have a work buffer with one complete packet, and an incomplete 
part of the next packet!"

Given the feedback from you and others, however, it looks either Beej's 
comments are wrong, or what he defines as a "packet" is something other 
than a UDP packet (e.g. his own packet structure, etc.) and thus 
possibly something that can become fragmented.  From looking at it, 
though, it doesn't seem to be the case (i.e. the example he gives of a 
simple chat protocol shouldn't have a case where it fragments at the 
application level).

Brian

Re: [GD-General] UDP questions

[Mads B. D. <ma...@ch...>]

On Sat, 29 Jun 2002, Brian Hook wrote:

> I'm working on a basic UDP implementation right now and I'm running into 
> conflicting information on packet fragmentation.

What are your sources?

> 
> My understanding of UDP is that it's unreliable, unguaranteed, and 
> basically not going to get you data in any particular order or even at all.

Same here.

> Fine, I can deal with that.
> 
> BUT, if it DOES get you data, that data is supposed to be "complete".  In 
> other words, if Server sends a packet of 500 bytes to Client, then Client 
> will either receive that packet or none at all when it calls recv() 
> (assuming that the recv() buffer is large enough).  Yes?

That is my understanding: the fragmentation you are worrying about is on 
the IP level, ie, the UDP packet _might_ be fragmented into several IP 
packets. The IP layer should handle this, iff all packets arrive. If not, 
you loose the UDP packet.

> If I get get a positive return value from recvfrom(), can I assume that 
> it's a single complete packet?  

For UDP, yes, this is my understanding.

> Is it possible that I'll receive multiple 
> packets in a single call to recvfrom()?  Is it possible to receive a 
> partial packet?

Don't think so.

> I know that send() can deliver less than the number of bytes you request, 
> but that's easy enough to handle.

I am unsure how send() works with UDP packets. I would assume that it is 
ready to handle all valid UDP packet sizes.

Mads

-- 
Mads Bondo Dydensborg.                               ma...@ch...
On the other hand, we can agree with Microsoft that the GPL is bad for
their current business. We can then proceed to use Microsoft's favorite word
as we reply: Innovation won't stop just because you're not ready for it. The
printing press was a good thing, after all, even though it forced
professional scribes to change their business model. Adapt or die
                               - Chip Salzenberg

[GD-General] UDP questions

[Brian H. <bri...@py...>]

I'm working on a basic UDP implementation right now and I'm running into 
conflicting information on packet fragmentation.

My understanding of UDP is that it's unreliable, unguaranteed, and 
basically not going to get you data in any particular order or even at all.

Fine, I can deal with that.

BUT, if it DOES get you data, that data is supposed to be "complete".  In 
other words, if Server sends a packet of 500 bytes to Client, then Client 
will either receive that packet or none at all when it calls recv() 
(assuming that the recv() buffer is large enough).  Yes?

I know that Server->Client can result in the packet becoming disassembled, 
fragmented and munged along the way by various routers (or even by the 
server's network layer), but I was assuming that the packet reconstruction 
in the socket driver was reasonably advanced -- i.e. it was responsible for 
making sure that a fully constructed and CRC-verified packet was actually 
sent down to the Client's socket (if at all).

If I get get a positive return value from recvfrom(), can I assume that 
it's a single complete packet?  Is it possible that I'll receive multiple 
packets in a single call to recvfrom()?  Is it possible to receive a 
partial packet?

If so, this means that my simple code is now going to be gross and nasty -- 
specifically, I'm going to have to create and maintain intermediate work 
buffers for every client connected to my server. Blech.

I know that send() can deliver less than the number of bytes you request, 
but that's easy enough to handle.

Anyway, before I get too far into my networking layer, I just want to make 
sure that I don't A.) overengineer for situations that won't happen or B.) 
make something that's not robust.

Thanks,

Brian

Re: [GD-General] Scaling GUI graphics

[Thatcher U. <tu...@tu...>]

On Jun 25, 2002 at 12:02 -0700, Brian Sharon wrote:
> 
> I'm ignorant of the legal issues involved, but it seems like it should
> be technically possible to process a TTF font into a bitmap font via
> freetype and end up with something that you can use in your game...

If I'm not mistaken, bitmap images of fonts are not copyrightable.
I.e. the ttf files are copyrighted, but you can do whatever you want
with any images you make by rendering text using that font.

IANAL though...

-- 
Thatcher Ulrich
http://tulrich.com

RE: [GD-General] Scaling GUI graphics

[<phi...@pl...>]

We do all our front-end / HUD stuff in Flash, although our scripting
support is limited to Flash 4, and there's a couple of rendering things we
don't do, either for performance, or 'bang for buck' issues. IIRC some of
our game logic was in it for a while (I certainly recall having to read
state changes back out of the flash interface class).

We're doing this for authoring reasons, rather than re-sizing issues
though. Closest we get to resolution changes are supporting 4:3 / 16:9 and
PAL / NTSC.

Cheers,
Phil

PS I'd like to point out that we did use flash during run-time, and still
maintained a solid 60 fps.



                                                                                                                                  
                    "Paul Bleisch" <pa...@mi...>                                                                        
                    Sent by:                                     To:     <gam...@li...>             
                    gam...@li...urc       cc:                                                              
                    eforge.net                                   Subject:     RE: [GD-General] Scaling GUI graphics               
                                                                                                                                  
                                                                                                                                  
                    06/25/2002 07:10 AM                                                                                           
                                                                                                                                  
                                                                                                                                  





We evaluated Flash for our gui/frontend.  The runtime
(we looked at an off the shelf flash renderer) was a
bit heavyweight for what we wanted, but one of the
guys here has an almost functional SVG renderer that
we might use in the future.

> -----Original Message-----
> From: Tom Forsyth [mailto:to...@mu...]
> Sent: Tuesday, June 25, 2002 8:56 AM
> To: gam...@li...
> Subject: RE: [GD-General] Scaling GUI graphics
>
>
> The other cute trick I've heard of is to bite the bullet and
> put Flash in the game. Then the artists can do front ends and
> GUIs all they like, and the graphics coders don't have to be
> involved at all, so we can get on with the actual 3D stuff.
> Which appeals to me, seeing as the last fancy GUI we did
> chewed 6 months of a programmer's time. That was bad.
>
> Tom Forsyth - purely hypothetical Muckyfoot bloke.
>
> This email is the product of your deranged imagination,
> and does not in any way imply existence of the author.
>
> > -----Original Message-----
> > From: Kent Quirk [mailto:ken...@co...]
> > Sent: 25 June 2002 14:59
> > To: Tom Forsyth
> > Cc: Brian Hook; gam...@li...
> > Subject: Re: [GD-General] Scaling GUI graphics
> >
> >
> > We just built a skinnable GUI for a game we're doing. All
> the graphics
> > in the game are built from individual components at load time, with
> > the layout described in an XML file. We haven't supported
> any form of
> > scaling, as it's my experience that scaling always looks like hell.
> > But we do support tiling, so what we do is build a
> background texture
> > that's seamlessly tileable, and lay that down first. Then we place
> > interface elements on top of that.
> >
> > It let me use mixed file formats (highly compressed jpgs for the
> > background, GIFs for basic interface elements, and PNGs
> where I needed
> > transparency), and in addition to increased flexibility, we
> took about
> > 1.5 meg of interface graphics and compressed it to less than 300K.
> >
> >     Kent
> >
> >
> > Tuesday, June 25, 2002, 6:19:55 AM, you wrote:
> >
> > TF> My favourite solution is to scale, but only by powers of
> > two. So you design
> > TF> GUI graphics for 640x480. For 800x600 and 1024x768, you
> > don't scale, you
> > TF> just move them further apart (i.e. the bits stick to the
> > cornr, edge or
> > TF> centre as required). For 1280x1024 and above, you double
> > the size so that at
> > TF> least stuff is readable.
> >
> > TF> To do the scale x2 you can either just do the bilinear
> > filter thing (which
> > TF> doesn't look too bad on power-of-two scales), or just get
> > the artists to do
> > TF> second versions. Or mix'n'match (e.g. hand-done bigger
> > font, but scaled
> > TF> backgrounds).
> >
> > TF> Tom Forsyth - purely hypothetical Muckyfoot bloke.
> >
> > TF> This email is the product of your deranged imagination,
> and does
> > TF> not in any way imply existence of the author.
> >
> > >> -----Original Message-----
> > >> From: Brian Hook [mailto:bri...@py...]
> > >> Sent: 25 June 2002 04:23
> > >> To: gam...@li...
> > >> Subject: [GD-General] Scaling GUI graphics
> > >>
> > >>
> > >> Okay, this is a common problem that I'm sure others have
> > >> dealt with on
> > >> numerous occasions, but I have yet to deal with it so I'm
> > >> curious what the
> > >> conventional wisdom is.
> > >>
> > >> Basically it's the age old problem of having an interface
> > >> that looks good
> > >> at all resolutions.  AFAICT, the standards are:
> > >>
> > >> - do low res graphics for everything, then zoom appropriately
> > >> (leads to
> > >> serious blurring)
> > >>
> > >> - do multiple resolution graphics (lots of content)
> > >>
> > >> - render GUI graphics into textures/images instead of using
> > >> prefab images
> > >> (i.e. actually write line/circle/filled region software
> renderers)
> > >>
> > >> - don't scale, just make everything smaller (8x8 font at
> > >> 1600x1200...ewwww)
> > >>
> > >> Anyone find a solution that doesn't have a glaring flaw,
> > unlike the
> > >> above?  I suppose the closest I can think of is to do
> > >> everything in high
> > >> resolution, e.g. for a 1600x1200 interface, and then resample
> > >> down for
> > >> lower resolution displays.
> > >>
> > >> Brian
> > >>
> > >>
> > >>
> > >> -------------------------------------------------------
> > >> Sponsored by:
> > >> ThinkGeek at http://www.ThinkGeek.com/
> > >> _______________________________________________
> > >> Gamedevlists-general mailing list
> > >> Gam...@li...
> > >> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> > >> Archives:
> http://sourceforge.net/mailarchive/forum.php?> forum_id=557
> > >>
> >
> >
> > TF> -------------------------------------------------------
> > TF> Sponsored by:
> > TF> ThinkGeek at http://www.ThinkGeek.com/
> > TF> _______________________________________________
> > TF> Gamedevlists-general mailing list
> > TF> Gam...@li...
> > TF>
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-gene
ral
> TF> Archives:
> TF> http://sourceforge.net/mailarchive/forum.php?forum_id=557
>
>
> --
> Kent Quirk, CTO, CogniToy
> ken...@co...
> http://www.cognitoy.com
>


-------------------------------------------------------
Sponsored by:
ThinkGeek at http://www.ThinkGeek.com/
_______________________________________________
Gamedevlists-general mailing list
Gam...@li...
https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_id=557


-------------------------------------------------------
Sponsored by:
ThinkGeek at http://www.ThinkGeek.com/
_______________________________________________
Gamedevlists-general mailing list
Gam...@li...
https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_idU7

RE: [GD-General] Scaling GUI graphics

[Brian S. <bs...@mi...>]

> > > FreeType (http://www.freetype.org) makes high quality output; it's
> > > what XFree86 uses these days.
> >=20
> > Cool, I'll check it out, thanks.
> >=20
> > I use that one would need a freely redistributable TTF=20
> fonts (are the=20
> > Windows ones in the free and clear?  I doubt it).
>=20
> Heh, I think you're right.  MS's web fonts are good quality and free
> as in beer, but I imagine there are limits on redistribution.  Debian
> Linux actually includes a package that downloads the fonts from MS's
> web site and installs them, so the fonts themselves aren't actually in
> the distro.  That's probably too complicated an approach to rely on
> for a mass market game though.
>=20
> What we did in Soul Ride, which worked well I think, was the artist
> made antialiased bitmapped fonts in Photoshop (from TrueType source
> fonts), and we rendered using alpha blending.  Scaling worked fine;
> the antialiasing will obviously work better if you don't scale too
> radically, but over a factor of 2 or 3x, they looked pretty nice IMO.

I've done this in the past too, and it looks pretty good.  The downside
is that if you just render a string by rendering individual characters,
you don't get nice things like pair kerning and ligature substitution
that make your text look really high-quality...still, for the average
game that's overkill.

I'm ignorant of the legal issues involved, but it seems like it should
be technically possible to process a TTF font into a bitmap font via
freetype and end up with something that you can use in your game...

--brian

Re: [GD-General] Scaling GUI graphics

[Thatcher U. <tu...@tu...>]

On Jun 25, 2002 at 11:09 -0500, brian hook wrote:
> > Note: you don't have to render into a bitmap -- you could render
> > directly using OpenGL/D3D.  Dunno if you're already assuming a 3D API
> > though.
> 
> Did you mean glLine/glPoint calls?

Not sure what glLine is, but yeah,
glBegin(GL_{POINTS,LINES,LINE_STRIP,LINE_LOOP}) or whatever.  Or use
artist-painted UI elements, and scale/tile them using triangles/quads
(that's what we did in Soul Ride).

> > FreeType (http://www.freetype.org) makes high quality output; it's
> > what XFree86 uses these days.
> 
> Cool, I'll check it out, thanks.
> 
> I use that one would need a freely redistributable TTF fonts (are the 
> Windows ones in the free and clear?  I doubt it).

Heh, I think you're right.  MS's web fonts are good quality and free
as in beer, but I imagine there are limits on redistribution.  Debian
Linux actually includes a package that downloads the fonts from MS's
web site and installs them, so the fonts themselves aren't actually in
the distro.  That's probably too complicated an approach to rely on
for a mass market game though.

What we did in Soul Ride, which worked well I think, was the artist
made antialiased bitmapped fonts in Photoshop (from TrueType source
fonts), and we rendered using alpha blending.  Scaling worked fine;
the antialiasing will obviously work better if you don't scale too
radically, but over a factor of 2 or 3x, they looked pretty nice IMO.

The MS fonts are the only scalable fonts that don't look like total
crap on my Linux box, so I suspect you won't find good free hinted TT
fonts.  Although antialiasing would help.

One of my super back-burner projects is a public domain Flash
parser/renderer.

-- 
Thatcher Ulrich
http://tulrich.com