qstat-devx

[QStat-devx] draft for new query protocol and API

[Ludwig N. <l-...@us...>]

Hi,

I wrote some specifications for a new query protocol as well as a
header file for a library I plan to write (no real code yet). You
can find them them at http://xqf.sourceforge.net/libqquery/.

Could you please have a look and comment at least on the protocol
specification (any ambiguities or other flaws)? If the protocol fits
our needs from qstat's point of view, I'll write a reference
implementation for BZFlag.

cu
Ludwig

-- 
(o_     Ludwig Nussel
//\   ICQ:     52166811
V_/_  PGP Key: FF8135CE

Re: [QStat-devx] draft for new query protocol and API

[Steve J. <ste...@ya...>]

Thanks, Ludwig.  I've made some comments on the package format.

But first, I'm not convinced the world needs another game query
protocol.  If you implement something for BZFlag that's compatible
with an existing game (HL or Quake), then other game browsers can
support it straight away.  That's good for BZFlag, right?

See my comments below on the query protocol.

I could comment on the API, but without knowing the architecture of
the application (BZFlag) I can't say if the API meets the needs of
the app.  From scanning the API, I imagine that the API might be
perfect, or it might be intrusive.  Depends on the BZFlag data
structures and main control flow.

If you go ahead with the API, let me know and I'll give you some feedback
on the API.  Just for giggles if nothing else.

Steve


> Query:
> ~~~~~~
> 
> QUERY ::= HEADER QUERYDATA
> 
> HEADER ::= 'q' 'c' PROTOVERS PKGID
> 
> PROTOVERS ::= uint8
> 
> PKGID ::= uint8
> 
> QUERYDATA ::= string
> 
> Query response:
> ~~~~~~~~~~~~~~~
> 
> QUERYRESPONSE ::= HEADER MANDDATA '\n' (RULE)* '\n' (TEAM)* '\n' (PLAYER)*
'\n'

I'm concerned about using \n since it might appear in a string (rule value).
I suggest using \0 since that's not a legal character.

> 
> HEADER ::= 'q' 's' PROTOVERS PKGID FRAGMENTNO (DATALEN)?
> 
> PROTOVERS ::= uint8
> 
> PKGID ::= uint8
> 
> FRAGMENTNO ::= uint8
> 
> DATALEN ::= uint32
> 
> MANDDATA ::= CURPLAYERS MAXPLAYERS PRODUCT SERVERNAME MAP
> 
> CURPLAYERS ::= uint32
> 
> MAXPLAYERS ::= uint32
> 
> PRODUCT ::= string
> 
> SERVERNAME ::= string
> 
> MAP ::= string
> 
> RULE ::= KEY VALUE
> 
> TEAM ::= KEY VALUE
> 
> PLAYER ::= KEY VALUE
> 
> KEY ::= string
> 
> VALUE ::= string
> 
> 
> 
> integers are in network byte order
> strings are null terminated C-strings in UTF-8 or iso8859-1 encoding

How do you distinguish between encodings?

> 
> PKGID has to be copied from the query packet to the query response packet by
> the server, it is used by the client to distinguish retry packets
> 
> DATALEN is only present if FRAGMENTNO == 0

Optional bits of data are annoying.  It's only a few extra bytes.

On the topic of DATALEN, I'd rather know the total number of packets
to expect.  Once I have all the packets, then I'll know how much data
there is; I don't have to be told.  The trouble is knowing if I got all
the packets.  If each packets tells me the total # packets, then it's
easier.

Change the header to:

HEADER ::= 'q' 's' PROTOVERS PKGID TOTALFRAGMENTS FRAGMENTNO

The API should build the whole packet, then divide by 1394 (1400 minus
the 6 byte header).  Then it will know how many fragments there will
be and can put this in each frag header.

I chose 1400 to avoid exceeding the MTU (usually 1500 on ethernet, but
occasionally smaller).

> the first key of each player or team must be "name"
> 
> PROTOVERS is 0x01 in this revision of the protocol
> 
> QUERYDATA is "sendreport"

Maybe allow fetching of different data sets:

"send:"          send just the MANDDATA
"send:all"       send everything
"send:rules"     send rules
"send:players"   send players
"send:teams"     send teams
"send:teams,players"    send just teams and players

Of course, MANDDATA is always included.

For now, you can support just "send:" and "send:all", but you've left
yourself room for expansion to support other queries.

The "send:" query is useful for collecting "ping" stats.  Pings are most
accurate when the data is small.

> 
> 
> 
> TODO:
> 
> - use fragment offset instead of fragment number?
> - are all integers of appropriate size?

32bits is overkill for most of the values.  DATALEN, CURPLAYER, and
MAXPLAYER can all be uint16 and still be more than enough.

Re: [QStat-devx] draft for new query protocol and API

[Ludwig N. <l-...@us...>]

Steve Jankowski wrote:
> Thanks, Ludwig.  I've made some comments on the package format.
> 
> But first, I'm not convinced the world needs another game query
> protocol.  If you implement something for BZFlag that's compatible
> with an existing game (HL or Quake), then other game browsers can
> support it straight away.  That's good for BZFlag, right?

Those protocols are flawed in one way or another. AFAIK HL does not
deal with fragmented packets and Quake(3) only has a fixed set of
player properties. The gamespy format is some kind of standard, but
we know it's not easy to deal with either.

> See my comments below on the query protocol.
> 
> I could comment on the API, but without knowing the architecture of
> the application (BZFlag) I can't say if the API meets the needs of
> the app.  From scanning the API, I imagine that the API might be
> perfect, or it might be intrusive.  Depends on the BZFlag data
> structures and main control flow.

I don't think the API doesn't assume any special preconditions. You
just tell it when some state of your server changed (rule changes,
player joins, frags, dies etc). On every incoming packet (that maybe
does not contain your own magic header) you have to call
qs_respond_to_query() to form a query response if needed.

> If you go ahead with the API, let me know and I'll give you some feedback
> on the API.  Just for giggles if nothing else.

I'd really like to hear your opinion.

> > [...]
> > Query response:
> > ~~~~~~~~~~~~~~~
> > 
> > QUERYRESPONSE ::= HEADER MANDDATA '\n' (RULE)* '\n' (TEAM)* '\n' (PLAYER)*
> '\n'
> 
> I'm concerned about using \n since it might appear in a string (rule value).
> I suggest using \0 since that's not a legal character.

Strings are always null terminated. \n is used to let the parser
know when the e.g. the rules section ends and the team section
starts. Like:

... 'r' 'u' 'l' 'e' '\0' 'v' 'a' 'l' 'u' 'e' \0' '\n' ...

> > [...]
> > integers are in network byte order
> > strings are null terminated C-strings in UTF-8 or iso8859-1 encoding
> 
> How do you distinguish between encodings?

You try to read a string in UTF-8. If that fails because it violates
the rules, it's assumed to be iso8859-1. I do this in XQF already
when using GTK2 for the UI. See
http://developer.gnome.org/doc/API/2.0/glib/glib-Unicode-Manipulation.html#g-utf8-validate

> > PKGID has to be copied from the query packet to the query response packet by
> > the server, it is used by the client to distinguish retry packets
> > 
> > DATALEN is only present if FRAGMENTNO == 0
> 
> Optional bits of data are annoying.  It's only a few extra bytes.
> 
> On the topic of DATALEN, I'd rather know the total number of packets
> to expect.  Once I have all the packets, then I'll know how much data
> there is; I don't have to be told.  The trouble is knowing if I got all
> the packets.  If each packets tells me the total # packets, then it's
> easier.

That's the part I'm unsure about. I specified it this way because
the API does not know how big each fragment can be. So it just
generates as much as needed on demand. The receiver knows how much
data to expect. As long as the sum of all received packets is
smaller than that it has to wait a little longer for more packet to
arrive.

> Change the header to:
> 
> HEADER ::= 'q' 's' PROTOVERS PKGID TOTALFRAGMENTS FRAGMENTNO
> 
> The API should build the whole packet, then divide by 1394 (1400 minus
> the 6 byte header).  Then it will know how many fragments there will
> be and can put this in each frag header.
> 
> I chose 1400 to avoid exceeding the MTU (usually 1500 on ethernet, but
> occasionally smaller).

If you want to make sure the packet can be tansmitted over the
internet you would need to restrict the packet size to 576 bytes.
That's the minimum size each device needs to be able to handle.

> > the first key of each player or team must be "name"
> > 
> > PROTOVERS is 0x01 in this revision of the protocol
> > 
> > QUERYDATA is "sendreport"
> 
> Maybe allow fetching of different data sets:
> 
> "send:"          send just the MANDDATA
> "send:all"       send everything
> "send:rules"     send rules
> "send:players"   send players
> "send:teams"     send teams
> "send:teams,players"    send just teams and players
> 
> Of course, MANDDATA is always included.
> 
> For now, you can support just "send:" and "send:all", but you've left
> yourself room for expansion to support other queries.

Maybe single bytes after the colon are better, no additional string
compares would be needed. E.g. send:a for all or send:tp for teams
and players.

cu
Ludwig

-- 
(o_     Ludwig Nussel
//\   ICQ:     52166811
V_/_  PGP Key: FF8135CE

Re: [QStat-devx] draft for new query protocol and API

[Steve J. <st...@qs...>]

--- Ludwig Nussel <l-...@us...> wrote:
> Steve Jankowski wrote:
> > Thanks, Ludwig.  I've made some comments on the package format.
> > 
> > But first, I'm not convinced the world needs another game query
> > protocol.  If you implement something for BZFlag that's compatible
> > with an existing game (HL or Quake), then other game browsers can
> > support it straight away.  That's good for BZFlag, right?
> 
> Those protocols are flawed in one way or another. AFAIK HL does not
> deal with fragmented packets and Quake(3) only has a fixed set of
> player properties. The gamespy format is some kind of standard, but
> we know it's not easy to deal with either.

Sure they're flawed, but do we need a perfect protocol just to support
BZFlag?  HL does support fragmented packets (some CS servers have entire
packets full of server rules).

> 
> > See my comments below on the query protocol.
> > 
> > I could comment on the API, but without knowing the architecture of
> > the application (BZFlag) I can't say if the API meets the needs of
> > the app.  From scanning the API, I imagine that the API might be
> > perfect, or it might be intrusive.  Depends on the BZFlag data
> > structures and main control flow.
> 
> I don't think the API doesn't assume any special preconditions. You
> just tell it when some state of your server changed (rule changes,
> player joins, frags, dies etc). On every incoming packet (that maybe
> does not contain your own magic header) you have to call
> qs_respond_to_query() to form a query response if needed.
> 
> > If you go ahead with the API, let me know and I'll give you some feedback
> > on the API.  Just for giggles if nothing else.
> 
> I'd really like to hear your opinion.

I haven't done a thorough review, but I thought of a couple things.

Memory allocations.  Some apps may use their own memory allocator.
Consider providing APIs to set the malloc and free functions used
by the API.

Multiple outstanding requests.  Given that a packet write is a non-
blocking operation, and a game server is likely to prioritize status
responses below actual game processing, the server could be sending
status responses to multiple destinations at the same time.  It seems
like your API can support this, but it requires the app to track the
state information.  If the app can only send 2 of 4 fragments, then it
has to save the IP, port, queryresponse, fragnr, and reslen in some
list.  The app will then need some code to occasionally run the list
to try sending more query responses.  Seems like the API should take
care of all these details.

while (1)  {
    char packet[4096];
    unsigned int ip;
    unsigned int port;
    int len;
    len= getPacketFromNetwork( packet, &ip, &port);
    if ( qs_is_query( packet, len))  {
        qs_start_query_response( packet, len, ip, port);
    }

    while ( nothingBetterToDo())  {
        char *response;
        int response_len;
        response_len= qs_next_query_response( &response, &response_len, &ip,
&port);
        if ( response_len)  {
            if ( sendPacket( response, response_len, ip, port) != -1)
                 qs_sent_packet( response);
            else
                 break;
        }
        else
            break;
    }

    // do game processing

}

The API would keep track of all queries, their destination, and their
current response state (how many fragments have been sent, etc).  This
reduces the burden on the app writer and gives the API more control.


> > > [...]
> > > Query response:
> > > ~~~~~~~~~~~~~~~
> > > 
> > > QUERYRESPONSE ::= HEADER MANDDATA '\n' (RULE)* '\n' (TEAM)* '\n'
> (PLAYER)*
> > '\n'
> > 
> > I'm concerned about using \n since it might appear in a string (rule
> value).
> > I suggest using \0 since that's not a legal character.
> 
> Strings are always null terminated. \n is used to let the parser
> know when the e.g. the rules section ends and the team section
> starts. Like:
> 
> ... 'r' 'u' 'l' 'e' '\0' 'v' 'a' 'l' 'u' 'e' \0' '\n' ...

Ok, what if a rule name starts with a \n ?   Seems unlikely, but
if you're going to make the perfect protocol, might as well avoid
all ambiguity.

> > > [...]
> > > integers are in network byte order
> > > strings are null terminated C-strings in UTF-8 or iso8859-1 encoding
> > 
> > How do you distinguish between encodings?
> 
> You try to read a string in UTF-8. If that fails because it violates
> the rules, it's assumed to be iso8859-1. I do this in XQF already
> when using GTK2 for the UI. See
>
http://developer.gnome.org/doc/API/2.0/glib/glib-Unicode-Manipulation.html#g-utf8-validate

Sounds good.

> 
> > > PKGID has to be copied from the query packet to the query response packet
> by
> > > the server, it is used by the client to distinguish retry packets
> > > 
> > > DATALEN is only present if FRAGMENTNO == 0
> > 
> > Optional bits of data are annoying.  It's only a few extra bytes.
> > 
> > On the topic of DATALEN, I'd rather know the total number of packets
> > to expect.  Once I have all the packets, then I'll know how much data
> > there is; I don't have to be told.  The trouble is knowing if I got all
> > the packets.  If each packets tells me the total # packets, then it's
> > easier.
> 
> That's the part I'm unsure about. I specified it this way because
> the API does not know how big each fragment can be. So it just
> generates as much as needed on demand. The receiver knows how much
> data to expect. As long as the sum of all received packets is
> smaller than that it has to wait a little longer for more packet to
> arrive.

Why doesn't the API know how big the fragments are?  Just supply a 1400
byte fragment and the app will either write the whole thing or it won't.
I don't think a UDP write can ever partially fail.  It will either
write the whole packet, or fail because the kernel packet or socket buffer
is full.  If apps want to tune the packet size, give them a single
config parameter:  qs_set_response_fragment_size( int n_bytes)

> 
> > Change the header to:
> > 
> > HEADER ::= 'q' 's' PROTOVERS PKGID TOTALFRAGMENTS FRAGMENTNO
> > 
> > The API should build the whole packet, then divide by 1394 (1400 minus
> > the 6 byte header).  Then it will know how many fragments there will
> > be and can put this in each frag header.
> > 
> > I chose 1400 to avoid exceeding the MTU (usually 1500 on ethernet, but
> > occasionally smaller).
> 
> If you want to make sure the packet can be tansmitted over the
> internet you would need to restrict the packet size to 576 bytes.
> That's the minimum size each device needs to be able to handle.

I'm really only worried about ethernet.  MTUs in the 500 range are
really only used by analog modems.  And no one is going to run a 
server behind a modem.

> 
> > > the first key of each player or team must be "name"
> > > 
> > > PROTOVERS is 0x01 in this revision of the protocol
> > > 
> > > QUERYDATA is "sendreport"
> > 
> > Maybe allow fetching of different data sets:
> > 
> > "send:"          send just the MANDDATA
> > "send:all"       send everything
> > "send:rules"     send rules
> > "send:players"   send players
> > "send:teams"     send teams
> > "send:teams,players"    send just teams and players
> > 
> > Of course, MANDDATA is always included.
> > 
> > For now, you can support just "send:" and "send:all", but you've left
> > yourself room for expansion to support other queries.
> 
> Maybe single bytes after the colon are better, no additional string
> compares would be needed. E.g. send:a for all or send:tp for teams
> and players.

Sure, sounds good.


Steve

Re: [QStat-devx] draft for new query protocol and API

[Ludwig N. <l-...@us...>]

Steve Jankowski wrote:
> --- Ludwig Nussel <l-...@us...> wrote:
> > Steve Jankowski wrote:
> > > But first, I'm not convinced the world needs another game query
> > > protocol.  If you implement something for BZFlag that's compatible
> > > with an existing game (HL or Quake), then other game browsers can
> > > support it straight away.  That's good for BZFlag, right?
> > 
> > Those protocols are flawed in one way or another. AFAIK HL does not
> > deal with fragmented packets and Quake(3) only has a fixed set of
> > player properties. The gamespy format is some kind of standard, but
> > we know it's not easy to deal with either.
> 
> Sure they're flawed, but do we need a perfect protocol just to support
> BZFlag?  HL does support fragmented packets (some CS servers have entire
> packets full of server rules).

It's not just for BZFlag, I hope to get this protocol into more
games so it has to be fairly generic. AFAICS HL too doesn't support
arbitrary player properties and transmits the time played a float.
(the code dealing with this in qstat breaks the strict aliasing
optimization of gcc3 btw.)

> Memory allocations.  Some apps may use their own memory allocator.
> Consider providing APIs to set the malloc and free functions used
> by the API.

Good point, easy.

> Multiple outstanding requests.  Given that a packet write is a non-
> blocking operation, and a game server is likely to prioritize status
> responses below actual game processing, the server could be sending
> status responses to multiple destinations at the same time.  It seems
> like your API can support this, but it requires the app to track the
> state information.  If the app can only send 2 of 4 fragments, then it
> has to save the IP, port, queryresponse, fragnr, and reslen in some
> list.  The app will then need some code to occasionally run the list
> to try sending more query responses.  Seems like the API should take
> care of all these details.

Isn't that's too complicated? Storing all that information probably
costs more CPU cycles than just calling a couple of send(). I didn't
want to deal with IP Adresses, otherwise it would be necessary to
support IPv4 and IPv6.

> > > I'm concerned about using \n since it might appear in a string (rule
> > value).
> > > I suggest using \0 since that's not a legal character.
> > 
> > Strings are always null terminated. \n is used to let the parser
> > know when the e.g. the rules section ends and the team section
> > starts. Like:
> > 
> > ... 'r' 'u' 'l' 'e' '\0' 'v' 'a' 'l' 'u' 'e' \0' '\n' ...
> 
> Ok, what if a rule name starts with a \n ?   Seems unlikely, but
> if you're going to make the perfect protocol, might as well avoid
> all ambiguity.

Right. That means if \0 is also used as delimiter between rules,
teams and players, a packet would look like

'r' 'u' 'l' 'e' '\0' 'v' 'a' 'l' 'u' 'e' \0' '\0' 'p' 'l' 'a' ...

So an empty string when e.g. a rule name is expected means switching
to player parsing. If no rules, players and teams are to be sent the
packet would be just '\0' '\0' '\0'

> Why doesn't the API know how big the fragments are?  Just supply a 1400
> byte fragment and the app will either write the whole thing or it won't.
> I don't think a UDP write can ever partially fail.  It will either
> write the whole packet, or fail because the kernel packet or socket buffer
> is full.  If apps want to tune the packet size, give them a single
> config parameter:  qs_set_response_fragment_size( int n_bytes)

Hmm, I'm still not convinced that the API needs to know the MTU
beforehand. qs_respond_to_query() returns a single string and the
game can call send() with whatever size it likes. If that size is
smaller than the string it just has to call qs_fragment_response()
until nothing is left. qs_fragment_response() updates the header and
does memmove() on the data.

> > > I chose 1400 to avoid exceeding the MTU (usually 1500 on ethernet, but
> > > occasionally smaller).
> > 
> > If you want to make sure the packet can be tansmitted over the
> > internet you would need to restrict the packet size to 576 bytes.
> > That's the minimum size each device needs to be able to handle.
> 
> I'm really only worried about ethernet.  MTUs in the 500 range are
> really only used by analog modems.  And no one is going to run a 
> server behind a modem.

What about clients? I suppose if you send a 1400 byte packet to an
IP behind a modem the packet gets dropped at the dialup router?

cu
Ludwig

-- 
(o_     Ludwig Nussel
//\   ICQ:     52166811
V_/_  PGP Key: FF8135CE

Re: [QStat-devx] draft for new query protocol and API

[Steve J. <ste...@ya...>]

--- Ludwig Nussel <l-...@us...> wrote:
> Steve Jankowski wrote:
> > --- Ludwig Nussel <l-...@us...> wrote:
> > > Steve Jankowski wrote:
> > > > But first, I'm not convinced the world needs another game query
> > > > protocol.  If you implement something for BZFlag that's compatible
> > > > with an existing game (HL or Quake), then other game browsers can
> > > > support it straight away.  That's good for BZFlag, right?
> > > 
> > > Those protocols are flawed in one way or another. AFAIK HL does not
> > > deal with fragmented packets and Quake(3) only has a fixed set of
> > > player properties. The gamespy format is some kind of standard, but
> > > we know it's not easy to deal with either.
> > 
> > Sure they're flawed, but do we need a perfect protocol just to support
> > BZFlag?  HL does support fragmented packets (some CS servers have entire
> > packets full of server rules).
> 
> It's not just for BZFlag, I hope to get this protocol into more
> games so it has to be fairly generic. AFAICS HL too doesn't support
> arbitrary player properties and transmits the time played a float.
> (the code dealing with this in qstat breaks the strict aliasing
> optimization of gcc3 btw.)

Yeah, you're right about HL being lame about the player info.
I was annoyed at HL using a float for connect time, so my solution
isn't pretty.


> > Multiple outstanding requests.  Given that a packet write is a non-
> > blocking operation, and a game server is likely to prioritize status
> > responses below actual game processing, the server could be sending
> > status responses to multiple destinations at the same time.  It seems
> > like your API can support this, but it requires the app to track the
> > state information.  If the app can only send 2 of 4 fragments, then it
> > has to save the IP, port, queryresponse, fragnr, and reslen in some
> > list.  The app will then need some code to occasionally run the list
> > to try sending more query responses.  Seems like the API should take
> > care of all these details.
> 
> Isn't that's too complicated? Storing all that information probably
> costs more CPU cycles than just calling a couple of send(). I didn't
> want to deal with IP Adresses, otherwise it would be necessary to
> support IPv4 and IPv6.

Supporting IPv6 is not required (and why would you?  there's no IPv6
games and game developers are not talking about it).

A good API implements all the complicated stuff so the app doesn't
have to.  If the API is too "thin", that presents a barrier to
adoption.  That complication has to be some where.  If it's in the
API, then every app benefits.  If it's in the app, then every
app gets to make the same mistakes.

> > > > I'm concerned about using \n since it might appear in a string (rule
> > > value).
> > > > I suggest using \0 since that's not a legal character.
> > > 
> > > Strings are always null terminated. \n is used to let the parser
> > > know when the e.g. the rules section ends and the team section
> > > starts. Like:
> > > 
> > > ... 'r' 'u' 'l' 'e' '\0' 'v' 'a' 'l' 'u' 'e' \0' '\n' ...
> > 
> > Ok, what if a rule name starts with a \n ?   Seems unlikely, but
> > if you're going to make the perfect protocol, might as well avoid
> > all ambiguity.
> 
> Right. That means if \0 is also used as delimiter between rules,
> teams and players, a packet would look like
> 
> 'r' 'u' 'l' 'e' '\0' 'v' 'a' 'l' 'u' 'e' \0' '\0' 'p' 'l' 'a' ...
> 
> So an empty string when e.g. a rule name is expected means switching
> to player parsing. If no rules, players and teams are to be sent the
> packet would be just '\0' '\0' '\0'

That's right.

> > Why doesn't the API know how big the fragments are?  Just supply a 1400
> > byte fragment and the app will either write the whole thing or it won't.
> > I don't think a UDP write can ever partially fail.  It will either
> > write the whole packet, or fail because the kernel packet or socket buffer
> > is full.  If apps want to tune the packet size, give them a single
> > config parameter:  qs_set_response_fragment_size( int n_bytes)
> 
> Hmm, I'm still not convinced that the API needs to know the MTU
> beforehand. qs_respond_to_query() returns a single string and the
> game can call send() with whatever size it likes. If that size is
> smaller than the string it just has to call qs_fragment_response()
> until nothing is left. qs_fragment_response() updates the header and
> does memmove() on the data.

Ah, I didn't understand the API.  If the app decides to send less than
the header size, then the API will break.  I think you give up too
much control if you let the app chose the packet size.

I would avoid memmove() if possible.  It's inefficient.  Game programmers
will appreciate an efficient API.

> 
> > > > I chose 1400 to avoid exceeding the MTU (usually 1500 on ethernet, but
> > > > occasionally smaller).
> > > 
> > > If you want to make sure the packet can be tansmitted over the
> > > internet you would need to restrict the packet size to 576 bytes.
> > > That's the minimum size each device needs to be able to handle.
> > 
> > I'm really only worried about ethernet.  MTUs in the 500 range are
> > really only used by analog modems.  And no one is going to run a 
> > server behind a modem.
> 
> What about clients? I suppose if you send a 1400 byte packet to an
> IP behind a modem the packet gets dropped at the dialup router?

Yeah, that's something that never made sense.  People say that small
MTUs may be used on the internet, but then lots of apps (streaming media,
games) run across modems just fine.  I don't think they're all doing
an MTU negotiation.  I'd check my networking book, but it's walked away.

Steve

Re: [QStat-devx] draft for new query protocol and API

[Ludwig N. <l-...@us...>]

Steve Jankowski wrote:
> > > Multiple outstanding requests.  Given that a packet write is a non-
> > > blocking operation, and a game server is likely to prioritize status
> > > responses below actual game processing, the server could be sending
> > > status responses to multiple destinations at the same time.  It seems
> > > like your API can support this, but it requires the app to track the
> > > state information.  If the app can only send 2 of 4 fragments, then it
> > > has to save the IP, port, queryresponse, fragnr, and reslen in some
> > > list.  The app will then need some code to occasionally run the list
> > > to try sending more query responses.  Seems like the API should take
> > > care of all these details.
> > 
> > Isn't that's too complicated? Storing all that information probably
> > costs more CPU cycles than just calling a couple of send(). I didn't
> > want to deal with IP Adresses, otherwise it would be necessary to
> > support IPv4 and IPv6.
> 
> Supporting IPv6 is not required (and why would you?  there's no IPv6
> games and game developers are not talking about it).

Commercial game developers maybe. Some free games, especially
Quakeforge and Quake2 already have IPv6 support.

> A good API implements all the complicated stuff so the app doesn't
> have to.  If the API is too "thin", that presents a barrier to
> adoption.  That complication has to be some where.  If it's in the
> API, then every app benefits.  If it's in the app, then every
> app gets to make the same mistakes.

What about this interface:

QSQueryResponse* qs_prepare_response(const char* buf, size_t len);

char* qs_next_response(QSQueryResponse* response, size_t *len);

QSQueryResponse is some opaque structure to the application. The
library might store a linked list or an array of response fragments
inside. qs_prepare_response creates that structure. qs_next_response
removes one fragment from the structure and returns it. If no more
fragments are left, the structure is freed and NULL returned.

Additional functions for state management might be added if needed,
e.g.:

void qs_postpone_response(QSQueryResponse*, struct sockaddr_storage*);

QSQueryResponse* qs_recall_postponed_response(struct sockaddr_storage**);

cu
Ludwig

-- 
(o_     Ludwig Nussel
//\   ICQ:     52166811
V_/_  PGP Key: FF8135CE

Re: [QStat-devx] draft for new query protocol and API

[Steve J. <st...@qs...>]

--- Ludwig Nussel <l-...@us...> wrote:
> Steve Jankowski wrote:
> > > > Multiple outstanding requests.  Given that a packet write is a non-
> > > > blocking operation, and a game server is likely to prioritize status
> > > > responses below actual game processing, the server could be sending
> > > > status responses to multiple destinations at the same time.  It seems
> > > > like your API can support this, but it requires the app to track the
> > > > state information.  If the app can only send 2 of 4 fragments, then it
> > > > has to save the IP, port, queryresponse, fragnr, and reslen in some
> > > > list.  The app will then need some code to occasionally run the list
> > > > to try sending more query responses.  Seems like the API should take
> > > > care of all these details.
> > > 
> > > Isn't that's too complicated? Storing all that information probably
> > > costs more CPU cycles than just calling a couple of send(). I didn't
> > > want to deal with IP Adresses, otherwise it would be necessary to
> > > support IPv4 and IPv6.
> > 
> > Supporting IPv6 is not required (and why would you?  there's no IPv6
> > games and game developers are not talking about it).
> 
> Commercial game developers maybe. Some free games, especially
> Quakeforge and Quake2 already have IPv6 support.
> 
> > A good API implements all the complicated stuff so the app doesn't
> > have to.  If the API is too "thin", that presents a barrier to
> > adoption.  That complication has to be some where.  If it's in the
> > API, then every app benefits.  If it's in the app, then every
> > app gets to make the same mistakes.
> 
> What about this interface:
> 
> QSQueryResponse* qs_prepare_response(const char* buf, size_t len);
> 
> char* qs_next_response(QSQueryResponse* response, size_t *len);
> 
> QSQueryResponse is some opaque structure to the application. The
> library might store a linked list or an array of response fragments
> inside. qs_prepare_response creates that structure. qs_next_response
> removes one fragment from the structure and returns it. If no more
> fragments are left, the structure is freed and NULL returned.

Yeah, that looks good.  I'd also add the IP and port so the app doesn't
have to maintain any data structures.  If you want to be IPv6 compatible,
then pass params like:

..., void *ip, int ipLen, unsigned short port)

Steve

[QStat-devx] CVS playing up?

[Steven H. <ki...@mu...>]

I cant seem to perswade CVS to update correctly.
Its saying it all done but the version numbers are all
out compared with the web CVS. Even tried deleting
all and checking back out.
Anyone else seeing this?

    Steve / K

================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 

In the event of misdirection, illegible or incomplete transmission please telephone (023) 8024 3137
or return the E.mail to pos...@mu....

Re: [QStat-devx] CVS playing up?

[Steve J. <ste...@ya...>]

Dunno, I'll check it tomorrow from work.

Steve

--- Steven Hartland <ki...@mu...> wrote:
> I cant seem to perswade CVS to update correctly.
> Its saying it all done but the version numbers are all
> out compared with the web CVS. Even tried deleting
> all and checking back out.
> Anyone else seeing this?
> 
>     Steve / K
> 
> ================================================
> This e.mail is private and confidential between Multiplay (UK) Ltd. and the
> person or entity to whom it is addressed. In the event of misdirection, the
> recipient is prohibited from using, copying, printing or otherwise
> disseminating it or any information contained in it. 
> 
> In the event of misdirection, illegible or incomplete transmission please
> telephone (023) 8024 3137
> or return the E.mail to pos...@mu....
> 
> 
> 
> 
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> QStat-devx mailing list
> QSt...@li...
> https://lists.sourceforge.net/lists/listinfo/qstat-devx

Re: [QStat-devx] CVS playing up?

[Steven H. <ki...@mu...>]

YEp its just lost the changes that Ludwig did :(

Ludwig you'll need to add your latest changes in
again when it comes back up down atm.

    Steve / K

----- Original Message ----- 
From: "Steve Jankowski" <ste...@ya...>
To: <qst...@li...>
Sent: Friday, January 23, 2004 7:37 AM
Subject: Re: [QStat-devx] CVS playing up?


> Dunno, I'll check it tomorrow from work.
> 
> Steve
> 
> --- Steven Hartland <ki...@mu...> wrote:
> > I cant seem to perswade CVS to update correctly.
> > Its saying it all done but the version numbers are all
> > out compared with the web CVS. Even tried deleting
> > all and checking back out.
> > Anyone else seeing this?
> > 
> >     Steve / K
> > 
> > ================================================
> > This e.mail is private and confidential between Multiplay (UK) Ltd. and the
> > person or entity to whom it is addressed. In the event of misdirection, the
> > recipient is prohibited from using, copying, printing or otherwise
> > disseminating it or any information contained in it. 
> > 
> > In the event of misdirection, illegible or incomplete transmission please
> > telephone (023) 8024 3137
> > or return the E.mail to pos...@mu....
> > 
> > 
> > 
> > 
> > -------------------------------------------------------
> > The SF.Net email is sponsored by EclipseCon 2004
> > Premiere Conference on Open Tools Development and Integration
> > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> > http://www.eclipsecon.org/osdn
> > _______________________________________________
> > QStat-devx mailing list
> > QSt...@li...
> > https://lists.sourceforge.net/lists/listinfo/qstat-devx
> 
> 
> 
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> QStat-devx mailing list
> QSt...@li...
> https://lists.sourceforge.net/lists/listinfo/qstat-devx
> 
> 

================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 

In the event of misdirection, illegible or incomplete transmission please telephone (023) 8024 3137
or return the E.mail to pos...@mu....

Re: [QStat-devx] CVS playing up?

[Ludwig N. <l-...@us...>]

Steven Hartland wrote:
> YEp its just lost the changes that Ludwig did :(
> 
> Ludwig you'll need to add your latest changes in
> again when it comes back up down atm.

Which changes do you mean? I don't miss any.

cu
Ludwig

-- 
(o_     Ludwig Nussel
//\   ICQ:     52166811
V_/_  PGP Key: FF8135CE

[QStat-devx] qstat.c bad merges?

[Steven H. <ki...@mu...>]

There seems to be a number of fixes gone missing in the latest
version is this intentional or just a bad merge?
The main affected function I can see is:
send_qwmaster_request_packet

    Steve / K


================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 

In the event of misdirection, illegible or incomplete transmission please telephone (023) 8024 3137
or return the E.mail to pos...@mu....

[QStat-devx] Change of PacketFunc definition?

[Steven H. <st...@mu...>]

To prevent issues like:
qstat.h:895: warning: initialization from incompatible pointer type

I propose we change the definition of PacketFunc from:
typedef void (*PacketFunc)( struct qserver *, char *rawpkt, int pktlen);
to:
typedef int (*PacketFunc)( struct qserver *, char *rawpkt, int pktlen);

And update all the relavent calls to return pertainent values.
This is not the simplest of changes so does anyone have any objections?

    Steve / K

================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 

In the event of misdirection, illegible or incomplete transmission please telephone (023) 8024 3137
or return the E.mail to pos...@mu....

Re: [QStat-devx] Change of PacketFunc definition?

[Ludwig N. <l-...@us...>]

Steven Hartland wrote:
> To prevent issues like:
> qstat.h:895: warning: initialization from incompatible pointer type
> 
> I propose we change the definition of PacketFunc from:
> typedef void (*PacketFunc)( struct qserver *, char *rawpkt, int pktlen);
> to:
> typedef int (*PacketFunc)( struct qserver *, char *rawpkt, int pktlen);

What's the return value of deal_with_halflife_packet good for? It
doesn't seem to be used.

cu
Ludwig

-- 
(o_     Ludwig Nussel
//\   ICQ:     52166811
V_/_  PGP Key: FF8135CE

Re: [QStat-devx] Change of PacketFunc definition?

[Steve J. <ste...@ya...>]

--- Ludwig Nussel <l-...@us...> wrote:
> Steven Hartland wrote:
> > To prevent issues like:
> > qstat.h:895: warning: initialization from incompatible pointer type
> > 
> > I propose we change the definition of PacketFunc from:
> > typedef void (*PacketFunc)( struct qserver *, char *rawpkt, int pktlen);
> > to:
> > typedef int (*PacketFunc)( struct qserver *, char *rawpkt, int pktlen);
> 
> What's the return value of deal_with_halflife_packet good for? It
> doesn't seem to be used.
> 
> cu
> Ludwig
> 

Looks like a good idea, Steven.

I just looked through the code, and I did some wierd stuff for HL
when implementing the packet combiner.  HL is the only one that
returns int.  (see combine_packets())

So what are you going to return for the rest of the PacketFuncs?
I guess you could just ignore the return for all cases except HL.

Steve