Re: [Alephmodular-devel] Copy-on-write in practice

[Woody Z. I. <woo...@sb...>]

On Wednesday, January 15, 2003, at 07:42  PM, Br'fin wrote:

> Hsm. I admit that I'm having trouble following the discussion of 
> Copy-on-write. This might be due to lack of sleep or the minutiae that 
> these discussions are going into.
>
> Also, we don't necessarily have to keep it similar to the current code. 
> Prime importance is having a clear description of how networking ties 
> in with input and game core.
>
> How does the GUI layer and the core layer work together? How does the 
> network layer and the core layer work together? What scenarios do you 
> see happening?

I see, yes since I didn't really have any obvious problems with the 
current code in this regard, I didn't spend much time thinking about 
these issues, and so my suggestions were generally made with something 
rather like the current scheme in mind.

For my overview of the way things currently work (in M2/A1), see

http://source.bungie.org/_enginedevelopment/reference/networking-
input.html

(Note that contrary to what the end of that says, I now believe that 
heartbeat_count is probably important for proper timing of film playback 
(but not single-player games), though I still haven't bothered to 
analyze it in detail.)

> If we assumed an 'animation layer' was appropriate, would the following 
> be acceptable
> 	User enters input
> 	Input is sent to other servers (Send to network)
> 	Other players inputs are accumulated (Receive from network)
> 	Update game core with new info
> 	Game core overwrites the animation layer's dynamic data with new info.
> 	Go back to process user's input
>
> 	The animation layer itself runs faster than the game core. It's 
> information tends to keep track of a simple subset of Game Core info. 
> (Game core decides that a player is at x,y and is turning/firing, 
> monster x is charging in a particular vector) Game Core does all the 
> heavy lifting of AI and the animation layer would just do fluff based 
> on last known info to add frames between decisions.

Maybe we're confusing each other because we have different terminologies 
in mind.  :)  I am also suspecting that what we're picturing is more 
similar than it might seem at a glance.

Oh first off for clarity, going back to the above, I tend to use the 
term GUI exclusively to refer to things like the game's main menu, the 
dialogs, etc.  I would tend to use the term "renderer" when talking 
about drawing game frames.  (What terms you use is up to you as long as 
we know what you mean :) )  (Does this change anything about the 
previous SDL discussion?  Nah let's not get back into that... :) )

It seems to me that what you're calling an "animation layer" is what 
would be responsible for the interpolative segment of my three-stage 
game-update scheme.  (Which I _have_ posted here, haven't I?  With real 
game states, predicted game states (optionally), and interpolated game 
states (optionally))?

In the existing scheme (and thus in mine) there's only one type of 
"game-state object" (which consists, conceptually, of essentially all 
the stuff that you'd see in a saved-game, basically) which the 
"game-core" (the part that does what I'd call "real updates", I gather - 
currently update_world() does zero or more real updates when called), 
interpolative updater (in my scheme), and renderer all share to do their 
work.

In my scheme, the interpolative updater only makes "small" modifications 
to the (interpolative) game state, i.e. moving players and monsters and 
projectiles and platforms etc. just the way they've been moving and so 
on.  What you call "heavy lifting" (monster AI etc.) is done only by the 
real and predictive updaters (which in my scheme are essentially the 
same code operating on different copies of the game-state object - 
sounds like you're instead picturing that the predictive and 
interpolative updaters have a lot in common - I guess which way to go 
depends on how expensive the different kinds of updates are).

> I should emphasize that this is one possibility. I would like to see 
> the discussion a little higher level in description right now. you're 
> speaking of copy on write, but which side of the code is responsible 
> for that? In retrospect what I'm thinking of as an 'animation layer' 
> might just be a different way of expressing the same issues. Then 
> again, it's also trying to address decoupling framerate from the game's 
> current ~30fps game running

Though my COW discussion was heavily biased towards prediction for 
simplicity, the same mechanism would be used for interpolation.  The 
basic idea is to be able to split off a copy of the game-state, mess 
around with it, render from it, and then throw it away (returning to the 
original state).

> (I don't currently know the feasibility of having the world itself be 
> actively managed over 30 fps, would we have to make sure that M2 style 
> monster aren't allowed to readjust their AI except every 1/30th of a 
> second?)

If we didn't care about 100% compatibility with M2, i.e. a film recorded 
in M2 and played back in AM would produce *exactly* the same results (or 
the same results but rendered more prettily etc.), (which I should 
hasten to say, I think we *do*) then it would probably be reasonably 
straightforward to do what I'd call "real updates" more frequently than 
30 ticks/sec.  But since we do care, I think we need to preserve the 
"real update" code as-is (reorganization notwithstanding), including the 
30 ticks/sec restriction.  So I agree that some kind of lightweight 
"move some things a little bit between actual ticks" interpolative 
scheme is the way to go.

"Which side of the code" is responsible?  Well neither I guess, again I 
think this is an oversimplification.  There would be some kind of game 
manager that would know what needs to happen (apply real updates to real 
game state, copy that state (whether an actual copy or marking as COW) 
to make a predictive state, apply predictive updates to that state, copy 
the resulting state (maybe COW) to make an interpolative state, apply 
an  interpolative update to that (pretty sure we should always only do 
one of these, representing the activity of some fraction of a 
game-tick), and call the renderer).

Whether you want this to be a function like update_world with all the 
actual update_players(), update_monsters(), etc. calls sliced out to a 
sub-function, or whether it's an object in the C++ sense and thus can be 
of different classes depending on whether it's a netgame or whether 
interpolation is desired etc., or whether it's set up by virtue of 
correct registration of a group of routines in some sort of master list 
of routines to call (in some order) repeatedly, is in some sense 
irrelevant here (but could be a related topic to discuss).

The object-level COW scheme (which covers both OPTION 1 and OPTION 2 
previously discussed) has the advantage that we can effectively copy the 
entire game-state very cheaply if only a small part of it is expected to 
be changed.  Using a bulk-copy mechanism (like the OPTION 3 I outlined) 
you'd probably want to slice up the game-state into stuff that never 
changes during a game (since there's no need to copy it then), stuff 
that never changes during an interpolative update, and the stuff that 
changes most frequently, so that you could copy only what's really 
needed.  But doing that in a static sort of way (which is not strictly 
necessary I suppose but would be much easier I think) could limit the 
ability of scripts or the like to later come in and mess with things 
that M2 itself does not mess with (like, say, a polygon's floor transfer 
mode (right?), or a platform's speed, or even maybe general map geometry 
(though this could create some real headaches if there are any objects 
in the related polygons etc.), or something).

It sort of sounds like you're picturing a decoupling in the actual 
execution of the game-update stuff and rendering stuff (e.g. into 
separate threads or something).  I suppose there could be advantages to 
that on a multiprocessor system (and yes, there are some of those out 
there) but on a uniprocessor it seems like it could only make 
performance worse (and complicate the code)... and might gratuitously 
break Mac OS 9 compatibility (in case that's a problem).

I suppose if you wanted to move toward a client/server approach where 
one machine does the updating and the other machines effectively only do 
the rendering portion, it might be good to let the two segments run 
independently so that e.g. rendering on the server does not bog down 
everyone else's game (since they're dependent on receiving the new 
game-state data from the server).  In some sense this is exactly why the 
existing scheme handles user-input and protocol processing stuff in 
separately-scheduled tasks.  (If I say "threads", will people 
automatically know that I don't really mean threads in the Mac OS 9 
version?)

Hmm note that an object-grained COW approach could help a potential 
server figure out which objects need to be sent out on the wire to the 
clients.  (If you didn't want full-blown COW for some reason, you'd use 
the same sort of mechanisms to mark the object "dirty".)

Note that object-grained COW (whether for efficient-copy-the-game-state 
reasons or client-server-scheme reasons) could potentially benefit from 
decomposing some existing objects into likely-to-change-often and 
likely-to-remain-static sub-objects, which in some sense is the static 
"slicing up" of the game-state I talked about with regard to 
bulk-copying above.  But with the COW scheme the likely-to-remain-static 
stuff would still be automatically copied in those rare cases when it's 
needed, so scripts (or later update-logic changes compiled into the code 
proper) would still be able to do weird things without breaking (unlike 
in a statically-sliced bulk-copying scheme, where one _must_ not modify 
stuff outside the current "slice").

(I just got this picture of Conker asking the gargoyle "Isn't it a bit 
early in the morning to be talking about gothic architecture...?")

Woody