[Alephmodular-devel] Referencing, Accessors, and Object-Orientation

[Woody Z. I. <woo...@sb...>]

These messages seem to fit together, so I'm sending them both together 
despite the resulting e-mail taking half a year to read.  :)

----- Original Message -----
From: "Woody Zenfell" <woo...@ho...>
To: "Br'fin" <br...@ma...>
Sent: Wednesday, December 11, 2002 12:29 AM
Subject: Re: Fw: [M-dev]comments


> Random thought: the Marathon data structures should always have their 
> values
> set (if not both set and read) by accessors.  Probably easier to start 
> this
> early on than to stick it in later.  Of course, accessors in most cases 
> can
> collapse into inline code for hitting the data directly, at least for 
> now.
> But the 'hooks' will be established.
>
> I think the data structures refer to each other indirectly already 
> right?
> Basically one structure stores like the index of another structure, and 
> asks
> for the other structure by calling a function.  That is, a structure 
> does
> not ever store a direct pointer to another structure.  This is good.
>
> With hooks at data-structure writes and at "dereference object 
> reference"
> points, it should be reasonably straightforward to make these data
> structures effectively copy-on-write, for network prediction and
> between-tick interpolation.  (I tried copying and restoring all the data
> structures every game-tick, but there was far too much data.  If we can 
> copy
> and restore only the objects that change, I suspect it's a MUCH smaller
> amount.)
>
> Finding the access points should be straightforward in most cases, but
> aliasing of course makes things awfully sticky.
>
> Of course, locating dependencies on game-tick duration would be 
> necessary
> for between-tick interpolation.
>
> Sorry, know these things are a lot of grunt-work.  Not trying to dump 
> them
> off on you.  Just wanted to have them in your head as you consider the 
> grand
> reworking scheme, in case they inform some decisions.  :)
>
> Woody


----- Original Message -----
From: "Woody Zenfell" <woo...@ho...>
To: "Br'fin" <br...@ma...>
Sent: Wednesday, December 11, 2002 11:36 AM
Subject: Re: Fw: [M-dev]comments

>> I'm unsure about the copy on write stuff.
>
> Well, my ideas work as follows:
>
>
> Network prediction
>
> At any given time, you have a "real" game state, which is the state of 
> the
> game at the most recent game-tick for which you have complete 
> information.
> Additionally, you have a "predicted" game state, which is computed 
> based on
> partial information and represents the best guess at what the game state
> looks like for the game-tick whose input was just recorded.
>
> So, when a game-tick elapses on the local machine, the predicted game 
> state
> advances a notch.  When new data comes in from the network to give us
> complete data about the tick after the real game-state's tick, the real
> game-state advances a notch.  Note that the predicted state should then 
> be
> recomputed also (perhaps going through several game-ticks' updates, if 
> we're
> predicting ahead a little while) when the real state changes.  So the 
> old
> predicted state will be discarded.
>
> With a little cleverness/trickery it should be possible to use the same
> update routines to compute both real and predicted game-state 
> updates.  (I
> have already planned in A1 a scheme with multiple sets of action_queues 
> to
> handle real input vs. predicted input streams - will try to hunt that up
> somewhere.)  With copy-on-write objects, it should be possible to let 
> the
> two separate game states share most of the same objects.  Only objects 
> that
> get changed by a predictive update need to have a separate copy made.
>
> Initially I'd figured that the copied object would be the one then 
> written
> to.  But it would probably make coding more straightforward for the 
> copied
> object to be the one left unchanged.  Note that since deferencing object
> references is done explicitly (currently via functions like
> get_player_with_index() etc.), that routine can correctly return the
> real-state object (the copy, if there are two objects) or the
> predicted-state object as appropriate in context.  (There would 
> probably be
> some kind of widely-visible flag set that would be set during predictive
> updates and clear for real updates.)
>
> Hmm though, actually, copying the object, messing with the original, and
> discarding the original shifts things around in memory a lot more than 
> would
> copying the object, messing with the copy, and discarding the copy.  So
> maybe it still would be best for predictive writes to affect the copy.
>
> Note that this prediction scheme does not provide for interpolation 
> (e.g.
> lerping) between the old predicted state and the new, more accurate
> predicted state.  Players whose input is taking a long time to reach us
> would sort of teleport in little hops.  But at least the local player 
> would
> be seeing the best guess at any given moment - interpolation would 
> probably
> muddy things up in that regard.
>
> This scheme assumes that rendering is 'stateless' - i.e. you can render 
> any
> game-state regardless of the last game-state rendered.  (Obviously 
> texture
> caching etc. make the renderer not truly stateless, but at least the
> renderer will still produce correct results if that's the only state it
> maintains.)  Also, it does not take into consideration sound playback 
> or any
> other side-effects of updating.  And finally, it's still essentially a
> peer-to-peer symmetric-execution sort of scheme.  A cheating client 
> could
> show all players on map, or warn the player when somebody's pointing a
> weapon at him, etc.  It could also exploit this prediction mechanism: it
> could watch everyone else's up-to-date action, but delay sending its own
> moves a bit to give itself an advantage.  Rockets could seem to appear 
> near
> your head (or worse, feet) if they were fired a while ago but you didn't
> learn about them until just now.  Modern games seem to avoid this sort 
> of
> thing by making the server sort of "certify" actions, and things like
> shooting aren't allowed to happen until the server receives the "shoot
> command" from the shooter - which could of course be some time after the
> shooter pressed his mouse button.
>
>
> Between-tick interpolation
>
> Between-game-tick interpolation would work somewhat similarly.  In 
> addition
> to the above game-states, you'd have an interpolated state.  You would 
> need
> new update-the-world routines that can update things less than a 
> game-tick.
> But they don't need to be all that clever - maybe just move things that 
> are
> moving.  The real action still happens at game-tick updates.
>
> So you'd have, say, a real game-state for tick 450, and a predicted
> game-state for tick 453.  We could produce an interpolated game-state 
> for
> tick 453.4.  When it's time for tick 453.8, we throw out the 453.4
> game-state and make a new interpolated state from 453 to avoid 
> accumulating
> errors.  The quality of these interpolated updates doesn't need to be 
> all
> that great - just something to smooth out the action.  It's probable 
> that
> the interpolated states would be the only ones ever rendered (unless
> interpolation is disabled altogether, perhaps by a Preference).
>
> Does that make sense?  You'd still use the copy-on-write and the
> demultiplexing-object-reference-dereferencing mechanisms like above to
> maintain (and discard) these in-between states.  But you don't have to 
> worry
> about messing with input queues, and since you have new update 
> routines, you
> don't play any sounds, so that's not an issue, etc. etc.
>
> Actually, there may be a problem with this.  The game-tick updaters 
> probably
> assume that, say, an action_flag describes what a player's been doing 
> with
> the most recent game-tick of his life.  So if the player was moving 
> forward
> in his last action_flag, but now is moving left in the current 
> action_flag,
> the game-tick updater will (I suspect) produce a game-state that 
> reflects
> the player, moving left for the past game-tick.  But since the 
> interpolated
> updater didn't know that the player was going to be moving left (since 
> we
> don't collect the action_flag until game_tick time), it kept moving the
> player forward a little bit.  So in effect the interpolated updater is 
> going
> to be predicting ahead some 0 < amount < 1 game-tick, and it will be 
> wrong
> whenever the user changes what he's doing.
>
> I suspect given the small amounts by which the interpolator would be 
> off,
> this is probably only really going to be unpleasant as we mispredict the
> local player.  I mean, I doubt minute jitters in other objects' and 
> players'
> locations are going to be off-putting... but jittering the viewpoint 
> around
> could be rather unpleasant.
>
> Hmm so a few options are:
>
> 1.  Let it be wrong a little bit and hope it's not too disconcerting.
>
> 2.  Try to peek at the local user's input before game-tick time to 
> make a
> better prediction.
>
> 3.  Delay everything by a game-tick so we can see what the players 
> really
> did, instead of guessing.  But of course this introduces a little bit of
> latency, which could be less pleasant than the results of the above.  
> OTOH
> 1/30th of a second latency might be a small price to pay for smooth, 
> solid
> interpolation vs. the results of the above.  I really don't know!
>
>> As for interpolation, I've been wondering if there's a way to split
>> physics/AI from animations. I know animations currently have spaces
>> that both denote keyframe and sound to play. I'm not quite sure where
>> the sound information should go. But what if at shape load time the
>> monster physics noted how many ticks into the animation a keyframe
>> occurs. And then the animations were put elsewhere.
>>
>> Something like, a physics tick says a monster should move from point X
>> to Y using walking animation. (It no longer cares what frame you're on,
>> so as long as it keeps specifying walking animation, the walking
>> animation is used) The animation system would then have a record saying
>> 'move X to Y using walking animation by the time of next physics tick'
>> So if your renderer can pump out screens faster than your next physics
>> tick your animations are smoother.
>>
>> I suspect myself guilty of some kind of off by one error here.
>> (Normally it would be immediately move X to Y then render the new
>> screen...) I wonder if that's noticeable 30 times a second...
>
> I probably should have read this more carefully before writing my 
> text.  I
> think we're identifying the same problem ("off by one error") as we're
> thinking about the same scheme (rendering between the ticks).  But, in 
> my
> case, I could argue that _I_ had just gotten up when I first read it.  
> Heh
> heh.
>
> Anyway I don't think I know enough detail currently about the 
> frame-based
> animation in Marathon to comment intelligently on that.  Note that in my
> proposed scheme, you'd only care about keyframes in the game-tick update
> code.  You could have additional frames in between (whether provided
> statically or generated dynamically e.g. with the 3d model stuff) that 
> the
> interpolator and renderer would use merely for rendering.  You'd have to
> keep track of (only in the game-tick updater) "did I pass the 
> keyframe?",
> and you'd have to latch that value so you could detect the transition, 
> so
> you know which tick should perform the action.  Alternatively, yes you 
> could
> figure out ahead of time which integral game-tick after the start of
> animation should trigger the action, use an "equals"-style comparison, 
> and
> skip the latching and "greater-than-or-equal"-style comparison I just
> suggested.  It could analyze the animation's frames and timing to 
> calculate
> the "key time" for animations with "key frames", or (at some point) some
> alternative format could specify the "key time" (either on a game-tick
> boundary or rounded to one by the engine at load-time, of course) for 
> any
> given animation sequence, independent of the frames and frame-rate.  
> Yeah OK
> I think that's probably what you're getting at.  Sounds good to me!  :)
>
> Or heck, ask the animation or animation system about whether this is the
> keytick.  Let it decide whether it should round and == or latch and >=, 
> etc.
> Hmm that way animations could even have (if they can't already) multiple
> keyticks, playing different sounds and causing different actions.
>
> Ok ok ok let the above stand as some kind of insight into my reasoning.
> Here's what I'm now thinking:
>
> Animation knows its keytick, sound, effect, etc.  Game-tick updater 
> gives
> animation the opportunity to perform actions at game-tick time.  
> Animation
> is responsible for analyzing its frame-rate and keyframes etc. to come 
> up
> with a keytick.  Animation can choose whether it wants to play a sound, 
> tell
> monster to launch its projectile, spawn an effect, etc. etc.  In this 
> way an
> animation becomes more than just a sequence of frames, it's more like a
> little script for the monster (or whatever's animating) to execute.
> Renderer asks animation for best frame to show for the (non-integral)
> game-time it's rendering.
>
> Hmm then the animation needs sort of a "runtime context".  At the 
> least, a
> given instance of the animation needs to know when it was started; it 
> might
> want to keep track of more state too.  So I guess there should be a
> distinction between an "animation process" (the runtime context) and an
> "animation program" (the static supporting data with the various frames,
> keytime, etc.).  Multiple processes could refer to the same program.
>
> The monster or effect or whatever would have a reference to its
> currently-executing animation process (if there is one).  This 
> reference,
> like all others, would of course be "hooked" - i.e. somewhat indirect - 
> so
> copy-on-write could apply to animation processes etc. letting them be
> predicted and rolled back.
>
> Hmm, reference hooking in general could use the current
> "index-and-lookup-function" scheme, or could involve small "object 
> locator"
> objects that effectively encapsulate the reference and the dereferencing
> operation.  But some thought would be needed in the latter case with 
> regard
> to copy-on-write and how the locators would dynamically find the new 
> copy
> etc.  The locator could use index-and-lookup internally of course.  Or 
> maybe
> there's one or more maps of "real" object pointers (as keys) to 
> "alternate"
> object pointers (as values).  If we're operating in "real mode", we can 
> just
> return the stored real pointer.  If we're operating in an alternate 
> mode, we
> do the lookup.  Or maybe objects themselves (probably through 
> inheritance)
> have support for multiple versions of themselves, and overload * and -> 
> to
> work with the proper variant, letting you store direct pointers to the
> "base" versions of objects but still get "hooked" dereferencing.  (Does 
> that
> work?  Aha so then this->method() could potentially do something 
> different
> from just method()??)
>
> There would be centralized objects that vend (hooked) references to 
> other
> objects.  You know, locate other objects by name or by some other 
> (stable,
> unlikely-to-conflict) identifier, to encourage modularity.  It could 
> have
> standard names for the objects that are reanimated from existing data 
> files.
> New data files could add new objects, not merely replace existing ones
> (though I suppose that should be an option too).  Multiple new data 
> files
> could be loaded together without conflicts (assuming people chose 
> reasonably
> distinct names).  So these little reference objects, or these tricky
> reference-aware objects, could initially refer to other objects by type
> (well type-code of some sort I guess since C++ classes, sadly, aren't
> objects) and by name.  At the first dereference (or as part of some
> post-loading phase) these would be replaced by more-specific (faster)
> references to actual instances in memory.
>
> Hmm am I getting a bit carried away...?  Nah I don't think so.  I think 
> it's
> high time that the stored data break free from this index-based stuff 
> and
> move to something more flexible.
>
> And hey yeah, a file is just an archive of various objects.  There's no
> "map" and "shapes" and "sounds" files etc., there are just "level" and
> "bitmap" and "terminal" objects etc. and they're located in whatever 
> files
> they damn well please.  The loading code would be able to read the 
> existing
> file formats of course, but would present the objects found therein to 
> the
> rest of the code in a way that's consistent with the more-general 
> scheme.
> And maybe the engine supplies some pull-them-all-together objects, like 
> one
> named "m2trooper".  Then when it loads a map file that's detected as an 
> M2
> map, it converts references to monster index 15 (or whatever) into
> references to "m2trooper".  So you could then drop in a new-style file 
> that
> replaces the standard m2trooper object, and old M2 maps use the new 
> trooper.
> Of course m2trooper is distinct from minftrooper and m1trooper, so when 
> you
> load an M1 map you don't get an Moo trooper.  etc.  There should be some
> thought on "reference scope" I guess - the specifics of the
> locate-the-desired-object-by-name behavior - and also hmm there should 
> be
> ways to reroute references, so that e.g. an M2 Map file that's part of 
> some
> custom scenario produces (upon loading) references to 
> myScenarioTrooper, and
> there's a myScenarioTrooper object created by the engine that loads
> myScenarioTrooperAnimations etc.  This way merely adding some kind of
> "scenario description file" lets the existing files stay intact as 
> standard
> M2-format files or whatever, but the names of the objects provided and
> referred to change.  Then, a new-style map file could refer to a
> myScenarioTrooper AND an m2trooper, and it would work.
>
> Well there's a big dump of raw ideas, much refinement needed.
>
> Woody

Note: the m2trooper object created by the engine (the one that ties 
together the various bits and pieces) is effectively "Bio" information, 
if I'm reading this right.

Note also: Marathon frequently divides objects into "static" (shared) 
and "dynamic" (per-instance) parts.  (In some sense, this is like 
classes and objects.)  In any event, I'd support some effort that 
explicitly classifies the various structures/classes as one or the 
other, if it's not already obvious from naming conventions etc.

And finally, here's a message I was working on as a new message to send 
here, that puts the 'generalized reference' stuff another way (probably 
duplicates some of the above, but goes into more detail I think):

>> [Br'fin]
>> Well, the hardest part of saying 'no code' on the Bios is in the areas 
>> where current code links definitions to external resources. The second 
>> hardest part involves 'How do we set up one copy of AlephModular to 
>> handle M2 files versus Minf files versus M1 files?' Each one is a 
>> mildly different Bios. So even if the specific map loader isn't coded 
>> right into the Bios file, there still needs to be a way to specify 
>> which Map Loading object to apply. Similar details surround terminal 
>> rendering.
>
> [Woody]
> On all this, my instinct is to have these "Bios" (for want of a better 
> term) be real objects, with data _and code_.  Now, there could (and 
> probably should) be a movement of the various data values out into a 
> file, but that's sort of a detail of the Bio class (or a 
> Marathon-specific subclass).
>
> A while ago I sent a couple notes directly to Br'fin - the list was 
> just getting set up at that point.  In at least one of them I lobbied 
> for a very dynamic, object-oriented system.  In particular, classes and 
> objects would have general identifiers that the engine would use to 
> connect things up.  For example, there might be a class M2Trooper 
> (which perhaps inherits from MarathonTrooper, which inherits from 
> MarathonMonster, or whatever).  Conceptually, a Map file would include 
> a directive like "Create an M2Trooper here".  In practice of course we 
> want to maintain compatibility with the existing M2 Map format though, 
> so instead the M2MapLoader will rewrite existing references to "monster 
> index 19" (or whatever) to "M2Trooper".  The actual object would be 
> created with a phrase along the lines of 
> CreateObjectOfClassNamed(objectClass);.  The latter would look up the 
> class by name and call its Create() method, or somesuch.
>
> This seems awfully roundabout I'm sure, but remember that modularity is 
> at least as much about loosening the connections between modules as it 
> is about consolidating related stuff into modules.
>
> So a scheme like this buys you things like:
>
> + M1, M2, and Moo (and various other existing scenarios?) maps always 
> get you the corresponding type of Trooper.  (And they always get you a 
> Trooper, even if the monster index is different between them.)  None of 
> this "Environment" nonsense where the user has to get all the settings 
> right for things to make sense.
> + New scenarios can add new monster types (like "MyTrooper") but still 
> reference existing types (like "M2Trooper").  (This assuming there's 
> eventually a map file format that supports the generalized referencing.)
> + Scenarios can replace monster types, effectively providing a 
> "MyTrooper" whenever an "M2Trooper" is referenced.
>
> And I think you'll see how deep the rabbit-hole goes if you think about 
> M2Troopers referencing (indirectly, of course) "M2TrooperGrenade"s for 
> their projectiles, "M2TrooperMonsterAnimation"s for their animation 
> scripts (which reference "M2TrooperShape"s for their bitmaps), etc. etc.
>
> Hmm right so anyway I guess in this scheme the Bio and the MapLoader 
> are intimately related (maybe they're even the same, effectively, or 
> maybe the MapLoader implements the Bio by virtue of the names of the 
> things it references).  Of course naturally the code that loads M2 
> Shapes files provides the M2TrooperShapes and the 
> M2TrooperMonsterAnimations, and the code that loads/provides the M2 
> Physics helps the M2Troopers know they should use M2TrooperGrenades.  
> In this case, knowing that an M2Trooper *uses* M2TrooperShapes and 
> M2TrooperMonsterAnimations is part of the "M2 Bio".  So perhaps you 
> stick this information into a Bio object, and end up doing something 
> like:
>
> M2Trooper::foo() {
> this->GetBio()->GetMonsterAnimationForClassNamed(this->GetClass()->GetName(
> ));
> }

But OTOH if it's not really meaningful to have M2 Troopers operating 
with the M1 "Bio", then we can return to viewing the "Bio" information 
as living in the game-specific monster classes, projectile classes, 
etc. - it would not be explicitly split out into a different "Bio" 
object.

Note that "classes" here may or may not correspond directly to C++ 
classes.  For example, an object of the "m2trooper" class might really 
be an object of a C++ class called M2Monster.  The AM class-name might 
(in this case) only be useful for locating (via the Bio) the related 
resources (physics info, graphics, etc.).  At some point though the code 
needs to be able to create a C++ object for the AM object of course.  So 
perhaps the "m2trooper" object itself (regardless of its implementation 
in C++) serves as a factory for C++ objects of AM class "m2trooper".

Note that a lot of this referencing stuff would be useful for a 
potential eventual scripting system as well.

Now hmm, C++ is not that pleasant a platform to use for 
object-orientation... so maybe AM should get together with the "Cocoa 
porting" guy, and produce a modular Marathon that's written in 
Objective-C, but uses just Foundation Kit stuff so that it can be taken 
to a variety of platforms via GNUstep-base or libFoundation, and uses 
SDL for graphics, sound, input, etc. (as much as possible), again for 
cross-platformability.  Then the files with map info and physics model 
info and shapes and sounds and so forth really could be archives of 
objects.  :)  (Unless the Cocoa porting guy is missing the point and is 
only using Cocoa for the UI and other high-level OS services, or 
something.)

Well I think that's plenty to chew on for now.  Sorry for the length and 
general disorganization.  Hope an ensuing discussion provides some 
illumination.  It may seem that most of these ideas are currently 
outside the scope of this project.  Guess it depends on the intended 
extent of the "modular" in "AlephModular".  :)  I guess many of the 
ideas (like the multiple-game-state stuff) are presented as rationale 
for the proposed inclusion of certain low-level mechanisms.  Obviously 
higher-level things like between-tick interpolation or run-time 
replacement of Marathon 2 Troopers with some kind of custom objects are 
pretty far down the road.  (Though not as far as they would be without 
the supporting mechanisms ;) ).

Woody