[Muq-coder] Re: a way to force arity?

[Cynbe ru Taren <cynbe@mu...>]

Artiste Extrordinaire! <artie@...> writes:

> I have a snippet that messes up the arity evaluator.  It's something I
> whipped up for borogove to show that it's possible to take a group of
> strings in a block and convert them to one block of joined characters.  He
> wanted a way to do it without generating garbage. :)
> 
> : stringchars { [] -> [] }
>   |length 1 - -> len
>   |for i do{ i stringChars[ }
>   for i from 0 below len do{ ]|join }
> ;
> 
> I think this should be { [] -> [] [] } for the arity, but that causes an
> arity error that refuses to define the function.  Changing it to what is
> listed above works.
> 
> Example code:
> 
> root: [ "abc" "def" | stringchars
> root: [ "abc" "def" | [ 'a' 'b' 'c' 'd' 'e' 'f' |
> 
> -Andy



You can force any arity you want using '!'


    :   stringchars { [] -> [] [] ! }
        |length 1 - -> len
        |for i do{ i stringChars[ }
        for i from 0 below len do{ ]|join }
    ;

You want to be a little careful when doing this, since if you
make a mistake, or if you're lying to the compiler, all other
functions calling that function will be using an incorrect
arity.

But there are times when '!' is the right solution, and this
is one of them.

Correctly computing the arity of this function is far beyond
the capabilities of the current logic, which simply does a
simple-minded constant-time symbolic execution of the code.

(It would be nice if it at least knew that it doesn't know...)

The current arity checker has no way of knowing that the second
line in the function will push as many blocks as the third line
eats (but one) -- this would involve much more sophisticated
theorem proving techniques (probably much slower than the current
arity checker, compiler and assembler combined) or else a very
special-case hack.



That's a pretty nice solution to the problem as stated, btw.

Since Muq's stacks are currently limited to fairly small sizes,
to prevent one infinite recursion error from thrashing the
whole system to death, you do run a risk of running out of
stack space on relatively small problems.

Having people start worrying about not generating garbage and
distorting their programming style to avoid this is of course
one of the disadvantages of not having a really first-rate
garbage collector implemented yet *wrygrin*.



  *  *  *  *  *

By the way, GeomView is now on SourceForge and out in a Linux edition.
GeomView is a very nice little 3D graphics package driven by a tiny
lisp interpreter.  I've liked it for years.  It was developed at an
American university (U Maryland? I forget) to support mathematicians
and such, and consequently has cool hacks like support for
non-Euclidean spaces.  The developers originally had even more
ambitious plans, but their funding source and available time
didn't allow for it. They've since been defunded and moved to
geomview.sourceforge.net and become primarily a Linux open
source project.  I haven't looked at the new release, but I'd
be surprised if it isn't worth a look by anyone interested in
smart 3D graphics servers for Linux. :)

 -- Cynbe

[Muq-coder] mudshell design philosophy

[Jerry Hsu <jhsu@cj...>]

Looking through the oldmud/oldmush example, it appears that commands take
the form of msh reads teh command, does some parsing and passes a request
on to the mud which processes the request.

Presuming my understanding of that is correct, I'm unclear as to why this
approach was used. My intuitive design for a shell woudl be something
along the lines of pass most commands straight through to the server and
let the server handle parsing.  My vision for the shell would be to
process the return and maybe do some formatting on it.  I'd also envision
the shell providing some other support like possible filtering and command
macroing.

It seems to me that following the oldmud/oldmsh example would require new
commands to be implemented both on the mud and in every shell that is
written.  I guess this would facilitate something like having both a
graphical shell and a text shell, the graphical shell woudln't present a
text UI to the user.  Or maybe having foreign language shells.  But those
don't seem like common cases.

So overall, can anyone (Cynbe) give me some reasoning behind designing
things this way and the advantages it would present?

Re: [Muq-coder] mudshell design philosophy

[Cynbe ru Taren <cynbe@mu...>]

Jerry Hsu <jhsu@...> writes:

> Looking through the oldmud/oldmush example, it appears that commands take
> the form of msh reads teh command, does some parsing and passes a request
> on to the mud which processes the request.
> 
> Presuming my understanding of that is correct, I'm unclear as to why this
> approach was used. My intuitive design for a shell woudl be something
> along the lines of pass most commands straight through to the server and
> let the server handle parsing.  My vision for the shell would be to
> process the return and maybe do some formatting on it.  I'd also envision
> the shell providing some other support like possible filtering and command
> macroing.
> 
> It seems to me that following the oldmud/oldmsh example would require new
> commands to be implemented both on the mud and in every shell that is
> written.  I guess this would facilitate something like having both a
> graphical shell and a text shell, the graphical shell woudln't present a
> text UI to the user.  Or maybe having foreign language shells.  But those
> don't seem like common cases.
> 
> So overall, can anyone (Cynbe) give me some reasoning behind designing
> things this way and the advantages it would present?
> 
> _______________________________________________
> Muq-coder mailing list
> Muq-coder@...
> http://lists.sourceforge.net/mailman/listinfo/muq-coder


I suppose it depends where you're coming from, and where you see the
system as headed.

I'm sure what you're envisioning makes sense relative to what you're
interested in doing;  I'm inclined to think my design approach makes
sense in terms of my vision.  So here's a sketch of my notions on
this subject.  Keep in mind that your design problems and solutions
may well vary for very good reasons, and that I'm not presenting
oldmud/oldmsh as -the- way everything should be done on Muq, just
as one direction in which people can get started.



Personally, I remember the Bad Old Days when operating systems like
TOPS-10 essentially did do all user shell parsing &tc in the kernel.

Then I ran into Unix, where the user shell is just an unprivileged
process, and Joe Random User is free to write and use his own shell
without any impact on system security or stability -- the critical
system services are religiously separated out from the random C
library code, commandline parsing code and such.

To me, this was just a Great Leap Forward!  Now we can have our
choice of sh bash ksh rsh csh tcsh wsh &tc &tc &tc -- support
for diversity instead of imposition of The Army Way From On
High.  Not to mention people can layer on graphical shells of
various sorts, again without impacting system security or sanity,
since none of the shell code need be trusted by the system.



Without necessarily taking the position that what current online text
or graphics worlds do is wrong or inadequate, I'm very interested in
using Muq to explore possibilities not well served by current servers.

One particular frontier which I think is interesting to explore is
scalable, decentralized worlds.  Many mu* have in the past scaled
to saturation of the central server and then either stalled or died.

One can of course take this as an indication that allowing such
unbridled growth is Just Plain Stupid and admins should impose
more sensible limits sooner.  There Aint No Such Thing As A Free
Lunch.  No exponential growth phase can last very long in an
effectively finite universe. &tc.

But one can also take the position that such experiences demonstrate
that there is an existing demand for larger-scale online communities,
and take it as a challenge to provide suitable support for them.

Muq oldmud is in part an experiment in laying the groundwork for an
online community which can scale fairly trivially to dozens or
hundreds of servers and thousands to tens of thousands of users, with
reasonable prospects of scaling up in upward compatible fashion to
millions if the demand proves to be there.

So it supports an automatically updated local db of known servers
and users, transparent migration of servers from one IP address
to another, transparent migration of users from one Muq server to
another, connection of any exit to any room regardless of server
boundaries, paging and posing between users without regard to
machine boundaries, automatic transparent authentication of all
cross-server user<->user traffic using Diffie-Hellmen public key
encryption, automatic transparent encryption of all cross-server
user<-> user traffic using Twofish encryption, and various other
scalability hacks like that which aren't common in contemporary
mu* systems.

In the context of a shared world with potentially millions of users
and server machines, there seem to me to be good reasons for
maintaining a pretty clean firewall between the shell, which contains
code local to the user and trusted by the user, and the world server,
which contains code maybe owned and operated by "il4q2" physically
located in Outer Neferia.  Some specific examples:

o  I don't think il4q2 should be able to access or modify my local
   information as a user, to successfully fake messages so that
   they appear to come from me, to send me messages which appear
   to come from someone I trust (but do not actually do so), or
   to eavesdrop on private whispers I send to someone else in a
   room hosted by il4q2.
     oldmud accomplishes this by performing most of the relevant
   processing locally in the user's shell:  il4q2 is allowed to
   list who is currently present in the room he hosts (which is
   his perogative as owner and implementor of the room), which
   list we check with the people listed -- an object or user is
   counted as present in a room only if both room and object/user
   agree on that fact -- but all pages/poses/whispers to those
   objects/users are sent direct and do not in general even go
   through the room server -- if it crashes, people chatting in
   the room might not notice for some time.
     Similarly, the oldmud message display format is carefully
   structured to prevent malicious spoofing of trusted agents
   or people -- messages from the room server are always
   syntactically distinguishable from those by people within
   the room, and those of people in the room from each other.

o  I don't think all people passing through a given room should
   have to switch to using the favorite shell syntax of the
   user implementing that room.  When server boundaries are largely
   invisible, this could create all sorts of major and minor problems.

o  I think shell design will evolve much faster if it is decoupled
   from world design issues, and in particular if you don't need a
   wizbit just to fiddle with your shell.  Imagine if you had to
   go to the campus system administration people every time you
   wanted to switch to using a different shell or mail client. :-/



Now, someone may well -want- to implement a worldserver on Muq that
doesn't scale past one host machine, which is highly vulnerable to
spoofing, where all control is in the hands of the All Powerful,
All Knowing, All Benevolent Wizards, where processing proceeds for
only one user at a time, and so forth.

That is fine!

Muq has the tools, in many ways it is a much simpler implementation
problem than what oldmud does, and it is a popular, proven model with
a demonstrated audience.  There's nothing wrong with that, and in fact
I hope people write a whole variety of servers on top of Muq,
including some on that model.  That's why I bent over backwards to
make the Muq server policy-free in the frequently excoriated X Window
System tradition, and why I named oldmud "oldmud" as a hint that I
expect -- hope! -- to see a variety of other mudservers to be written
for it in due course.



But my personal motivation comes in large part from my delight in
seeing what novel things people create in shared online worlds,
so oldmud is heavily oriented towards:
 o Implementing things which haven't been done before;
 o Pointing the way towards interesting new possibilities;
 o Maximizing the potential for others to create new things.
Letting Joe Schmoe Average User hack up a killer new shell without
having to go humbly begging to the wizards for special help is one
aspect of that.



Personally, if I were using oldmud much, the first thing I'd do
would be to replace oldmsh with a custom shell of my own.  I think
traditional MU* syntax sucks.  E.g., on typical social mu*, the
most common commands are say and pose, so they should have the
most convenient syntax, which for most people on most keyboards
would be something like taking any line starting with an alphabetic
as a 'say' and any line starting with a blank as a pose.  (I'd use
a leading '.' as an escape for the rest of the commandset.)
  But converting people to ergonomic mudshell designs is not a
that's not a battle which it makes sense to fight at the
same time as trying to establish Muq!  There are right and wrong
times to open a second front.  So I implemented oldmsh as a
painfully plain, middle-of-the-road syntax.



Does that clarify the motivation behind oldmud/oldmsh's design
and organization?

Again, I don't want to get into a debate over whether the design
goals and decisions for oldmud/oldmsh are The One True Religion.
I'm just trying to clarify why they are the way they are, whether
or not that is right.



BTW, oldmud/oldmsh are among other things experiments in using
Flavors-style multiple-inheritance heavily.  I try to try something
new on every project, and I hadn't tried using multiple inheritance
heavily before.

If I were to do it again now, I'd retain the basic structure, but
base it instead on Dave Cheriton's warthog/trampoline style, which
has the virtue of elevating interfaces to first-class objects.
(That's a nice approach which would be nicer if given appropriate
support by the compiler.)

It would probably also be nice to support calling functions located on
other hosts (with appropriate safeguards!) so that the oldmud network
protocol can be more transparently extensible.  Right now it has a
painfully hardwired feel.  I attribute that partly to Muq being young
and providing less support than one might wish to distributed
programming, and mostly to the fact that constructing really seriously
scalable, decentralized shared worlds being a very new art involving
making novel design trade-offs between lots of nontrivial issues.

When the server software runs on one machine, with the wizards
completely trusted and the users basically not allowed to do
anything to speak of but run around and admire, the trust issues
are pretty minimal.  When you have lots of separately administered
servers and lots of users free to move between servers and in
general limited trust between any two admins or users, -plus-
you want to maximize the creative space available to everyone
involved, then things get ... well, frankly, more interesting!

Fascinating design trade-offs.

Which of course open lots of space for software designers to
disagree as to what the Right Answer is. :)

I hope Muq allows lots of designers to implement lots of variations,
and for a larger community to form which decides which solution it
likes best.

 Cynbe

Re: [Muq-coder] mudshell design philosophy

[Cynbe ru Taren <cynbe@mu...>]

Jerry Hsu <jhsu@...> writes:

> Looking through the oldmud/oldmush example, it appears that commands take
> the form of msh reads teh command, does some parsing and passes a request
> on to the mud which processes the request.
> 
> Presuming my understanding of that is correct, I'm unclear as to why this
> approach was used.
>
> [...]

Actually, now that I think about it, I may have been misinterpreting
the above question, or at least concentrating too much on one issue
to the exclusion of other interesting and relevant design issues in
oldmud/oldmsh.

There are a number of interesting division of responsibility issues
involved, in terms of what the relative roles should be of:

  The user;
  The shell coder;
  The room builder/coder;
  The world builder/coder;
  The worldkit builder/coder.

One of the design decisions in oldmud/oldmsh is to have per-user
code which includes

  o  A persistent daemon which is responsible for animating all
     of that user's possessions;

  o  A shell job which is responsible for parsing and executing
     input from the user.  This job goes away when the user logs
     out.

The persistent per-user daemon shows off Muq's multitasking ability,
and forms an interesting alternative to the more typical
single-threaded mudserver design, and more importantly, it provides
a firewall between world functionality and user functionality.

Without this sort of firewall, allowing Joe Random User to hack
code seriously can quickly compromise world stability, security
or sanity, so there is a strong incentive for world admins to
clamp down strongly and only allow very trusted users to do much
in the way of hacking.  This is a drag, in that it reduces the
amount of interesting new hacking likely to happen.

With a solid such firewall in place, users can be allowed to hack
weird per-room code to their heart's content, and if they crash
their room or area daemon *shrug* ok, so vistors have to go
elsewhere for awhile.  No big deal.



A second design decision is to have almost all the "real work"
done by/for a given user go through this per-user persistent
daemon.

The biggest advantage of this is that it avoids almost all locking and
consistency sorts of issues: If objects are only modified by the
player owning them, and only via a single animation daemon, then the
scope is vastly reduced for the typical sorts of synchronization
problems involving two separate jobs reading and writing the same
information at about the same time.  This is good because the sort
of naive end-user who often winds up doing most of the creative
building on mu* is quite unlikely to write correct locking.

(I looked for a long time at transparent rollback based handling
of such interactions, but concluded it would likely halve system
performance, and that most worlds/admins would in the end rather
have the performance.)



The oldmud/oldmsh 'task' facility serves to implement this design
philosophy in the face of arbitrary network lag by managing multiple
outstanding requests, continuing processing in the interim.  As long
as responding to each request can be done in a fraction of a second,
running them all out of the single persistent per-user daemon is a
viable way to go.  Operations which take significantly longer can
always be handled as special cases by fork()ing off a separate thread,
at which point one may have to start thinking about explicit locking.
I think this reasonably well satisfies Alan Kay's dictum that

   Simple things should be simple;
   Complex things should be possible.

The task mechanism keeps simple things simple;  The primitives are
available to do more complex things for those who have the need and
skill to do so.



It seems to me to make a lot of sense to separate off the user's
shell from the user's persistent animation daemon:

 o  The user's daemon needs to be very persisent and reliable;
    The user's shell will typically be stopped and started each
    time a telnet connection is made.

 o  The user may want to run the shell on a different server from
    the one currently hosting most of his state -- logged in remote
    on a laptop or whatever.

 o  The user might want to run several shells at once, maybe with
    a friend logged in or running multiple windows or running a
    batch script in one or whatever.

 o  The user may want to experiment running different shells, without
    wanting to affect the stability of the persistent daemon.

 o  If the user's persistent daemon -does- crash due to buggy per-user
    code, it is very nice if the user's shell is still running, so as
    to provide a springboard from which to fix and reboot the daemon!



So the general architecture of a running oldmud/oldmsh system with
two users sitting in a room chatting looks like:


 +--------------+        +---------------+
 | User A shell | <----> | User A daemon |
 +--------------+        +---------------+
                                ^               +---------------------+
                                | <-----------> | Room owner's daemon |
                                v               +---------------------+
 +--------------+        +---------------+
 | User B shell | <----> | User B daemon |
 +--------------+        +---------------+


The room owner's daemon is just one more persistent daemon, running
exactly the same code as the User A and B daemons, it is drawn
differently here because we are interested in this context in slightly
different uses of it.

When User A enters the room, the command to do so is parsed by the
shell, then passed to the daemon for execution, since this involves
changes in the general user state.  The User A daemon asks the room
daemon for permission to enter, and having recieved it, asks for a
list of objects in the room and caches the list locally to speed
up room operations.

Being told that User B is in the room, the User A daemon queries
the User B daemon to find out if this is true (the Room owner's
daemon might be lying or out of date or buggy) and if the User
B daemon agrees (which might involve a netlag of 30 seconds on
a bad net day), the User A daemon then has the User a shell tell
User A that User B is in the room.

If User A and User B chat with each other, their messages will
be passed back and forth directly between their two daemons,
without involving the Room Owner's daemon at all.  


Naturally, each of these the above illustration boxes might be running
in a different server on a different machine in the most general case,
although in general each shell/daemon pair is likely to be on the same
machine and Muq server, and I don't think the current release yet
supports separating them.  But the three daemons could easily be on
different machines in the current release without anybody noticing any
difference.  If machine or server boundaries -are- crossed, appropriate
authentication and encryption will be done automatically:  User
daemons can trust Muq-supplied source attributions.

 Cynbe