RE: [Jscheme-devel] Module implementation

[Derek U. <Der...@on...>]

I've put a copy of the tree at

  http://www.serv.net/~sand/jscheme-envs

I've removed the JAR files, class files, and compiled Java files (and the
whole of "src/elf/eopl2") to save space.

My design goal for the new code was to let the language support existing
practice.  Existing practice for Scheme typically involves adding a module
name prefix to the front of each top-level object; the entire body of the
library has to be made module-aware (or jump through hoops such as massive
"(let ...)" statements, which amounts to the same thing).  The new code
pushes the prefixing burden onto the loading application, and does it
automatically.

Regarding loading favorite code into each environment: if the favorite code
satisfies the idempotence requirement, then it's possible to cache the
environments created by the import.  You could import "elf/basic.scm" ten
times with ten different prefixes, but only need to load and evaluate it
once.  You are creating more dynamic variables than you would otherwise, of
course (the originals, plus one set for each import).

Regarding the choice of using prefixes or no: if you pass #f as the prefix
value, the symbols get imported without any prefix, but the separate
environments still prevent collisions in the imported libraries.  If two
libraries conflict, you can pick one to assign a prefix to it.

Alternatively, we always do prefix-less importing and expose the
"import-bindings" functionality:

    (define (environment-import-binding src sym new-sym)
      (let ((env (loading-environment src)))
        (eval `(define ,new-sym (eval (quote ,sym) ,env))))))

So if there's a collision, you can explicitly re-import to resolve it.
(This is just a sample; a real implementation would need to support merging
Generics, and they're weirder.  Of course, this sample becomes much more
efficient if "loading-environment" caches its results.)

Derek

-----Original Message-----
From: Ken Anderson [mailto:kan...@bb...]
Sent: Tuesday, October 01, 2002 12:37 PM
To: Derek Upham; 'Timothy Hickey'
Cc: 'jsc...@li...'
Subject: RE: [Jscheme-devel] Module implementation


You've been busy.  I haven't even checked in your last extension yet.
Can you post the code so we can take a look at it?

An earlier version of Jscheme had multiple environments, though it did not 
have a module system.  One problem with it was that you always wound up 
loading your favoite code into each enviornment before you started.  Also, 
the environments had to be large enough to have a value for all symbols.

I'm not suggesting your approach has these problems.

I have a strong reaction against allowing someone to reach across module 
boundaries by saying foo:bar.  This is based on my experience with Common 
Lisp.  Common Lisp has a package system.  You could import a package into 
your package and refer to exported symbols simply as 'bar.  You could also 
refer to exported symbols in an arbitrary packagage as foo:exported.  And 
you could access any symbol with foo::bar.

My belief is there should be not package prefixes in your code because it 
make the code maintainable, you don't know where the symbol comes from.
Others liked to use the foo:bar to know where the symbol came from.

It became tricky to manage a complex package network.

Tim and i one day played with a simple module system which used lexical 
scope, but it did not handle macros.

At 01:51 PM 10/1/2002, Derek Upham wrote:
>I don't know enough about the PLT unit modules to comment on future
>conflicts there, except in the actual use of symbols in the global
>environment: this system allows the importing program to choose the
prefixes
>used.  From my brief examination of the MzScheme documentation, their
>"namespace" mechanism is roughly similar, but they always add bindings to
>the local environment with no prefixes and (complicated) overriding rules.
>
>Regarding macro/compilation compatibility, the idempotence requirement
saves
>us.  If the imported code satisfies the requirement, there should be no way
>of telling whether the code was loaded at compile time or at run time, one
>time or multiple times.  Of course, if the imported code changes between
the
>time it was compiled and the time it was loaded, the idempotence
requirement
>is violated.
>
>BTW, I've already modified the compiler to support this system.  Just pass
a
>class to "environment-import" instead of a file.  The compiled class loads
>into the ``current'' environment instead of the interactive one.
>
>Derek
>
>-----Original Message-----
>From: Timothy Hickey [mailto:tim...@ma...]
>Sent: Monday, September 30, 2002 8:55 PM
>To: Derek Upham
>Subject: Re: [Jscheme-devel] Module implementation
>
>
>Hi Derek,
>
>    We'll have to see what Ken thinks, but it looks like a very interesting
>extension. Ken and I had discussed earlier the idea of making Symbols
>into "Dynamic Variables,"  in a way similar to what you have done.
>
>It looks like this should allow current code to run unchanged but it will
>add an extra lookup for each free variable evaluation.
>
>I've been starting to feel the pinch from not having a module system
>and what I like about your system is that it is very simple (Hence easy
>to understand and easy to implement efficiently). My only concern is
>whether including this module system will prevent us from adding a
>"better" module system later (e.g. PLT's unit modules???), but I think
>any module system should be able to provide the kind of functionality
>yours is giving so that shouldn't be a problem.
>
>If we do check it in, then we'll need to modify the compiler to handle
>modules (shouldn't be hard).
>
>Have you thought about how the module system interacts with macros?
>At first glance it looks like macros should behave as expected when
>in an imported file....
>
>This is exciting!
>
>---Tim---
>
>
>On Monday, September 30, 2002, at 06:27 PM, Derek Upham wrote:
>
> > This weekend I got a simple module system up and running.  Simple, in
> > this
> > case, means that it is compatible with the "module:symbol" naming
> > convention
> > used in a lot of Scheme libraries.  The steps were as follows:
> >
> > 1. Symbols no longer contain their values.  There are separate
> > first-class
> > dynamic environments that contain the values.
> >
> > 2. Free variables are no longer Symbols after code analysis; they are
> > DynamicVariable objects, bound to a particular Symbol and
> > DynamicEnvironment.  This means that free variables are always looked
> > up in
> > the environment they were defined in.
> >
> > 3. A new function "loading-environment" acts as "load", but all
> > definitions
> > are done in a new environment with ``standard'' initial bindings.  The
> > new
> > environment is returned.  You can pass that environment as the second
> > parameter to "eval", e.g.,
> >
> >       (let ((e (loading-environment "elf/basic.scm")))
> >         (eval 'iterate e))
> >
> > The new environment is locked-down; you can't change any of its dynamic
> > bindings.
> >
> > 4. A new builtin (non-exposed) function "importBindings" runs through
> > all
> > the bindings in an environment and adds similar bindings to the current
> > environment.  The new bindings can either be identical, or they can
> > have a
> > prefix attached.
> >
> > 5. A new function "environment-import" does both the loading and the
> > importing in succession.
> >
> >
> >
> > The upshot is that you can to the following:
> >
> >> (environment-import "elf/basic.scm" "foo:")
> >   #t
> >> (foo:map* (lambda (x) (* x 2)) #(1 2 3))
> >   (2 4 6)
> >> (map* (lambda (x) (* x 2)) #(1 2 3))
> >   (map* {jsint.Closure ??[1] (x)} #(1 2 3) )
> >
> >     ====================================
> >   SchemeException:[[ERROR: undefined variable "map*"""]]
> >
> > Note that "map*" is renamed to "foo:map*" for us, but internally it
> > still
> > uses the plain "iterate" binding that is always does.  You can also load
> > classes that use the normal "load()" method initialization convention:
> >
> >> (environment-import jlib.JLIB.class "jl:")
> >   #t
> >> (jl:menu "m1" (jl:menuitem "foo"))
> >   java.awt.Menu[menu0,label=m1,tearOff=false,isHelpMenu=false]
> >
> > If we pass #f for the prefix, nothing is prefixed; the function acts
> > like
> > "load".  Assigning prefixes at load time gives more flexibility than
> > depending on hierarchical module trees (c.f. namespace aliases in C++).
> > Note that generic functions are merged correctly, not treated like other
> > variables.
> >
> > This mechanism only works for libraries that are idempotent; it can't
> > matter
> > how many times you've loaded the library, or which of the times you've
> > loaded it you're referring to.  But a large class of Scheme code
> > satisfies
> > this restriction.  (Locking-down the new environments catches at least
> > some
> > violations of this rule.)
> >
> > The big chunks seem to be complete; i'm now looking for corner cases
> > that
> > might hide bugs.  Is this new functionality worth finishing up?
> >
> > Derek
> >
> > --
> > Derek Upham
> > Senior Software Engineer
> > Ontain
> > 1750 112th Ave NE, Suite C-245
> > Bellevue, WA 98004-3727
> > Tel: 425-460-1886
> > der...@on...
> >
> >
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf
> > _______________________________________________
> > Jscheme-devel mailing list
> > Jsc...@li...
> > https://lists.sourceforge.net/lists/listinfo/jscheme-devel
> >
>
>
>-------------------------------------------------------
>This sf.net email is sponsored by: DEDICATED SERVERS only $89!
>Linux or FreeBSD, FREE setup, FAST network. Get your own server
>today at http://www.ServePath.com/indexfm.htm
>_______________________________________________
>Jscheme-devel mailing list
>Jsc...@li...
>https://lists.sourceforge.net/lists/listinfo/jscheme-devel