exprla-devel — This is the Developers' list for the for XPL

[exprla-devel] DTD or namespace?

[reid_spencer <ras...@re...>]

--- In xpl-dev@y..., "Aly Hopkins" <p061949c@p...> wrote:
In reply to Jonathan Burns' letter of 29 Apr. (Re: It's Us....), I
was thinking that we had the option of switching languages on the fly
by reference to a different namespace, therefore:

xpl:constant-declaration
xpl:input
xpl:dataType
xpl:if
xpl:test
xpl:operator

and so forth.

I suspect that he & I are really talking about the same thing, just
applied a little differently. I would be happier with leaving the DTD
fixed in stone but allowing XPL to call in any language via namespace
prefix. As indeed XML is pretty good at calling in ANY resource, in
any number of ways, from anywhere.

     *  *  *  *  *  *  *  *  *  *  *  *  *
   * (  (  (  (  (  (  (  (  (  (  (  (  (  *
   *  )  )  )  )  )  )  )  )  )  )  )  )  ) *
   * (  (  ( A l y   H o p k i n s (  (  (  *
   *  )  )  )  )  )  )  )  )  )  )  )  )  ) *
   * (  (  (  (  (  (  (  (  (  (  (  (  (  *
   *  )  )  )  )  )  )  )  )  )  )  )  )  ) *     *  *  *  *  *  *  *
 *  *  *  *  *  *
--- End forwarded message ---

[exprla-devel] Re: [xpl] It's Us....

[reid_spencer <ras...@re...>]

--- In xpl-dev@y..., Jonathan Burns <saski@w...> wrote:
Michael Lauzon wrote:

>  No, XPL has no relation to what you termed XBasic (is that a
> language),

[Gulp...]

Serves me right, I should have spelled the "XBasic" thing
out more precisely. This is one misunderstanding I need to
lay to rest right away.

So far, all that most of us know about what XPL should be
like, is (1) the tag set that you've posted, and (2) the
prototype code which Robert Hopkins posted. My comments
started out from Robert's code, and wandered off into my
thoughts on parsing mechanisms.

Robert, when he introduced his code, said: "Note resemblance
to QBasic." Because of this, and also because my present
concept of XPL is of a language whose grammar is extensible,
and which is therefore in a sense a framework of related
languages, I went and named Robert's code "XBasic" for
temporary reference.

I sure do not want the name to stick. But I did want to
point out something by using it, and I'll expand on it
as follows:

If we set out a list of tags, and some examples of their
use, and then say: "There; now we've defined XPL, once and
for all", then we'd be missing out on something very elegant
and powerful.

Which is, that possibly for the first time ever, the grammar
of a language need not be fixed in concrete from the start.
We have the DTD at our fingertips. Change the DTD and we have
a different version of the language. Even a different language,
to all intents and purposes. Your and Robert's tags give us
an XPL which resembles Basic, but with a little work, we could
adjust it to resemble say, Fortran or Pascal or C.

In the olden days, say the '70s, but it's still the
conventional wisdom, suggesting the idea of an "adjustable"
grammar for a language would have caused heart failures, and
got you thrown into a padded cell for keeps. "But ... compiler
writing is hard work! You need a scanner, and a parser, and
a memory model, and a code generator, and ... (etc). And they
will all be full of bugs, and hard enough to maintain as it is.
If we go adding "adjustments", we'll complicate the compilation
task beyond reason!" they would scream. And more to the point,
"Our clients depend on the fact that the language will be
stable. If the base language begins to spawn all sorts of
extensions, then every time they get a bug, they will have to
work out whether it's their code at fault, or the particular
language extension."

BUT. In the context of XML, these objections no longer have
the same force. Suppose we have designed a new document format,
for describing invoices or something. Does the end user need
to install a special program to view them? No way. Just download
the DTD, which the document itself knows, and maybe a skeleton
parser in Java, and the user has a default program ready-made,
which will traverse the document, check its validity and
print it out. With a little work on the default program, you
can have a document do much more interesting thengs than
print itself out. Like maybe this invoice can wake up once a
day, check the date against its creation date, and email the
reciever saying, "14 days left for payment". E-commerce is
going to be made out of stuff like this.

So, it's the same for programs. If we have a family of XPL
programs, and they all work on the same kind of data, then
okay, they can all be XPL documents using the one DTD.
But different programming languages have different
specialties. And if we have two applications using radically
different kinds of data, then it only makes sense to provide
two versions of XPL, each one with an extension for its
special data type.

Case in point. Suppose we want XPL to have tables in it.
A table is pretty simple to define: just a list of fields,
each named, each with a data type, and some attribute tags.
You'll want to sort it on some of the fields; you'll want
to display table documents in a scrolling window, etc.
LOT OF WORK TO WRITE IT, with just your tags, but it could
be done.

But now, the 21st Century alternative. Teach XPL to read
SQL. How? Easy! Just append a DTD for SQL grammar to the
standard XPL DTD, and add

    | sql-statement

to the standard <xpl-statement> definition. The SQL grammar
already includes the EBNF for tables, and a generic

sql-statement ::= sql-maketable | sql-get-table-row | ...

<sql-make-table> ::= "query" "[" sql-query "]"

<sql-query> ::= [ the usual synatx for SQL queries ]

etc..

There's a bit more handshaking, which I won't do into:
like how to let the SQL part know just what data types
you want it to accept as data fields, and how to search
and sort them. But basically it's a stroll in the park.

In the current programming paradigm, if the Boss suddenly
issues a change order: "Oh, and we want it to have SQL",
well then there's screaming, and running, and staff
explaining to their tearful partners why the vacation
will have to be rescheduled, oh, it's frightful.

I reckon, with XPL technology, the job can be done in
half an hour. If we see the possibilities her and now.

It is basically a matter of regarding DTDs as we now
regard object classes.

> yes...it is us that is doing this work from the ground up;

WOW.

We can make history.

> any suggestions of what I should put on the website?
>

Well, this is where we come in, I think.

Add in an "About Us" page, and then beat on us if we don't
submit our About-Me's. Name, optional email, occupation,
software specialties, favourite languages, and optional
link to a resume.

Then, Links. We should all start keeping lists of any URL
we discover, which is related to XPL, or even just rings
a bell. I would say that XML, Java and scripting-language
sites would take pride of place. It's our web page, so we
really want it to have resource and ideas sites there, where
we can refer to them easily.

And when we submit a site to you, we should also send a
micro-review; as in "Just found this, it's about converting
Pagemaker files to XML. They use GNU m4. Ugh." If we're
feeling energetic, we can submit in-depth reviews. That
will add value to the site, and get people looking at us.

Once we have consensus on the XPL concept, we can add that.
And as soon as we do, you'll need to format up a dateline
page:

13-5-00    Java tutorials available

20-5-00    XP runs minimal parser

27-5-00    XP internals report, <URL>...</URL>
           XP parses <if>, <elseif>, <do> tags

and so on like that.

But mainly, twist our arms til we submit content.

[ sigh ]  Afraid I have to get back to the boring stuff now.

Goodwill
Jonathan

They finally realized that the ONE thing they just could not STAND
was a smartass commenting decompiler.
--- End forwarded message ---

[exprla-devel] [xpl] More Weirdness

[reid_spencer <ras...@re...>]

--- In xpl-dev@y..., Jonathan Burns <saski@w...> wrote:
Thanks, Garreth and Aly especially, for the affirmation.

I'll deal with Aly's ideas in a later post. Right now I
have to get these points down before I forget them.

1. The big issues are:

    What can we do with XML documents?
and
    What can we do to XML documents?

The first defines the area of XPL programming. For instance,
we can always parse them, and we can always bind operations
to elements, so that as a document tree is traversed in
parsing, and successively higher-level elements are recognized,
the operations are called. This is the generic scripting
language strategy, as employed with Perl, Tkl, etc. Very
significantly, there is XUL (pronounced so as to evoke
Sigourney Weaver in flame-red satin, i.e. "Zool"). XUL stands
for "eXtensible User-interface Language", and is being
used by the Mozilla open-source crowd to define much or all
the UI for the upcoming Mozilla beta release. This is something
I desperately want time to spend on. I haven't been into
the docs yet, but what I do know is, XUL elements are UI
widgets; mouse events must parse a XUL document, to discover
the element the mouse is focussed on; and Javascript
operations are bound to the events for each element.

Further, we know we can capture algorithms with xpl-basic
documents, such as we're now designing.

And there's more, my friends, where that came from! I'm
barely started with XPL functionalities.

Now here's what's wild. What we can generically do with
XML documents is exactly the same thing. Most XML types
exist to have operations (e.g. set font; make box; embed
text in Postscript, etc etc) bound to them.

I've got this book, XML Complete, Steve Holzner, McGraw-Hill.
Don't bother with it, except as an example to avoid. What
it will try to sell you on is that

    for every DTD
        for every application of that XML type
            you're gonna have to hand-write a Java
            program, containing all the app's operations
            and implementing their binding to the elements
            which trigger them

Which is nuts. Newbie though I be, even I can see that:

    If you have a generic binding point element in some DTD,
    say:

    <binding>
        <element in any other DTD>
        <reference to operation>
    </binding>

    and if you include the second (target) DTD in a copy
    of the first (application) DTD

    then you have enough information about any target
    element (its child elements, its basic types, heck,
    the whole EBNF sub-grammar defined for it) to pass
    references to the contents of an instance of the
    element through to the operation.

    Typically, the bundle of operations you'll call will
    be the methods of a single class: the interface to
    the execution methods you want the target document
    to call when it is parsed.

So what we can do, in principle, is this:

    Take one DTD for target.

    Take another DTD to represent sets of operations.
    There probably is one already. An XML document of
    this type will represent a pure class interface,
    exactly like a COM interface.

    Make up a document which consists of <binding>
    elements as above; with the <element> child of
    each binding being from the target document, and
    the <operation> child being from the interface
    document.

These three documents in combination specify everything
that a hand-coded Java binding specifies. So, except for
the generic process of automating the provision of target
element content to the operations, who needs Java? Well,
okay, we might want to write the operations in Java, but
that's not even typical. Perl has bindings for every
language in the known universe - Javascript has been
mentioned, but there are class interfaces, C libraries,
hardware APIs, and so on and on. Perl has shown the way.

I'm being vague (it's 5:45 am here), but once I've got
it properly worked out, I think I can convince you of it.
Bottom line: XPL needs <operation>, <element> and <binding>
built in. Because then it can bind and parse, re-bind and
parse again, according to whatever control flow we care
to write into an XPL program - i.e. in xpl-basic!!!

As an example of what I mean: Here as target type we have
the DTD for VRML 3D models; and here we have two interface DTDs,
one for SVGA and one for OpenGL. We should be able to write
an XPL program type, as a DTD which includes all three of
those other ones, so that an XPL document of this new
compound type can switch between interfaces conditionally
on whichever is available or preferred.

With our xpl-basic facilities, we should even be able to
do arithmetic on the VRML contents, and then pass the
results to the interface.

Which means, we have endowed VRML with arithmetic! And control
flow! Without doing a flaming thing to it internally!

Such is the power of the XPL concept.

Gee, and I'm only up to Point 1. :->

Back soon
Jonathan

God is on the side of the man with the most coloured balloons.
- Arthur Daley
--- End forwarded message ---

[exprla-devel] Re: [xpl] SOME IDEAS

[reid_spencer <ras...@re...>]

--- In xpl-dev@y..., "Garreth Galligan" <garreth.galligan@o...> wrote:

Well I agree with pretty much everything Jonathan has said, 
particularly 12.1 and 12.2 given the group's current lack of low-
level engineering experience (myself included). May as well approach 
something we have a realisitc chance of finishing.

>    AN XPL  DOCUMENT TYPE IS A PROGRAMMING LANGUAGE, 
>    AND ITS DTD IS  ITS GRAMMAR. DOCUMENTS OF ONE 
>    XPL TYPE ARE PROGRAMS IN  ONE  XPL LANGUAGE.

Very good :) Now the X in XPL is defined. I think this is the nearest 
we have got to a 'mission statement' of definition for XPL. Anyone?

A first possible step would be to compile a list of reference URLs 
related to some of the issues raised here. That way we can all taken 
a dip in a common knowledge-base.


  ----- Original Message ----- 
  From: Jonathan Burns 
  To: xpl@e... 
  Sent: Saturday, April 29, 2000 2:43 AM
  Subject: Re: [xpl] SOME IDEAS


  Michael Lauzon wrote: 
    
  >  I have made some XPL tags, which when I have time I'll 
  > upload them as a text file to our groups file site on Onelist. 
  > We can go from there,though this is a good start...also remember 
  > XPL is supposed to be a  compiled language, so if anyone knows 
  > how to program in Java (while  also using the XML parser [for 
  > Java] from James Clark; http://www.jclark.com/) would be 
  > helpful.  :) 
    
  Michael, are there working documents, any notes or old 
  email, on how XPL was conceived? Are we going to be the 
  ones who define the original charter? I ask because I'll 
  be injecting some weird stuff below, and I don't want to 
  be trying to hijack the group against its real intent. 
  (Later...) I've scanned your proposed tags. They're enough 
  to prototype an algorithmic language (like C, Pascal, Basic); 
  right now that's what we need. 

  On with the show... 
    
    
  THOUGHTS ON PARSING XPL(s) 

  Thanks to Robert Hopkins, for prototyping some XML-ized 
  Basic-like source code (dubbed "XBasic" here). I think 
  it throws a lot of major issues into relief... 

  1. First and fundamentally, XBasic is meant to be parsed. 
  There will be an XBasic Grammar, in the form of a DTD, 
  and an XBasic Document Parser,  say in the form of a  Java 
  module. 

  The Grammar determines  whether a block of text is truly 
  XBasic. This lets us guarantee that only correct XBasic 
  ever gets past  the Parser. 

  2. There is an intimate relationship  between XML document types 
  and programming languages:  both are defined by grammars. 

  This is a very strong  condition, and it provides enormous power. 
  All the flexibity of programming languages, and of XML document 
  types, is due to it. 

  3. The XBasic example implies: 

      AN XPL  DOCUMENT TYPE IS A PROGRAMMING LANGUAGE, 
      AND ITS DTD IS  ITS GRAMMAR. DOCUMENTS OF ONE 
      XPL TYPE ARE PROGRAMS IN  ONE  XPL LANGUAGE. 

  To me, this is the heart of XPL. 

  (So now I've clubbed you over the head with that ...) 

  4. In computer science tradition, XPL document types or 
  DTDs are called "regular languages". There is a huge body 
  of published results on "regular languages and automata". 
  The main users of these results are compiler  programmers, 
  and so every text in compiler design begins  with an 
  introduction to regular languages, and how to define them 
  in Backus-Naur Form (BNF). BNF has been  slightly expanded, 
  and integrated with lexical scanning, resulting in 
  Extended BNF (EBNF), in which DTDs are written. 

  5. What is an "automaton"? It is a finite state machine 
  which parses input according to a grammar. More obviously, 
  it is a lookup table, in which a parser looks up 

  {present machine state, incoming token} -> {next machine state} 

  And the point of this is, that the particular entries 
  in the lookup table are exactly what the grammar requires; 
  so that a correct sequence of tokens (e.g. XBasic elements) 
  causes the machine to pile up more and more recognized 
  elements, until it arrives at a state called, say, 

      "<XBasic-Parser-State:XPL-Document>" 

  at which it has recognized the "total document element", 
  so to speak; whereas an incorrect sequence of tokens 
  will lead it to a state called 

      "<XBasic-Parser-State:Error-condition> (error-number)" 

  for one error number or another. 

  <footnote-mark> * </footnote-mark> 

  6. What are these "tokens"? Well, they  are integers. 

  In a BNF parser, the elements of the grammar(usually 
  beginning with characters, punctuation marks and strings) 
  form a numbered list. Each element is known internally 
  by its token, which is its number in the list. 

  When the automaton is in one state and waiting for input, 
  the input comes as a token, which determines which particular 
  element is incoming. 

  (The content of the element is available  at this time, 
  but that is irrelevant to which element type is being 
  tested. And it's the type that determines grammatical 
  correctness.) 

  7. The key result in the theory of regular languages is, 

         FOR EVERY REGULAR LANGUAGE, THERE IS 
         AN AUTOMATON WHICH PARSES IT. 

  Better still, there are somewhat complex algorithms 
  which take in a grammar, and produce the very  automaton 
  (i.e. element list and lookup table) which parses it. 

  XML, and all markup languages, are regular AND SIMPLE. 

  In truth, they are a compiler-writer's dream, because 
  the tags can guide the parsing process. This eliminates 
  so much ambiguity and second-guessing, such as occurs 
  with non-tagged languages like C. With most programming 
  languages, a sequence of a few minor elements will not 
  be enough to specify what larger element is being 
  recognized. Nested tags eliminate this problem. Parsing 
  XPLs can therefore be fast. AND EASILY PROGRAMMED. 

  8. So fine for theory. Is there a free software tool 
  which does the job for us mortals? There is. 

  The tool is Yacc (or if we're in a GNU C environment, 
  Bison). Yacc takes a BNF grammar, and generates a parser 
  for it, in the form of a C binding, an element list, 
  and a quite large lookup table. The latter two form the 
  automaton for the grammar. 

  (The C binding is mostly for  handling the content of elements, 
rather than their type.) 

  Yacc output looks hideous. But once we have compiled it, 
  with a standard C  compiler, and linked the result into our 
  favourite C  program, then our C program has the ability 
  to parse  the grammar. With the speed of a greased weasel. 
  Because table lookup is as simple as can be. This is why 
  just about all compilers are built around Yacc parsers. 

  9. At this point, we have a rough analogy: 

  In C, the process goes like this: 

  Yacc: <BNF grammar> XBasic.y    ->    <lookup table etc> y.tab.c 
  gcc:  <C source> y.tab.c        ->    <C lib> y.tab.o 
  gcc:  <C lib> y.tab.o,main.o    ->    Executable, including XBasic 

  In Java, using say James Clark's XP parser, it goes: 

  XP: <EBNF grammar> XBasic.dtd   ->    <Java> com.jclark.xml.parse 

  The application's Java code is interfaced as callbacks to 
  the com.jclark.xml.parse, so that the parser can activate 
  the code as it recognizes elements. (At least I think that's it.) 

  10. Now let me emphasize that we have to go through one 
  of these processes, just to get a parser which recognizes 
  well-formed XBasic when it sees it. And whichever way we go, 
  the situation is already fairly messy. That is, I know how 
  messy the Yacc route can get; I'm not experienced in Java, 
  so the XP route may be less messy than it looks. 

  The Java in [XML Complete], which is called from Microsoft's 
  MSXML  parser, looks obscure to me - but then, callback code 
  always  does. For what it's worth, James Clark's  expat, a C 
  toolkit for adding XML parsing to C programs,  is being built 
  into Netscape 5 and Perl - so clearly he knows what he's 
  doing, and the open-source communities must agree. 

  11. If you feel you're getting out of your depth here, 
  don't worry. The essence of parsing is this two-step 
  process: 

  (1) Parser-Generator : Grammar -> Parser 
  (2) Parser: Correct XBasic -> "This is a true XBasic document." 
      or 
      Parser: Anything Else -> "This is not XBasic, and <here> 
                             is the first element which is 
                             not as expected." 
    
  I'm saying that there is already a body of theory about this, 
  and a free C-based parser-generator technology. Any old-school 
  compiler hacker would reach for the Yacc manual without a 
  second thought. However, (1) Java is cross-platform, and 
  downloadable via HTML, and (2) EBNF may not be easy to 
  convert to good old BNF. So Java parser-generators may 
  be the  way to go, and also Java parsers. 

  Comparing Yacc with the Java-based XP: 

  11.1  Yacc would probably produce an XPL parser faster; 
        but since loading and running an XPL application 
        requires the parser-generator to be produced ONCE, 
        and XPL programs to be parsed MANY TIMES, this is 
        not a compelling advantage by itself. 

  11.2  However, the XPL parser which Yacc produces would 
        certainly be faster: as in, There goes that greased 
        weasel. Producing a parser in the form of an automaton 
        is equivalent to compiling the parser. This is an 
        advantage which should eventually be addressed. 
        (I'd be interested in the exercise of writing a 
        lookup function for the Java Virtual Machine (JVM). 
        Half as fast as that there weasel, and cross-platform, 
        and dynamically downloadable like Java.) 

  11.3  Yacc may be overkill for markup languages, where 
        the elements are introduced by tags. 

  11.4  Yacc would create an XPL parser in the form of 
        a pair of C data structures, i.e. a list of 
        tokens (elements) and an array of integers 
        (lookup table). A little programming work would 
        be required to convert this into Java (but we 
        would only have to do that work once EVER). 

  The one thing we should NOT do, is hand-code a Java 
  program specifically for parsing XPL programs. Because 
  every time we changed the XPL specification during 
  development, we would have to modify the parser, and 
  debug and test it. This would slow research and 
  development badly, and unnecessarily. 

  So it seems that 

      parser-generator: xpl.dtd -> xpl.parser 

  is the only way to go. 

  12. Interestingly to me, some of the most active members 
  of the group are pretty new to programming, just as I am 
  pretty new to Java and XML. This means: 

  12.1 We should NOT take any long detours through C programming. 

  12.2 We SHOULD get as much leverage as possible from existing 
       Java efforts, such as James Clark's. 

  So, regardless of any advantages I think Yacc has, it should 
  not sway the group from its path in this experimental phase. 

  In other words, we should be following up Michael's and 
  Robert's initiative: to specify a straightforward 
  algorithmic language like Basic, and implement the parser 
  on James Clark's XP. 

  13. HOWEVER, I think there's a lot more to the total 
  situation than that. All the above is only the beginning 
  of what the XBasic example says to me. 
    

  <smiley> Whew. <\smiley> 
    

  Just about enough for now, but one last thing. 

  I think I see a niche for myself here, filling out the FAQ. 
  Can we settle on a mail-title prefix, say [xpl.faq], for 
  people to send in likely-to-be-frequently-asked questions? 
  I'll make it my business to collate, edit and recirculate 
  all answers, or else make up my own. 

  ASK ANYTHING, people. The more obvious, the better. 

  Aly's "What, in a very small nutshell, is compilation?" 
  is PERFECT for the Web community which we will want to 
  entertain, enlighten, and draw into our orbit. I'm going 
  to work hard on this one, it's a foundation stone. 

  More anon 
  Jonno 
    
  <footnote> "*", 
      <arfur-daley> 
      Yer gotta 'ave at least one sentence, that's full of 
semicolons, nested clauses, parentheses and 
      indentations and occupies the paragraph, Terence. Otherwise the 
punters will lose respect. 
   </arfur-daley> 
  </footnote> 
    

   
    
    
    
    
      


----------------------------------------------------------------------
--------
  pcworldani.gif 
----------------------------------------------------------------------
--------
  To unsubscribe from this group, send an email to:
  xpl-unsubscribe@o...
--- End forwarded message ---

[exprla-devel] More Wierd Ideas

[reid_spencer <ras...@re...>]

--- In xpl-dev@y..., "Aly Hopkins" <p061949c@p...> wrote:
Wow, awsome msg, Jonathan Burns. Did I understand half of it? While
we're on the subject of "wierd ideas", let's throw out the whole
notion of "structured programming" & "modular programming", as xml is
by it's nature modular; that is, any element in an xml document is
structurally identical to any other, or, the tree is made up of
smaller trees. There's no place in such a fractal structure for
"modules" and "sub-programs". Any element is directly addressable
from any other element by XPath language. Consider the network of
inter-relationships that can be established within one program .
Frightening thought, that we can be drowning in a sea of GOTO's, but
all of this is inherent to the structure of xml.

<unrelated_idea>
Throw all graphics procedures out of xpl, & make it completely
compatable with svg (& xsl). (Wish I knew more about svg.)
</unrelated_idea>

     *  *  *  *  *  *  *  *  *  *  *  *  *
   * (  (  (  (  (  (  (  (  (  (  (  (  (  *
   *  )  )  )  )  )  )  )  )  )  )  )  )  ) *
   * (  (  ( A l y   H o p k i n s (  (  (  *
   *  )  )  )  )  )  )  )  )  )  )  )  )  ) *
   * (  (  (  (  (  (  (  (  (  (  (  (  (  *
   *  )  )  )  )  )  )  )  )  )  )  )  )  ) *
     *  *  *  *  *  *  *  *  *  *  *  *  *
--- End forwarded message ---

[exprla-devel] Re: [xpl] SOME IDEAS

[reid_spencer <ras...@re...>]

--- In xpl-dev@y..., Jonathan Burns <saski@w...> wrote:
Michael Lauzon wrote:

>  I have made some XPL tags, which when I have time I'll
> upload them as a text file to our groups file site on Onelist.
> We can go from there,though this is a good start...also remember
> XPL is supposed to be a  compiled language, so if anyone knows
> how to program in Java (while  also using the XML parser [for
> Java] from James Clark; http://www.jclark.com/) would be
> helpful.  :)

Michael, are there working documents, any notes or old
email, on how XPL was conceived? Are we going to be the
ones who define the original charter? I ask because I'll
be injecting some weird stuff below, and I don't want to
be trying to hijack the group against its real intent.

(Later...) I've scanned your proposed tags. They're enough
to prototype an algorithmic language (like C, Pascal, Basic);
right now that's what we need.

On with the show...


THOUGHTS ON PARSING XPL(s)

Thanks to Robert Hopkins, for prototyping some XML-ized
Basic-like source code (dubbed "XBasic" here). I think
it throws a lot of major issues into relief...

1. First and fundamentally, XBasic is meant to be parsed.
There will be an XBasic Grammar, in the form of a DTD,
and an XBasic Document Parser,  say in the form of a  Java
module.

The Grammar determines  whether a block of text is truly
XBasic. This lets us guarantee that only correct XBasic
ever gets past  the Parser.

2. There is an intimate relationship  between XML document types
and programming languages:  both are defined by grammars.

This is a very strong  condition, and it provides enormous power.
All the flexibity of programming languages, and of XML document
types, is due to it.

3. The XBasic example implies:

    AN XPL  DOCUMENT TYPE IS A PROGRAMMING LANGUAGE,
    AND ITS DTD IS  ITS GRAMMAR. DOCUMENTS OF ONE
    XPL TYPE ARE PROGRAMS IN  ONE  XPL LANGUAGE.

To me, this is the heart of XPL.

(So now I've clubbed you over the head with that ...)

4. In computer science tradition, XPL document types or
DTDs are called "regular languages". There is a huge body
of published results on "regular languages and automata".
The main users of these results are compiler  programmers,
and so every text in compiler design begins  with an
introduction to regular languages, and how to define them
in Backus-Naur Form (BNF). BNF has been  slightly expanded,
and integrated with lexical scanning, resulting in
Extended BNF (EBNF), in which DTDs are written.

5. What is an "automaton"? It is a finite state machine
which parses input according to a grammar. More obviously,
it is a lookup table, in which a parser looks up

{present machine state, incoming token} -> {next machine state}

And the point of this is, that the particular entries
in the lookup table are exactly what the grammar requires;
so that a correct sequence of tokens (e.g. XBasic elements)
causes the machine to pile up more and more recognized
elements, until it arrives at a state called, say,

    "<XBasic-Parser-State:XPL-Document>"

at which it has recognized the "total document element",
so to speak; whereas an incorrect sequence of tokens
will lead it to a state called

    "<XBasic-Parser-State:Error-condition> (error-number)"

for one error number or another.

<footnote-mark> * </footnote-mark>

6. What are these "tokens"? Well, they  are integers.

In a BNF parser, the elements of the grammar(usually
beginning with characters, punctuation marks and strings)
form a numbered list. Each element is known internally
by its token, which is its number in the list.

When the automaton is in one state and waiting for input,
the input comes as a token, which determines which particular
element is incoming.

(The content of the element is available  at this time,
but that is irrelevant to which element type is being
tested. And it's the type that determines grammatical
correctness.)

7. The key result in the theory of regular languages is,

       FOR EVERY REGULAR LANGUAGE, THERE IS
       AN AUTOMATON WHICH PARSES IT.

Better still, there are somewhat complex algorithms
which take in a grammar, and produce the very  automaton
(i.e. element list and lookup table) which parses it.

XML, and all markup languages, are regular AND SIMPLE.

In truth, they are a compiler-writer's dream, because
the tags can guide the parsing process. This eliminates
so much ambiguity and second-guessing, such as occurs
with non-tagged languages like C. With most programming
languages, a sequence of a few minor elements will not
be enough to specify what larger element is being
recognized. Nested tags eliminate this problem. Parsing
XPLs can therefore be fast. AND EASILY PROGRAMMED.

8. So fine for theory. Is there a free software tool
which does the job for us mortals? There is.

The tool is Yacc (or if we're in a GNU C environment,
Bison). Yacc takes a BNF grammar, and generates a parser
for it, in the form of a C binding, an element list,
and a quite large lookup table. The latter two form the
automaton for the grammar.

(The C binding is mostly for  handling the content of elements, rather
than their type.)

Yacc output looks hideous. But once we have compiled it,
with a standard C  compiler, and linked the result into our
favourite C  program, then our C program has the ability
to parse  the grammar. With the speed of a greased weasel.
Because table lookup is as simple as can be. This is why
just about all compilers are built around Yacc parsers.

9. At this point, we have a rough analogy:

In C, the process goes like this:

Yacc: <BNF grammar> XBasic.y    ->    <lookup table etc> y.tab.c
gcc:  <C source> y.tab.c        ->    <C lib> y.tab.o
gcc:  <C lib> y.tab.o,main.o    ->    Executable, including XBasic

In Java, using say James Clark's XP parser, it goes:

XP: <EBNF grammar> XBasic.dtd   ->    <Java> com.jclark.xml.parse

The application's Java code is interfaced as callbacks to
the com.jclark.xml.parse, so that the parser can activate
the code as it recognizes elements. (At least I think that's it.)

10. Now let me emphasize that we have to go through one
of these processes, just to get a parser which recognizes
well-formed XBasic when it sees it. And whichever way we go,
the situation is already fairly messy. That is, I know how
messy the Yacc route can get; I'm not experienced in Java,
so the XP route may be less messy than it looks.

The Java in [XML Complete], which is called from Microsoft's
MSXML  parser, looks obscure to me - but then, callback code
always  does. For what it's worth, James Clark's  expat, a C
toolkit for adding XML parsing to C programs,  is being built
into Netscape 5 and Perl - so clearly he knows what he's
doing, and the open-source communities must agree.

11. If you feel you're getting out of your depth here,
don't worry. The essence of parsing is this two-step
process:

(1) Parser-Generator : Grammar -> Parser
(2) Parser: Correct XBasic -> "This is a true XBasic document."
    or
    Parser: Anything Else -> "This is not XBasic, and <here>
                           is the first element which is
                           not as expected."

I'm saying that there is already a body of theory about this,
and a free C-based parser-generator technology. Any old-school
compiler hacker would reach for the Yacc manual without a
second thought. However, (1) Java is cross-platform, and
downloadable via HTML, and (2) EBNF may not be easy to
convert to good old BNF. So Java parser-generators may
be the  way to go, and also Java parsers.

Comparing Yacc with the Java-based XP:

11.1  Yacc would probably produce an XPL parser faster;
      but since loading and running an XPL application
      requires the parser-generator to be produced ONCE,
      and XPL programs to be parsed MANY TIMES, this is
      not a compelling advantage by itself.

11.2  However, the XPL parser which Yacc produces would
      certainly be faster: as in, There goes that greased
      weasel. Producing a parser in the form of an automaton
      is equivalent to compiling the parser. This is an
      advantage which should eventually be addressed.
      (I'd be interested in the exercise of writing a
      lookup function for the Java Virtual Machine (JVM).
      Half as fast as that there weasel, and cross-platform,
      and dynamically downloadable like Java.)

11.3  Yacc may be overkill for markup languages, where
      the elements are introduced by tags.

11.4  Yacc would create an XPL parser in the form of
      a pair of C data structures, i.e. a list of
      tokens (elements) and an array of integers
      (lookup table). A little programming work would
      be required to convert this into Java (but we
      would only have to do that work once EVER).

The one thing we should NOT do, is hand-code a Java
program specifically for parsing XPL programs. Because
every time we changed the XPL specification during
development, we would have to modify the parser, and
debug and test it. This would slow research and
development badly, and unnecessarily.

So it seems that

    parser-generator: xpl.dtd -> xpl.parser

is the only way to go.

12. Interestingly to me, some of the most active members
of the group are pretty new to programming, just as I am
pretty new to Java and XML. This means:

12.1 We should NOT take any long detours through C programming.

12.2 We SHOULD get as much leverage as possible from existing
     Java efforts, such as James Clark's.

So, regardless of any advantages I think Yacc has, it should
not sway the group from its path in this experimental phase.

In other words, we should be following up Michael's and
Robert's initiative: to specify a straightforward
algorithmic language like Basic, and implement the parser
on James Clark's XP.

13. HOWEVER, I think there's a lot more to the total
situation than that. All the above is only the beginning
of what the XBasic example says to me.


<smiley> Whew. <\smiley>


Just about enough for now, but one last thing.

I think I see a niche for myself here, filling out the FAQ.
Can we settle on a mail-title prefix, say [xpl.faq], for
people to send in likely-to-be-frequently-asked questions?
I'll make it my business to collate, edit and recirculate
all answers, or else make up my own.

ASK ANYTHING, people. The more obvious, the better.

Aly's "What, in a very small nutshell, is compilation?"
is PERFECT for the Web community which we will want to
entertain, enlighten, and draw into our orbit. I'm going
to work hard on this one, it's a foundation stone.

More anon
Jonno

<footnote> "*",
    <arfur-daley>
    Yer gotta 'ave at least one sentence, that's full of semicolons,
nested clauses, parentheses and
    indentations and occupies the paragraph, Terence. Otherwise the
punters will lose respect.
 </arfur-daley>
</footnote>
--- End forwarded message ---

[exprla-devel] XPL

[reid_spencer <ras...@re...>]

--- In xpl-dev@y..., Michael Lauzon <ce940@f...> wrote:
I made a mistake with some of the tags I sent to the list, has anyone 
had 
a chance to take a look at them?  Here are the tags that need to be 
changed to Operators:

<and></and> = AND

<or></or> = OR

<eq></eq> = EQ

And here are two that can be taken out:

<variable></variable>

<value></value>

Okay, let's forget making XPL a compiled language for now, because 
this 
is Open Source, as well as making the engine that runs it Freeware.  
Let's give Miva (MivaScript [formally HTMLScript]) a run for it's 
users.  
All XPL programs must run with the extension .xpl; are there any 
programmers out there, if so...please speak up?!

Michael
http://www.geocities.com/SiliconValley/Way/9180/

'Eat, drink, and be merry, for tomorrow you may work.'
--- End forwarded message ---

[exprla-devel] Basic XPL tags....

[reid_spencer <ras...@re...>]

--- In xpl-dev@y..., "Michael Lauzon" <ce940@f...> wrote:
Okay, the person who did his idea of what XPL should look like is a 
good idea, but....  The language should be simple meaning it should
be 
basic and easy to use.  Here are some tags I made up, only two of
them 
have explanations, the rest should be self explanitory, if someone 
would like to write the explanations for the tags go ahead and do it.:

XPL is a XML-based programming language.  It should be simple to use 
so that anyone can learn it and add to it, if the need to do so is 
required.  It will be a compiled language, so that when someone 
updates the DTD the compiler will be scalable so it will recognise
the 
new tag(s) [if this is possible].  Here are some of the tags (all
tags 
attributes and their values MUST be in lowercase):

<xpl></xpl>; The XPL program will be written between these two tags.

<include content="xpl.dtd" version="x.x"/ >; This will be a stand 
alone tag that is required!  X is the number of the version.  As of 
this moment none of the other tags have attributes, but that will 
change.  Also, right now this is the only stand alone tag.

<comment></comment>; Anything between these two tags will be 
completely ignored by the compiler.

<stdin></stdin>

<stdout></stdout>

<if></if>

<do></do>

<loop></loop>

<try></try>

<then></then>

<variable></variable>

<where></where>

<for></for>

<function></function>

<public></public>

<let></let>

<when></when>

<slice></slice>

<not></not>

<splice><splice>

<else></else>

<elseif></elseif>

<unlock></unlock>

<lock></lock>

<run></run>

<end></end>

<private></private>

<class></class>

<start></start>

<stop></stop>

<case></case>

<switch></switch>

<name></name>

<value></value>

<and></and>

<or></or>
--- End forwarded message ---

[exprla-devel] SOME IDEAS

[reid_spencer <ras...@re...>]

--- In xpl-dev@y..., Robert A Hopkins <p061949c@p...> wrote:
I was fiddling around with the idea of an xml language and these are 
some 
ideas I had on what it might look like. Does any of this look at all 
like 
anything any of you had in mind? (Note resemblance to QBasic.)


<?xml version="1.0" xmlns:xpl="http://www.w3.org/something/or/other/"?
>
<xpl xmlns:xpl="http://www.w3.org/hoopla/about/xpl/">
  <!--Declare some constants.-->
  <xpl:constant-declaration>
    <xpl:constant name="qwerty" type="string">flagrent</xpl:constant>
    <xpl:constant name="pi" type="number">3.14168</xpl:constant>
    <xpl:constant name="gotAsparagus" 
type="boolean">true</xpl:constant>
  </xpl:constant-declaration>
  <!--Or maybe this way:-->
  <!--This is better. The constant is an element which contains a 
value.-->
  <xpl:constant-declaration>
    <qwerty type="string">flagrent</qwerty>
    <pi type="number">3.14168</pi>
    <gotAsparagus type="boolean">true</gotAsparagus>
  </xpl:constant-declaration>
  <!--Let's put some values into the variables "rows" & "columns".-->
  <xpl:input src="keyboard" type="integer">rows</xpl:input>
  <xpl:input src="keyboard" type="integer">columns</xpl:input>
  <!--Create a complex type to store coordinates within a grid and a 
boolean value representing a light turned on or off.-->
  <xpl:dimension>
    <!--I'm not entirely happy with "1 to rows" etc.-->
    <xpl:dataType name="lights" xpl:dimension-1="1 to rows" 
xpl:dimension-2="1 to columns">
      <lights>
        <xpl:data name="x" type="integer"/>
        <xpl:data name="y" type="integer"/>
        <xpl:data name="onOff" type="boolean"/>
      </lights>
    </xpl:dataType>
  </xpl:dimension>
  <!--Only one general type of loop is needed. Needs an attribute 
like 
xpl:counter="??"-->
  <!--Loops need work.-->
  <xpl:loop>
    <xpl:exit-before/>
    <xpl:exit-loop/>
  </xpl:loop>
  <!--Let's create a general-purpose conditional form. Is there a 
fundamental difference between "if...then" & "select...case"? Why not 
use 
exactly the same form for all conditional statements.-->
  <xpl:input src="keyboard" type="string">tgif</xpl:input>
  <!--A string entered from the keyboard is assigned to the variable 
"tgif".-->
  <xpl:if xpl:argument="tgif" xpl:test="boolean">
    <xpl:case>Thursday<isTgif>false</isTgif></xpl:case>
    <xpl:case>Saturday<xpl:output xpl:where-to="screen">Too 
late!</xpl:output></xpl:case>
    <xpl:case>Friday<isTgif>true</isTgif><xpl:output 
xpl:where-to="screen">Thangodd!</xpl:output><!--In the last case both 
the 
isTgif and the xpl:output elments are contained within the xpl:case 
element, so both are output when the value of tgif is found to be 
"Friday"--></xpl:case>
  </xpl:if>
  <!--Or this form:-->
  <xpl:if xpl:argument="lights{rows.columns}.x">
    <xpl:case 
xpl:operator="greater-than">75<lights
{rows.columns}.onOff>false</lights{rows.columns}.onOff></xpl:case>
  </xpl:if>
  <!--Now for an equation:-->
  <!--Rather than use the form "area=expression", I think that a 
variable 
element enclosing or containing a function which returns a value is 
better xml.-->
  <!--Not particularly easier to read, though.-->
  <xpl:input src="keyboard" type="number">radius</xpl:input>
  <area>pi * radius ^ 2</area>
  <xpl:output xpl:where-to="screen">area</xpl:output>
  <!--Brainstorm! How about using an xsl stylesheet to control output 
to 
the screen, to a file, even to a com port?-->
  <xpl:output xpl:where-to="screen">Hell, oh world!</xpl:output>
  <xpl:wait-a-sec xpl:until="true">
    <xpl:until xpl:string="uppercase">Q</xpl:until>
  </xpl:wait-a-sec>
  <xpl:end-program/>
</xpl>


               *** *  * ***  * *             
       * ** **   *  ** **     ***    **  *** 
       ** * * *   ***    *** ** *   **  ** * * *  
     * ** *   *  * ALY HOPKINS * ** *  * *  * ** * * 
   * * ** interplanetary illuminations * * * *** ** **
** ** * * *  *   * a specialty * ** * *   * *  * *  ** ** 
    ** * * * p061949c@p... * ** * **   * * **    
  *  * * * ******* * *   *   * *  * ** *  *  * ** *   *
   *    * ** * ** ** ***  * *  ** * ** * **   *** *  
     * * **   ***   * *   **  * ***   * *   * **                   
         * **  *   **  * *    *   * *   *
--- End forwarded message ---

[exprla-devel] Yahoo Group Email Transfer

[Reid S. <dev...@re...>]

*This message was transferred with a trial version of CommuniGate(tm) Pro*
Hello, my name is Reid Spencer. I'm a professional programmer, systems
architect, consultant, and open source hacker. I have the literate
programming skills, imagination and common sense to build XPL from the
ground up by myself, but I'd rather not.  Its so much more fun when we
design, discuss, build, and test together.  You won't find me haunting any
cathedrals so lets make this bazaar work.  I'm going to kickstart this
project and hope that some enlightened individuals will contribute as we
go along.

As my first gesture, I'm going to consolidate this project onto
SourceForge, including the discussion groups.   So, you should expect to
see a flurry of messages from me  as I transfer the "interesting" messages
from the Yahoo! Group xpl-dev.  Please don't flame me for this. I'm trying
to consolidate the specification, design, and development effort into one
place and SourceForge is THE one place these days.

Once I've done that and become familiar with the thoughts put forth so
far, I'll introduce my own unique ideas about XPL and what it should be
when it grows up.

Let's make the best programming language in the world that will still be
used when the next century marker rolls over.

Best Regards,

Reid Spencer
ras...@re...

[exprla-devel] First XPL thoughts (attached)

[reid_spencer <ras...@re...>]

--- In xpl-dev@yahoo..., Jonathan Burns <saski@w...> wrote:

Thanks to all who verified my mail. All seems well now.

Since the list is new, and has several beginners like me,
I'd like to share some general thoughts.

When I read that there was a proposal for an XML-based language,
it rang a bell - something about it seemed absolutely right.
But what? Exactly what is the advantage to be gained?

After thought, it seems to come down to these:

1. An XPL program is an XML document.

2. An XPL program operates on XML documents.

3. Put them together, and we have a reflective programming system,
   in which some programs can operate on other programs.

Now this is also true of Lisp-family languages, which also handle
headed hierarchic lists, including Lisp functions themselves.
What was missing here?

Well, obviously, programs are meant to be executed by interpreters,
while documents are supposed to be read and written, relatively
freely, by users.

A Lisp list makes a fairly horrible document to read. It could be
otherwise, if there were a standard formatting and typesetting
program, to display Lisp documents.

A Lisp list is also a horrible document to edit, because it is
easy to edit errors into the source code (all those parentheses).
It could be otherwise, if there were standard programs for finding,
cutting, pasting and editing Lisp documents, integrated into a
pleasant GUI, and preferably allowing the user to operate on
an active version of the typeset form of the document.

Neither "otherwise" was true in the days when Lisp had a wide
research community. However, the first is certainly true of XML,
and the second will be true. For one thing, the upcoming open-
source version of Netscape, Mozilla, is said to be "fully XML-
based"; exactly what this means, I am not certain, but there
will surely be support for a wide variety of text-document
style-sheets and data formatting like tables, and I would guess
that editable system parameters like menu lists will be XML
documents.

Which brings me to another reason that Lisp and related languages
failed to capitalize on simplicity and reflectivity: there was no
standard I/O interface for them to address. Java currently faces
the same problem; as soon as it addresses the file system provided
by a platform, its platform-independence vanishes, and its reflective
powers turn against its vaunted security.

However, if Netscape is your environment, and http your filesystem
interface, then as far as I can see, Apache can be your filesystem
implementation, permitting read/write privileges to HTML documents,
and that is all the capability you need for static documents.
The whole range of platform-specific security threats is thereby
nullified - although we will then have to build firewalls within
our Apache servers. (I think.)

Given all of this, I think an XPL could become an extremely
useful language for Web-content manipulation, in the same class
as Tcl, Python and Perl; and would handsomely reward whatever
effort we could give to it.


How should XPL be designed?

Above all, Interfaces. I used to be a skeptic about object-
oriented programming, but my experience with the Melbourne
Pattern Languages discussion group over the past year has
made me a believer.

That, and reading magazines mostly about C++ and Windows
programming, and about the huge struggles going on, to
define a style of programming which is both distributed and
platform-independent. Roughly, when we have data which is
tagged as being objects of a class, but the class implementation
could be anywhere in a network, then the only form of
communication known which is location- and implementation-
independent, is pure interface or protocol. Interfaces are
the currency of COM and CORBA, and I am certain that we
will only get the XML virtues into a language if we design
the language in terms of interfaces.


There's another aspect, though, which I don't have clearly
defined. What do you do with an XML document? You parse it;
that is, it's a tree, you traverse it. Unless this is the
final operation, what this gives you is another tree,
constrained by the same grammar, or a grammer related to it.

I can imagine an XPL program as a pipeline of smaller XPL
programs, modifying a document in turn, maybe with side-effects.
That's one style of programming.

I can also imagine taking a grammar, say grammar-1, and changing
one of the primitives, the leaf-tags, so that it now reads
<grammar-2-document>, within a new grammar-1.1; and thus building
ever deeper trees of data.

And I can also imagine a program that takes a grammar-1.1 document,
and reduces each grammar-2 subtree within it, replacing it with
a final grammar-1 leaf, and leaving a grammar-1 document behind.
Something of this kind is the essential interpretive mechanism of
Wolfram Research Mathematica, a rule-based expression substitution
system in which I've done a lot of work. There is also TXL, a
tree manipulation system which looks powerful, of which the older
versions are freely downloadable (not open-source, however).

I've had vague ideas for a couple of years, about designing
a Mathematica-type language based on lex/yacc, which would
apply reduction rules to a document, in the course of parsing
it. ("Bisonica" - should I copyright the name, or am I among
friends? :-))

XML would be a very apt place for working this idea out.


OK, enough for starters. Does any of this strike chords for
people here?

Goodwill
Jonno
--- End forwarded message ---

[exprla-devel] Hello!

[reid_spencer <ras...@re...>]

--- In xpl-dev@yahoo, "Michael Lauzon" <ce940@f...> wrote:
Hello all, and thanks for joining (hopefully more people will join
eventually).  One thing I ask is, for all members to update their
profiles please.  I will be uploading a text file with the XPL tags,
we need someone to work on the DTD, compiler (hopefully in Java using
XP from James Clark [http://www.jclark.com/xml/xp/]).  Also, we need
to find a programming (scripting?) language to use with the XPL tags
or create our own.  Please email me with what you want to work on.

Michael

P.S. I am looking for a place to host a homepage for XPL that has
CGI-BIN access for free, any ideas...please email me.  Thanks.
--- End forwarded message ---