sicsh-develop

Re: [sicsh-develop] Hello? Anyone here?

[Gary V. V. <ga...@gn...>]

Ken Weinert wrote:
> Just curious if anyone is still using this.
> 
> I got into in in a sort of backwards way - from the autotools book. I
> am attempting to learn the proper way of setting up a somewhat major
> open source project so I jumped into the complex version of the code.
> At first it was just for setup and such, but after looking at the code
> I thought maybe I'd try to use it as a base since my project is also
> designed to use "plugins" (loadable modules.)
> 
> I am having a small problem though. It doesn't look as if any of the
> modules are getting loaded. I figured that would be normal enough
> since I made all those changes so I thought I'd go back to the
> original and try and track down where I'd gone wrong in my changes.
> 
> Unfortunately the latest code downloaded from SF has exactly the same
> problem. from appearances none of the modules are loaded.
> 
> I'll presume this is some sort of basic dumb thing I've done and it
> will sound familiar and someone here will say "Did you try ...?" and
> I'll be back on track.
> 
> Thanks for any info.

Hi Ken,

Yep, I'm still here :-)

I expect the problem is that the sicsh code is bitrotted with respect
to the latest autotools.  The goat book is over 4 years old now, and
a lot has changed since then...

I'll have a little time in the new year to bring it up to date.  In
the mean time you might want to check out CVS GNU M4 which also has
a loadable module system built around libltdl.  It makes use of newer
features in CVS Libtool (branch-2-0), and is a bit of a pain to
bootstrap from CVS.  If you get stuck with this, or you'd rather see
a sicsh that works with current autotools, then ping me in the new
year and I'll do what I can to help.

Cheers,
	Gary.
-- 
Gary V. Vaughan      ())_.  gary@{lilith.warpmail.net,gnu.org}
Research Scientist   ( '/   http://tkd.kicks-ass.net
GNU Hacker           / )=   http://www.gnu.org/software/libtool
Technical Author   `(_~)_   http://sources.redhat.com/autobook

[sicsh-develop] Hello? Anyone here?

[Ken W. <kwe...@gm...>]

Just curious if anyone is still using this.

I got into in in a sort of backwards way - from the autotools book. I
am attempting to learn the proper way of setting up a somewhat major
open source project so I jumped into the complex version of the code.
At first it was just for setup and such, but after looking at the code
I thought maybe I'd try to use it as a base since my project is also
designed to use "plugins" (loadable modules.)

I am having a small problem though. It doesn't look as if any of the
modules are getting loaded. I figured that would be normal enough
since I made all those changes so I thought I'd go back to the
original and try and track down where I'd gone wrong in my changes.

Unfortunately the latest code downloaded from SF has exactly the same
problem. from appearances none of the modules are loaded.

I'll presume this is some sort of basic dumb thing I've done and it
will sound familiar and someone here will say "Did you try ...?" and
I'll be back on track.

Thanks for any info.
-- 
Ken Weinert   kwe...@gm...

[sicsh-develop] Re: Patch for sic-1.0b (common-h.in in sic/)

[Gary V. V. <gv...@te...>]

On Wednesday 18 April 2001  6:17 pm, you wrote:
> Hi,

Salut Guillaume!

> I'm very interesting by this project.

Great.  It's good to hear from you.

> So, I am reading the source to understand.
> If you have any docs, I will be happy to read it (I have already look
> at the autotools doc).

Actually, I started the project as a model solution for application of the 
autotools to a GNU style project layout, and to form the backbone of `GNU 
Autoconf, Automake and Libtool' from New Riders publishing.  You can read it 
online at http://sources.redhat.com/autobook, and it walks through the 
reasoning behind much of the code.  I'd be happy to answer any questions you 
have, especially if they reveal gaps in my reasoning behind the narrative in 
the related chapters in the Goat Book.

Eventually, I hope to make Sic useable as a language prototyper.  Currently I 
am working on a new lexical analyser which rebuilds an internal recogniser 
DFA on the fly as new tokens are added by runtime modules.  I also have some 
ideas about adding a byte compiler to speed up the back end.  You would be 
very welcome to join in development of these or other areas that interest you.

> Here a very little patch which is a simple macro for not cast malloc
> functions in C, and do it in C++.

Is it bad to cast in C++ like this?  (I am quite new to C++).  I have seen 
some projects use something similar but with the expansion of the macro being 
`static_cast(type)' in C++.  What do you think?

> I know it a detail. Do what you want with it.
> I often use it.

I have a bunch of changes which show the direction I am planning to take, but 
they are not quite complete enough to commit to CVS -- I am busy trying to 
release libtool-1.4, and m4-1.5, and with my new book so I haven't had time 
to polish this up yet.

I can mail you a tarball if you like.

Cheers,
	Gary.
-- 
  ___              _   ___   __              _         mailto: gv...@te...
 / __|__ _ _ ___ _| | / / | / /_ _ _  _ __ _| |_  __ _ ___       ga...@gn...
| (_ / _` | '_|// / |/ /| |/ / _` | || / _` | ' \/ _` | _ \
 \___\__,_|_|\_, /|___(_)___/\__,_|\_,_\__, |_||_\__,_|//_/
home page:  /___/                      /___/                  gpg public key:
http://www.oranda.demon.co.uk           http://www.oranda.demon.co.uk/key.asc

Re: [sicsh-develop] Redir and streams

[Gary V. V. <gv...@te...>]

On Thu, Apr 13, 2000 at 04:51:38PM -0400, Ezra Peisach wrote:
> 
> I indicated that I was planning to work on the redir module. (hopefully 
> I will have something working before I leave town next week). 

Anywhere interesting?

> Gary suggested three options for handling output streams:
> 
> >    i)  go ahead and add a FILE* stream for now so that you can keep
> >        going, and we'll worry about snprintfv later.
> >    ii) take a look at snprintfv and install it on your development box,
> >        and reference it as an external library for now.
> >    iii) take the bull by the horns and add snprintfv in a subdirectory,
> >       and make libsic dependent on it.
> 
> I have done a hodgepodge for now, based a little on the snprintv idea
> and a little craziness - which I will probably have to cleanup.

Heh =)O|

> In the struct sic, I have made a pointer to an opaque structure (similar to the
> builtintab) which represent the streams of the particular sic. 

Yup.

> [T]his Streams structure will eventually be made up of structures
> representing input and output streams. (maybe eventually g_stream in snprintv)

Definitely.

> Currently these substructures are a FILE *, a flag indicating if the
> stream needs to be closed (we don't want to close stdout).

Hadn't considered that.  Good point!

> Eventually, I will probably store a filename and other pertinent information).

There is already room for that sort of thing in the stream_gpointer
field of STREAM.

> I am still working on this, but interfacing to the streams will be
> sic_printf(Sic *sic,...), sic_flushout(Sic *sic) and other access
> methods as needbe.

Cool.

> I did this to allow me to rip the guts out of the stream handling once
> I figure out what is really needed. 
> 
> I am still trying to get the basic functionallity working before
> getting back to redir. What I am finding is that we might need to
> flush stdout before forking in dollar.c and unknown.c - otherwise the
> state of stdout is indeterminate - and we get repeats in the output
> stream. (this only shows up in non-interactive mode during the tests
> because stdout is unbuffered).
> 
> When I have an initial version working, I will submit a patch for
> review.

Excellent.  Thanks again Ezra.

Gary.

-- 
  ___              _   ___   __              _         mailto: gv...@te...
 / __|__ _ _ ___ _| | / / | / /_ _ _  _ __ _| |_  __ _ ___       ga...@gn... 
| (_ / _` | '_|// / |/ /| |/ / _` | || / _` | ' \/ _` | _ \      
 \___\__,_|_|\_, /|___(_)___/\__,_|\_,_\__, |_||_\__,_|//_/
home page:  /___/                      /___/                  gpg public key:
http://www.oranda.demon.co.uk           http://www.oranda.demon.co.uk/key.asc

Re: [sicsh-develop] Re: Syntax hooks

[Gary V. V. <ga...@or...>]

On Thu, Apr 13, 2000 at 04:25:21PM -0400, Ezra Peisach wrote:
> 
> 
> Sorry for not getting back sooner:

No worries... I'm busy too (as you know) =)O|

> >I wonder whether we also need to provide hooks at various points in
> >the code between building the input buffer and passing parts of the
> >Tokens* array into the evaluator...
> This would 

would what?

> >A cleaner way to implement `$' substitution would be let the dollar
> >module register a hook function after tokenisation that rewrites any
> >token that begins with a `$'.  The module would still need to provide
> >a syntax handler to keep all of $(foo bar baz) in a single token (or
> >maybe just a flag like we do for strings?).
> 
> This could also work with equals, etc. This makes the syntax tables
> responsible for the breaking the line into tokens. The Syntax_Hook can
> then handle the context specific requirements and rewriting. 

Cool.  I'll consider that as an endorsement.  Feel free to implement
it if you want, otherwise I'll get to it in the first few weeks of May.

> >The only sane way I can think of to implement globbing is to register
> >a hook function that rewrites the buffer contents with any glob
> >expansions done.  In this case the function would need to be called
> >before the syntax pass in tokenize().
> 
> I do not follow this reasoning. What exactly are you trying to expand
> with glob expansion? Filenames? Commands? Filenames could be handled
> at the Syntax_Hook level. Please elaborate.

The current mechanism simply can't express code to implement bourne
shell file globbing:

	$ ls *.[ch]
	
With an syntax hook function, words which contain unquoted globbing
meta-characters could be rewritten into the buffer as their expansion
(unless there are no matches, when the original string is kept --
meta-chars and all).


> Therefore, I like the Syntax_Hook, but am not convinced of the need
> for the init_hooks.

I think the globbing needs to happen before word splitting, otherwise
the glob expansion will be a single token (like $ expansion).  If it
is expanded before the word splitting phase, each expanded file would
be correctly alloted to a single token (think: `for file in *').  The
alternative is pass the expanded glob expression back through
tokenize, and squash the returned words into the array from before the
glob expansion.  Yuck.  init_hooks allow for much nicer code.

Cheers,
	Gary.
-- 
  ___              _   ___   __              _ mailto:ga...@or...
 / __|__ _ _ ___ _| | / / | / /_ _ _  _ __ _| |_  __ _ ___       ga...@gn... 
| (_ / _` | '_|// / |/ /| |/ / _` | || / _` | ' \/ _` | _ \      
 \___\__,_|_|\_, /|___(_)___/\__,_|\_,_\__, |_||_\__,_|//_/
home page:  /___/                      /___/                  gpg public key:
http://www.oranda.demon.co.uk           http://www.oranda.demon.co.uk/key.asc

[sicsh-develop] Redir and streams

[Ezra P. <epe...@zi...>]

I indicated that I was planning to work on the redir module. (hopefully 
I will have something working before I leave town next week). 

Gary suggested three options for handling output streams:

>    i)  go ahead and add a FILE* stream for now so that you can keep
>        going, and we'll worry about snprintfv later.
>    ii) take a look at snprintfv and install it on your development box,
>        and reference it as an external library for now.
>    iii) take the bull by the horns and add snprintfv in a subdirectory,
>       and make libsic dependent on it.

I have done a hodgepodge for now, based a little on the snprintv idea
and a little craziness - which I will probably have to cleanup.

In the struct sic, I have made a pointer to an opaque structure (similar to the
builtintab) which represent the streams of the particular sic. 

Shis Streams structure will eventually be made up of structures
representing input and output streams. (maybe eventually g_stream in snprintv)
Currently these substructures are a FILE *, a flag indicating if the
stream needs to be closed (we don't want to close stdout). Eventually,
I will probably store a filename and other pertinent information).

I am still working on this, but interfacing to the streams will be
sic_printf(Sic *sic,...), sic_flushout(Sic *sic) and other access
methods as needbe.

I did this to allow me to rip the guts out of the stream handling once
I figure out what is really needed. 

I am still trying to get the basic functionallity working before
getting back to redir. What I am finding is that we might need to
flush stdout before forking in dollar.c and unknown.c - otherwise the
state of stdout is indeterminate - and we get repeats in the output
stream. (this only shows up in non-interactive mode during the tests
because stdout is unbuffered).

When I have an initial version working, I will submit a patch for
review.

	Ezra

[sicsh-develop] Re: Syntax hooks

[Ezra P. <epe...@zi...>]

Sorry for not getting back sooner:

>I wonder whether we also need to provide hooks at various points in
>the code between building the input buffer and passing parts of the
>Tokens* array into the evaluator...
This would 

>A cleaner way to implement `$' substitution would be let the dollar
>module register a hook function after tokenisation that rewrites any
>token that begins with a `$'.  The module would still need to provide
>a syntax handler to keep all of $(foo bar baz) in a single token (or
>maybe just a flag like we do for strings?).

This could also work with equals, etc. This makes the syntax tables
responsible for the breaking the line into tokens. The Syntax_Hook can
then handle the context specific requirements and rewriting. 


>The only sane way I can think of to implement globbing is to register
>a hook function that rewrites the buffer contents with any glob
>expansions done.  In this case the function would need to be called
>before the syntax pass in tokenize().

I do not follow this reasoning. What exactly are you trying to expand
with glob expansion? Filenames? Commands? Filenames could be handled
at the Syntax_Hook level. Please elaborate.

Therefore, I like the Syntax_Hook, but am not convinced of the need
for the init_hooks.

    Ezra

[sicsh-develop] Syntax hooks

[Gary V. V. <ga...@or...>]

On Thu, Apr 06, 2000 at 12:54:02PM -0400, Ezra Peisach wrote:
> 
> We need a way to say - recognize, but don't execute. The same with
> $(eval). 

I wonder whether we also need to provide hooks at various points in
the code between building the input buffer and passing parts of the
Tokens* array into the evaluator...

A cleaner way to implement `$' substitution would be let the dollar
module register a hook function after tokenisation that rewrites any
token that begins with a `$'.  The module would still need to provide
a syntax handler to keep all of $(foo bar baz) in a single token (or
maybe just a flag like we do for strings?).

The only sane way I can think of to implement globbing is to register
a hook function that rewrites the buffer contents with any glob
expansions done.  In this case the function would need to be called
before the syntax pass in tokenize().

I haven't tried this... just playing for the moment:

typedef struct init_hook {
  struct init_hook *next;
  int (*func) (BufferIn *pin);
} Init_Hook;

typedef struct syntax_hook {
  struct syntax_hook *next;
  int (*func) (BufferOut *pout);
} Syntax_Hook;

typedef struct tokenize_hook {
  struct tokenize_hook *next;
  int (*func) (Tokens **ptokens);
} Tokenize_Hook;

typedef struct sic {
  char *result;			/* result string */
  size_t len;			/* bytes used by result field */
  size_t lim;			/* bytes allocated to result field */
  struct builtintab *builtins;	/* tables of builtin functions */
  SyntaxTable **syntax;		/* dispatch table for syntax of input */
  List *syntax_init;		/* stack of syntax state initialisers */
  List *syntax_finish;		/* stack of syntax state cleanup funcs */
  List *init_hooks;		/* pre-syntax hook functions */
  List *syntax_hooks;		/* post-syntax hook functions */
  List *tokenize_hooks;		/* post-tokenization hook functions */
  SicState *state;		/* state data from syntax extensions */
} Sic;

int
tokenize (sic, ptokens, pcommand)
     Sic *sic;
     Tokens **ptokens;
     char **pcommand;
{
  BufferIn in;
  BufferOut out;
  List *head;
  int status = SIC_CONTINUE;

  SIC_ASSERT (sic->syntax);

  bufferin_init (&in, *pcommand);
  bufferout_init (&out, in.buf.lim);

  /* Run any registered pre-syntax pass hooks. */
  for (head = (List *) sic->init_hooks; head; head = head->next)
    if ((*((Init_Hook *) head)->func) (&in) != SIC_OKAY)
      break;

  /* Perform any user initialisation for syntax readers. */
  for (head = sic->syntax_init; head; head = head->next)
    (*(SyntaxInit *) head->userdata) (sic);

  /* Dispatch to handlers by syntax class of character, or
     simply copy from input to output by default. */
  while (status == SIC_CONTINUE)
    {
      SyntaxHandler *handler = syntax_handler (sic, in.buf.start[in.buf.i]);

      if (handler)
	status = (*handler) (sic, &in, &out);
      else
	bufferout_append (&out, in.buf.start + in.buf.i++, 1);
    }

  /* Perform any client finalisation for syntax reader. */
  for (head = sic->syntax_finish; head; head = head->next)
    (*(SyntaxFinish *) head->userdata) (sic, &in, &out);
	
  /* Run any registered post-syntax pass hooks. */
  for (head = (List *) sic->syntax_hooks; head; head = head->next)
    if ((*((Syntax_Hook *) head)->func) (&out) != SIC_OKAY)
      break;

  {
    /* Can't fill ARGV on the fly incase BUF moved during realloc. */
    Tokens *tokens = XMALLOC (Tokens, 1);
    
    tokens->argv = XMALLOC (char *, 1+ out.offc);

    for (tokens->argc = 0; tokens->argc < out.offc; ++tokens->argc)
      tokens->argv[tokens->argc] = out.buf.start + out.offv[tokens->argc];

    tokens->argv[tokens->argc] = NULL;
    tokens->lim  = out.buf.lim;

    if (!*tokens->argv)
      XFREE(out.buf.start);
    XFREE (out.offv);

    *pcommand += in.buf.i;
    *ptokens = tokens;
  }

  /* Run any registered post-tokenisation pass hooks. */
  for (head = (List *) sic->tokenize_hooks; head; head = head->next)
    if ((*((Tokenize_Hook *) head)->func) (ptokens) != SIC_OKAY)
      break;

  return status;
}

Thoughts?

Cheers,
	Gary.
-- 
  ___              _   ___   __              _ mailto:ga...@or...
 / __|__ _ _ ___ _| | / / | / /_ _ _  _ __ _| |_  __ _ ___       ga...@gn... 
| (_ / _` | '_|// / |/ /| |/ / _` | || / _` | ' \/ _` | _ \      
 \___\__,_|_|\_, /|___(_)___/\__,_|\_,_\__, |_||_\__,_|//_/
home page:  /___/                      /___/                  gpg public key:
http://www.oranda.demon.co.uk           http://www.oranda.demon.co.uk/key.asc

[sicsh-develop] Re: Additional evaluation layer

[Gary V. V. <ga...@or...>]

On Thu, Apr 06, 2000 at 12:54:02PM -0400, Ezra Peisach wrote:
> 
> >I have been thinking (dangerous, I know) and it occurs to me that the
> >syntax mechanism could be extended to recognise tokens and mark them
> >in the Tokens* generated by tokenize() -- either in addition to, or
> >probably as a replacement for the string that is generated currently.
> 
> >A third type of module would then be possible to implement higher
> >level semantic extensions.  For example a loadable module could
> >install a syntax handler that recognises "if", "then", "else" and
> >"fi", and registers (enumerated) tokens `SYNTAX_IF', `SYNTAX_THEN',
> >`SYNTAX_ELSE' and `SYNTAX_FI'.  The tokenize() function would be
> >changed to recognise these tokens while it is building the Tokens* and
> >insert the enumerated values rather than the raw strings.
> 
> I like the idea. One of the concerns I had in writing the equals
> handler is understanding the context that the equals sign is to be
> used.

You have the right solution though:  `=' in leading expressions are
handled syntactically.  Other uses for equals would be determined by
the builtin that handles the rest of the Tokens*.

> Clearly "if a=1 then" is very different from "a=1" vs "typo a=1". The
> first a conditional, the second an assignment, the third a syntax error.

These all work with the current system:

  - if a=1 then ...
    `-> if
        a=1
	then
	...
    the `if' builtin would then extract the tokens between `if' and
    `then' and pass it to an expression evaluator.
    
  - a=1
    `-> syntax handler sets `a' and consumes the string
    
  - typo a=1
    `-> eval() will complain that `typo' doesn't exist, or else the
    `typo' builtin will diagnose the error.

> It would be nice if the recognition could happen durring the
> syntax_handler phase - before generation of Tokens *. Perhaps a syntax
> handler can indicate that it is okay to create tokens for everything
> to where I am. This would allow differentiation of the above test cases.

I can't think of any situation where the additional complexity of doing
this way would win anything.  If the syntax handler (for `=' say)
can't/doesn't want to handle an instance, it should just copy it to
the output buffer and punt to whichever builtin handler gets the argv
from Tokens*.

The clear advantage of having a new pass over the Tokens* is to
simplify reserved word recognition in the builtins.  I now think that
the enumerated token should be in addition to the string token, which
gives the additional advantage of being able to tag classes of syntax
(e.g. you could register a SYNTAX_NUMBER tokeniser which would tag
sequences of digits).  By registering these in the module that has
builtins that use them it would be much easier to write an `if'
builtin -- it would initially scan the for balanced SYNTAX_THEN and
SYNTAX_FI rather than comparing every element with "then" and "fi".

> >Finally, the new type of module entry point (in addition to syntax and
> >builtin entry points) would determine allowable enumerated token
> >orderings.  In this case, it would check that the above enumerated
> >tokens were well formed and correctly nested -- generating a SIC_ERROR
> >for semantic errors, SIC_INCOMPLETE if more lookahead is required, or
> >SIC_OKAY if the evaluator should search for a suitable builtin handler.
> I like the idea, but I am not sure how you plan to describe orderings.
> You need to be able to describe optional  orderings, vs required ones.
> 
> IF ... THEN ... ENDIF
> IF ....THEN ... ELSE ... ENDIF
> 
> are straight forward with ELSE being optional - but would you ever
> need syntax that is more complex. (we can say no and be done with it)
> 
> A complex example might be:
> a ... b ... c
> a ... c ... b ... d
> 
> Where d or c comes after b depending on if b comes after c or a. (a 
> bit contrived - but makes a description more difficult"

The `a' builtin would check for whatever combinations it wants to
allow.  No problem.

> >The builtin initialiser would also need extending to allow builtins to
> >be registered against tokens rather than just single characters.
> Logically...
> 
> 
> I think it is a good idea that deserves merit. The big win can be for
> the syntax handlers knowing the context. It also allows for real
> structure.

Yup.

> Long term, we eventuall need a way to disable builtin syntax handlers
> that have side affects.
> 
> Consider:
> if a=1 then a=2 else b=3

But the `=' syntax handler only sets variables which occur at the left
of a command.  Doesn't it?  

It will be the responsibility of the `if' (or presumably SYNTAX_IF)
handler to evaluate only the suitable portions.  The expression
between `if' and `then' (i.e. the argv elements between these two)
will be passed to (a new) expression evaluator which return true or
false, and will then recursively eval() *either* the commands (i.e.
argv elements) between `then' and the balanced `else',  or balanced
`else' and `fi'.  Balanced is the key to doing this properly, hence
the advantage of being able to scan through the argv looking for
unique tokens rather than zillions of calls to strcmp (or a hideous
big switch).

> currently in parsing the line a and b will be assigned. We need a way
> to say - recognize, but don't execute. The same with $(eval).

We already have it.  Just don't rescan the string with tokenize, and
the syntax handlers aren't run again.

>    Ezra

The bit I haven't thought right through yet, is how to build a Tokens*
with one pass over the input buffer.  The current method will redo all
of the syntax handling for the entire command each time it is rejected
as incomplete, just adding a new line to the input buffer until it
finally gets through.

By adding the new pass, we can recognise a big class of incomplete
commands before passing things into the builtin handler (once the
builtin handler has it and rejects it, we have no choice but to add
another line ad try again).   Instead of repeatedly adding another
line and letting the if builtin decide whether the command is complete
yet, we could register a token scanner that knows that a command must
have an equal number of SYNTAX_IF, SYNTAX_THEN and SYNTAX_FI tokens.
By the time the if builtin sees the argv, it is already balanced!

I think we must solve the former before the latter is worthwhile
though.

Cheers,
	Gary.
-- 
  ___              _   ___   __              _ mailto:ga...@or...
 / __|__ _ _ ___ _| | / / | / /_ _ _  _ __ _| |_  __ _ ___       ga...@gn... 
| (_ / _` | '_|// / |/ /| |/ / _` | || / _` | ' \/ _` | _ \      
 \___\__,_|_|\_, /|___(_)___/\__,_|\_,_\__, |_||_\__,_|//_/
home page:  /___/                      /___/                  gpg public key:
http://www.oranda.demon.co.uk           http://www.oranda.demon.co.uk/key.asc

Re: [sicsh-develop] Additional evaluation layer

[Ezra P. <epe...@zi...>]

>I have been thinking (dangerous, I know) and it occurs to me that the
>syntax mechanism could be extended to recognise tokens and mark them
>in the Tokens* generated by tokenize() -- either in addition to, or
>probably as a replacement for the string that is generated currently.



>A third type of module would then be possible to implement higher
>level semantic extensions.  For example a loadable module could
>install a syntax handler that recognises "if", "then", "else" and
>"fi", and registers (enumerated) tokens `SYNTAX_IF', `SYNTAX_THEN',
>`SYNTAX_ELSE' and `SYNTAX_FI'.  The tokenize() function would be
>changed to recognise these tokens while it is building the Tokens* and
>insert the enumerated values rather than the raw strings.

I like the idea. One of the concerns I had in writing the equals
handler is understanding the context that the equals sign is to be
used.


Clearly "if a=1 then" is very different from "a=1" vs "typo a=1". The
first a conditional, the second an assignment, the third a syntax error.

It would be nice if the recognition could happen durring the
syntax_handler phase - before generation of Tokens *. Perhaps a syntax
handler can indicate that it is okay to create tokens for everything
to where I am. This would allow differentiation of the above test cases.


>Finally, the new type of module entry point (in addition to syntax and
>builtin entry points) would determine allowable enumerated token
>orderings.  In this case, it would check that the above enumerated
>tokens were well formed and correctly nested -- generating a SIC_ERROR
>for semantic errors, SIC_INCOMPLETE if more lookahead is required, or
>SIC_OKAY if the evaluator should search for a suitable builtin handler.
I like the idea, but I am not sure how you plan to describe orderings.
You need to be able to describe optional  orderings, vs required ones.

IF ... THEN ... ENDIF
IF ....THEN ... ELSE ... ENDIF

are straight forward with ELSE being optional - but would you ever
need syntax that is more complex. (we can say no and be done with it)

A complex example might be:
a ... b ... c
a ... c ... b ... d

Where d or c comes after b depending on if b comes after c or a. (a 
bit contrived - but makes a description more difficult"


>The builtin initialiser would also need extending to allow builtins to
>be registered against tokens rather than just single characters.
Logically...


I think it is a good idea that deserves merit. The big win can be for
the syntax handlers knowing the context. It also allows for real
structure.

Long term, we eventuall need a way to disable builtin syntax handlers
that have side affects.

Consider:
if a=1 then a=2 else b=3

currently in parsing the line a and b will be assigned. We need a way
to say - recognize, but don't execute. The same with $(eval).

   Ezra

[sicsh-develop] Additional evaluation layer

[Gary V. V. <ga...@or...>]

I have been thinking (dangerous, I know) and it occurs to me that the
syntax mechanism could be extended to recognise tokens and mark them
in the Tokens* generated by tokenize() -- either in addition to, or
probably as a replacement for the string that is generated currently.

A third type of module would then be possible to implement higher
level semantic extensions.  For example a loadable module could
install a syntax handler that recognises "if", "then", "else" and
"fi", and registers (enumerated) tokens `SYNTAX_IF', `SYNTAX_THEN',
`SYNTAX_ELSE' and `SYNTAX_FI'.  The tokenize() function would be
changed to recognise these tokens while it is building the Tokens* and
insert the enumerated values rather than the raw strings.

Finally, the new type of module entry point (in addition to syntax and
builtin entry points) would determine allowable enumerated token
orderings.  In this case, it would check that the above enumerated
tokens were well formed and correctly nested -- generating a SIC_ERROR
for semantic errors, SIC_INCOMPLETE if more lookahead is required, or
SIC_OKAY if the evaluator should search for a suitable builtin handler.

The builtin initialiser would also need extending to allow builtins to
be registered against tokens rather than just single characters.

Thoughts?

Gary.
-- 
  ___              _   ___   __              _ mailto:ga...@or...
 / __|__ _ _ ___ _| | / / | / /_ _ _  _ __ _| |_  __ _ ___       ga...@gn... 
| (_ / _` | '_|// / |/ /| |/ / _` | || / _` | ' \/ _` | _ \      
 \___\__,_|_|\_, /|___(_)___/\__,_|\_,_\__, |_||_\__,_|//_/
home page:  /___/                      /___/                  gpg public key:
http://www.oranda.demon.co.uk           http://www.oranda.demon.co.uk/key.asc

Re: [sicsh-develop] sic: redir module

[Gary V. V. <ga...@or...>]

On Wed, Apr 05, 2000 at 11:20:41AM -0400, Ezra Peisach wrote:
> 
> I've started working on file redirection. (module redir.la)

Excellent!  Thankyou =)O|

You should add yourself to the AUTHORS file too methinks.

> Two issues:
> 
> All output is currently hodgepodge - individual modules use printf right now,
> errors are through the sic_result_... handler.

Indeed.  This does need some work, but I didn't think it through at
the time =(O|

> I will need to control the output...
> a) Should the modules be using sic_result_set/append for normal messages?
> b) If not, I need to provide a FILE * for these builtins to use for sending
> output. Should it be a global or a sicstate?
> 
> I prefer (a) - because then the I/O can happen at the top.
> But then there is the issue of stdout vs. stderr. Consider the unknown
> handler. What if we want the stdout to go one place and stderr another? This
> would imply (b) would be easier.

I nearly went for (a), but the problem here would be that perfectly
valid output might be generated deep within the call stack, and an
error (say at the end of a loop) that forces the stack to unwind (so
that the error is caught and displayed) would obliterate it.

After a bit more thought, I think the best solution is to add a stream
to the Sic structure and use that.  Ultimately, I plan to integrate my
snprintfv package (on my homepage) which would do the following:

    i)  add generalised streams so that we're not limited to FILE*
    ii) gives access to dynamically self allocating strings which are
        8-bit clean.
    iii) has all the frills of glibc printf and pals, plus a few more --
        (asprintf, vsprintf, dsprintf, %1$s format parsing) all of
        which can be used with impunity without worrying about providing
        fallback implementations for non-glibc targets.
       
You have three options at this point:

    i)  go ahead and add a FILE* stream for now so that you can keep
        going, and we'll worry about snprintfv later.
    ii) take a look at snprintfv and install it on your development box,
        and reference it as an external library for now.
    iii) take the bull by the horns and add snprintfv in a subdirectory,
       and make libsic dependent on it.

>       Ezra

Cheers,
	Gary.
-- 
  ___              _   ___   __              _ mailto:ga...@or...
 / __|__ _ _ ___ _| | / / | / /_ _ _  _ __ _| |_  __ _ ___       ga...@gn... 
| (_ / _` | '_|// / |/ /| |/ / _` | || / _` | ' \/ _` | _ \      
 \___\__,_|_|\_, /|___(_)___/\__,_|\_,_\__, |_||_\__,_|//_/
home page:  /___/                      /___/                  gpg public key:
http://www.oranda.demon.co.uk           http://www.oranda.demon.co.uk/key.asc

[sicsh-develop] sic: redir module

[Ezra P. <epe...@zi...>]

I've started working on file redirection. (module redir.la)

Two issues:

All output is currently hodgepodge - individual modules use printf right now,
errors are through the sic_result_... handler.

I will need to control the output...
a) Should the modules be using sic_result_set/append for normal messages?
b) If not, I need to provide a FILE * for these builtins to use for sending
output. Should it be a global or a sicstate?

I prefer (a) - because then the I/O can happen at the top.
But then there is the issue of stdout vs. stderr. Consider the unknown
handler. What if we want the stdout to go one place and stderr another? This
would imply (b) would be easier.

      Ezra

[Sicsh-develop] Testing

[Gary V. V. <ga...@or...>]

Am I subscribed yet?
-- 
  ___              _   ___   __              _ mailto:ga...@or...
 / __|__ _ _ ___ _| | / / | / /_ _ _  _ __ _| |_  __ _ ___       ga...@gn... 
| (_ / _` | '_|// / |/ /| |/ / _` | || / _` | ' \/ _` | _ \      
 \___\__,_|_|\_, /|___(_)___/\__,_|\_,_\__, |_||_\__,_|//_/
home page:  /___/                      /___/                  gpg public key:
http://www.oranda.demon.co.uk           http://www.oranda.demon.co.uk/key.asc

[Sicsh-develop] sic: memory leak in modules/dollar.c

[Ezra P. <epe...@zi...>]

Index: dollar.c
===================================================================
RCS file: /4/usr2/epeisach/CVSROOT/sic/modules/dollar.c,v
retrieving revision 1.1.1.2
retrieving revision 1.2
diff -c -r1.1.1.2 -r1.2
*** dollar.c	2000/03/31 18:43:09	1.1.1.2
--- dollar.c	2000/04/01 13:45:11	1.2
***************
*** 107,112 ****
--- 107,113 ----
  	else
  	  bufferout_append (out, value, -1);
  
+ 	XFREE(name);
  	status = SIC_CONTINUE;
        }
        break;