simpleparse-users — Primary discussion list for users (and developers) of the SimpleParse system

[SimpleParse] Re: Case insensitivity

[Paul P. <pau...@ho...>]

I tried the new version 2.0.1a2 and it works a treat. Strangely it seems
to run quicker on my grammar. For my application I get the following,
 
Case sensitive:
 
Ran 414 tests in 171.785s
 
Case insensitive
 
Ran 414 tests in 155.531s
 
 
Possible causes:
 
0. PC doing other things
1. In the second case 8 tests failed because the case insensitivity
showed a problem in my tests (I was accidentially using a lower case
keyword as a variable).
2. .... ?
 
I'll fix my tests and see what happens - but in any event, this is a
great feature!
 
Paul

[SimpleParse] Re: Case insensitivity

[Paul P. <pau...@ho...>]

Mike Fletcher wrote:

>Paul Paterson wrote:
> 
> > How do people usually handle case-insensitive matching?
> >
> Usually (until now) I've just hacked it by explicitly creating
[aA][tT] 
> ranges in the very few cases it came up.  However, given that it's an 
> occasional request, I've just added support for it to SimpleParse.
> 
> Usage is like so:
>
>     production := c"match without case" # not 'c' prefix
> 
> which will match at run-time as if the production had been
> spelled:
> 
>     production := ([mM],[aA],[tT],[cC],[hH],' ',[wW],...)
> 
> Implication being that case-insensitive literals are *far* heavier
than 
> regular literals (probably a few orders of magnitude slower), as under

> the covers they are actually creating a sequential group with literal 
> and range sub-elements.  Runs of multiple non-case-carrying letters
will 
> be treated as a single literal, BTW.

Wow - thanks! I'll try this out as soon as SF CVS catches up.

I'll do some timing comparisons and let you know what effect it has. I'm
not too concerned about absolute speed for my application because I
think that I can have an option for case insensitive searching (when I
will switch grammars). 

Paul
 

Re: [SimpleParse] Case insensitivity

[Mike C. F. <mcf...@ro...>]

Paul Paterson wrote:

> How do people usually handle case-insensitive matching?
>
Usually (until now) I've just hacked it by explicitly creating [aA][tT] 
ranges in the very few cases it came up.  However, given that it's an 
occasional request, I've just added support for it to SimpleParse.

Usage is like so:

    production := c"match without case" # not 'c' prefix

which will match at run-time as if the production had been
spelled:

    production := ([mM],[aA],[tT],[cC],[hH],' ',[wW],...)

Implication being that case-insensitive literals are *far* heavier than 
regular literals (probably a few orders of magnitude slower), as under 
the covers they are actually creating a sequential group with literal 
and range sub-elements.  Runs of multiple non-case-carrying letters will 
be treated as a single literal, BTW.

> My source text (VB) is generally well behaved with respect to case 
> (the VB IDE formats all keywords etc) but now I am planning on trying 
> to parse VBScript and I can't make the same assumption since a lot of 
> it is edited outside the VB IDE.
>
> The options seem to be:
>
> 1. duplicate all keywords in the grammar (but what if someone types 
> "SubRoutine"?) 2. convert all text to lower case prior to parsing (but 
> my text contains strings and variable names where case is important) 
> 3. parse the lower case text but then get the parse tree elements from 
> the original text
>
That might work (it's pretty easy too, as the dispatchprocessor has an 
argument for which source-string to use), but could mess up when a 
keyword is case-sensitive.

> 4. do a pre-parse to replace all keywords with a standard form 5. 
> assume everything is ok and try to recover when parsing fails 6. 
> something else ...
>
> Has anyone else "solved" this problem?
>
Hopefully the CILiteral feature will solve it for you.  Let me know,
Mike

_______________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/

[SimpleParse] Case insensitivity

[Paul P. <pau...@ho...>]

How do people usually handle case-insensitive matching?

My source text (VB) is generally well behaved with respect to case (the
VB IDE formats all keywords etc) but now I am planning on trying to
parse VBScript and I can't make the same assumption since a lot of it is
edited outside the VB IDE.

The options seem to be:

1. duplicate all keywords in the grammar (but what if someone types
"SubRoutine"?) 2. convert all text to lower case prior to parsing (but
my text contains strings and variable names where case is important) 3.
parse the lower case text but then get the parse tree elements from the
original text 4. do a pre-parse to replace all keywords with a standard
form 5. assume everything is ok and try to recover when parsing fails 6.
something else ...

Has anyone else "solved" this problem?

Paul

 

[SimpleParse] RE: More Simpleparse advice

[Paul P. <pau...@ho...>]

Mike C. Fletcher wrote:
> 
> Paul Paterson wrote:
>
> >
> >... And <a bad line here> doesn't parse, then the "line" 
> doesn't match 
> >and so I get a report that the "sub_definition" doesn't 
> match. This is 
> >entirely correct, but quite hard for the end-user to work 
> out what is 
> >going wrong in a long subroutine.
> >
> You are probably looking for a construct like so:
>     block := blockstart, !, blockcontent?, blockend
> 
> or more concretely...
> 
>     ifblock := 'If', condition, !, blockcontent, 'End',ts,'If'
> 
> That is, once you hit a block-start, you must hit a block-end 
> or there's 
> a syntax error, so you include the cut and a syntax error 
> will be raised 
> when the system can't find the rest of the construct.  

This is exactly what I was looking for! Many thanks again!

Paul

[SimpleParse] Re: More Simpleparse advice

[Mike C. F. <mcf...@ro...>]

Paul Paterson wrote:

>Mike,
>
>Thanks to your advice last time my VB grammar is now working very well.
>Simpleparse is just amazing! However, I wondered if you had any advice
>on how to handle parsing errors. When I throw some badly formed VB at
>the grammar then the parser quite rightly reports an error. The issue is
>that, since the grammar is deeply nested, the error is reported at the
>outermost structure and it takes a long time to work out which is the
>offending part.
>
>Have you had this issue before?
>
>Here's an simple example ... The grammar for a subroutine looks like
>this:
>
>sub_definition ::=
>             label_definition?, (scope, wsp*)?, "Sub", wsp+, identifier,
>
>             formal_param_list, line_end, block?, label_definition?,
>"End Sub"
>
>And the "block" looks like this,
>
>block ::= 
>             block_content+
>
>block_content ::=
>             ?-block_terminator, line
>
>And the "line" looks like ... you get the idea (it's deeply nested!)
>
>Now if I throw some VB that looks like,
>
>Sub A()
>   For I = ...
>	While ...
>         If ..
>            <a bad line here>
>         End If
>      Wend	
>   Next I
>End Sub
>
>... And <a bad line here> doesn't parse, then the "line" doesn't match
>and so I get a report that the "sub_definition" doesn't match. This is
>entirely correct, but quite hard for the end-user to work out what is
>going wrong in a long subroutine.
>
You are probably looking for a construct like so:
    block := blockstart, !, blockcontent?, blockend

or more concretely...

    ifblock := 'If', condition, !, blockcontent, 'End',ts,'If'

That is, once you hit a block-start, you must hit a block-end or there's 
a syntax error, so you include the cut and a syntax error will be raised 
when the system can't find the rest of the construct.  Similarly, a line 
in a block is something like:

    blockcontent := lines+
    line := !, line_content, line_end

that is, if you are in a place where you need a line and there are no 
other options to match, and you can't match a line, that's a syntax 
error, so the error will get reported as coming from the start of the 
line.  You need to be careful that you really do put the cuts (!) in 
after you have finally determined that a construct *must* match in all 
cases where it gets checked.  There's no way to stop propagation of the 
error once it's raised.

>I thought of skipping a line and then continuing but this is fraught
>with pitfalls since a logical line can span multiple actual lines. Is
>there a more robust way? Have I designed my grammar to be too nested?
>
Nesting is the "normal" approach for EBNF grammars.

>Thanks for your patience in reading this long note - is there a more
>appropriate place to ask questions like this? I don't want to clog your
>inbox!
>
Clogging my inbox isn't a significant issue, I get > 1000 emails a day 
what with various lists and the like :) .  SimpleParse users is the 
"proper" place to direct questions, so that other users can benefit from 
the discussion, I've copied it in this response.

Have fun,
Mike

_______________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/

[SimpleParse] FYI, un example of using simpleparse : amandalogp

[Olivier B. <oli...@in...>]

Hi.

Just FYI, I've used simpleparse to build a parser of the reports sent by
the amanda (www.amanda.org) backup system (conversion to HTML
highlighting errors).

It's probably a big hack, and I must say I didn't spend much time trying
to understand simpleparse in details, so it may not be the best example
to using simpleparse...

But in any case, if someone would like to take it as an example of a
parser built around simpleparse... here it is :=20

http://picolibre.int-evry.fr/projects/amandalogp/


Many thanks for such a nice and easy to use parser. That helped alot, to
make amandalogp.

Best regards,


--=20
Olivier BERGER <oli...@in...>
Ing=E9nieur Recherche - Dept INF
INT Evry (http://www.int-evry.fr)
OpenPGP-Id: 1024D/6B829EEC

Re: [SimpleParse] Look ahead in the tree

[Mike C. F. <mcf...@ro...>]

Martin d'Anjou wrote:

>Hi,
>
>It appears I've defined a syntax where some amount of looking ahead in the 
>tree is required. What support is there when I parse the tree with a 
>DispatchProcessor extension?
>
Not anything specific. Keep in mind, however, that dispatch processor is
just a simple method-dispatching mechanism.  You can readily tell your
higher-level dispatch method to pass multiple values to a method for
dealing with the complex data-set and then have that method use the
dispatching mechanism when it wants to recurse:

class X( DispatchProcessor ):
    def parent( self, ... ):
       result = []
       for childIndex in range(len(children)):
          result.append( self.subElements( children[childIndex:] ))
    def subElements( self, elements ):
       current = self.dispatch( elements[0] )
       do_stuff_that_needs_to_know_about_rest_of_set(current, elements)
       return current

or the like...

HTH,
Mike

_______________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/

[SimpleParse] Look ahead in the tree

[Martin d'A. <Mar...@s2...>]

Hi,

It appears I've defined a syntax where some amount of looking ahead in the 
tree is required. What support is there when I parse the tree with a 
DispatchProcessor extension?

Thanks,
Martin

Re: [SimpleParse] Re: Post-installation problem when using pydoc with simpleparse

[Martin d'A. <Mar...@s2...>]

> Afraid I don't have much more advice.  It does sound like a site 
> configuration problem, I just can't see what it would be from what 

Mike,

You were correct. I re-installed and it works.

Thanks,
Martin

[SimpleParse] Re: Post-installation problem when using pydoc with simpleparse

[Mike C. F. <mcf...@ro...>]

It appears to work fine here (w/ Python 2.2.3 on Win2K).  Can you import 
parser in a regular Python session?  Does help( parser ) produce this 
error?  site-packages is normally already in your path, and the fact 
that parser is getting found suggests that you've got simpleparse 
showing up as available already.  I'm assuming you're using the 
web-server interface, BTW.

Afraid I don't have much more advice.  It does sound like a site 
configuration problem, I just can't see what it would be from what 
you've described.

Good luck,
Mike

Martin d'Anjou wrote:

>Hi,
>
>I am getting the following post-installation problem:
>
>problem in /usr/lib/python2.2/site-packages/simpleparse/parser.pyc - ImportError: cannot import name baseparser
>
>I can type "from simpleparse import baseparser" without problems.
>
>I can view the pydoc for baseparser without any problems... Is this a 
>problem with my local installation? Does my PYTHONPATH have to point to 
>site-packages?
>
>Thanks,
>Martin
>  
>

[SimpleParse] Post-installation problem when using pydoc with simpleparse

[Martin d'A. <Mar...@s2...>]

Hi,

I am getting the following post-installation problem:

problem in /usr/lib/python2.2/site-packages/simpleparse/parser.pyc - ImportError: cannot import name baseparser

I can type "from simpleparse import baseparser" without problems.

I can view the pydoc for baseparser without any problems... Is this a 
problem with my local installation? Does my PYTHONPATH have to point to 
site-packages?

Thanks,
Martin

Re: [SimpleParse] How to "compile" the tree

[Mike C. F. <mcf...@ro...>]

Martin d'Anjou wrote:

>>or just call the processor object with the tag-tree result-set and the 
>>data (the string source):
>>    
>>
>
>Fantastic. I got my processor it to traverse the entire tree. Now, I would
>like to output the result in a C++-like language. What should be the next
>step?
>
Normally the next step is to model your domain as objects and have 
SimpleParse instantiate those objects so that you can process them in 
whatever way you want. 

For instance, OpenGLContext has a VRML 97 Node object model.  It's 
ParseProcessor creates nodes from the parse-tree and then passes the 
tree off to OpenGLContext for interpretation. OpenGLContext doesn't know 
about the parse-processor's internals other than how to register objects 
to be instantiated, and the parse-processor only knows about the objects 
it's supposed to instantiate.  In this case, OpenGLContext processes the 
tree by rendering it to a 3D window, but performing transformations and 
creating some form of output are about the same process in the end.

SimpleParse really is a parser, not a compiler, it doesn't attempt to 
provide services beyond "loading data into an in-memory graph".  So you 
won't find code in it for tree transformations or the like, that's all 
considered out-of-scope for the project.
...

>I will have a bunch of tasks and functions doing things to stuff to car1
>like setting the color to red and the address to 723, hooks for repainting
>etc. There will be lots of tasks, and each task will need to do things to
>almost every class member.
>
>Should I put everything in myProcessor, or is there a better suited
>approach that SimpleParse provides?
>  
>
Putting everything into ParseProcessor probably isn't a good idea. It's 
designed to walk a result tree and generate some domain-specific format, 
not to provide complex random-access operations on a tree/graph of 
objects/nodes.

Have fun,

Mike

[SimpleParse] How to "compile" the tree

[Martin d'A. <Mar...@s2...>]

> or just call the processor object with the tag-tree result-set and the 
> data (the string source):

Fantastic. I got my processor it to traverse the entire tree. Now, I would
like to output the result in a C++-like language. What should be the next
step?

For example, I have parsed the following:

car1 at address 723
doors 4
color red

And the final output should look like this:

class car1 {

   integer address;
   integer color;
   integer ndoors[4];

}

task car1::repaint() { ... }

I will have a bunch of tasks and functions doing things to stuff to car1
like setting the color to red and the address to 723, hooks for repainting
etc. There will be lots of tasks, and each task will need to do things to
almost every class member.

Should I put everything in myProcessor, or is there a better suited
approach that SimpleParse provides?

Thanks,
Martin

Re: [SimpleParse] How to process the tree?

[Martin d'A. <Mar...@s2...>]

> >instantiated MyProcessor class once. Now I don't now what to do. What
> >function should I call to get the ball rolling?
> >
> Pass the processor in to the parse method:
> 
Thanks Mike! I am slowly getting it... the problem is I was confusing 
myself with what David Mertz's proposed in his article with what was in 
the documentation you posted to the simpleparse web page. Now it clear 
that I should seperate the two in my mind, at least for now!

Thanks again,
Martin

Re: [SimpleParse] How to process the tree?

[Mike C. F. <mcf...@ro...>]

Martin d'Anjou wrote:

>Hi,
>
>  
>
...

>I tried to subclass the DispatchProcessor, and added a function for each
>production I had (with a simple print statement in each one). I
>instantiated MyProcessor class once. Now I don't now what to do. What
>function should I call to get the ball rolling?
>
Pass the processor in to the parse method:

    def parse( self, data, production=None, processor=None, start=0, 
stop=None):
        """Parse data with production "production" of this parser

        data -- data to be parsed, a Python string, for now
        production -- optional string specifying a non-default 
production to use
            for parsing data
        processor -- optional pointer to a Processor or MethodSource 
object for
            use in determining reporting format and/or post-processing 
the results
            of the parsing pass.  Can be None if neither is desired 
(default)
        start -- starting index for the parsing, default 0
        stop -- stoping index for the parsing, default len(data)
        """

or just call the processor object with the tag-tree result-set and the 
data (the string source):

    processor( result, data )

HTH,
Mike
_______________________________________

  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/

[SimpleParse] How to process the tree?

[Martin d'A. <Mar...@s2...>]

Hi,

I have got the following program which works very well, but does not do 
anything useful after it produces a correct taglist:

import os
from sys import stdin, stdout, stderr
from simpleparse import generator
from simpleparse.common import comments, numbers
from mx.TextTools import TextTools
import pprint

input = stdin.read()
decl = open('pif_works.def').read()
parser = generator.buildParser(decl).parserbyname('source_text')
taglist = TextTools.tag(input, parser)
pprint.pprint(taglist)

Now I want to create 4 different output formats (HTML and LaTeX are two of 
them).

I tried to subclass the DispatchProcessor, and added a function for each
production I had (with a simple print statement in each one). I
instantiated MyProcessor class once. Now I don't now what to do. What
function should I call to get the ball rolling?

I have read the examples in simpleparse/common as recommended, but I just
don't see how tree processing gets started.

Thanks,
Martin

Re: [SimpleParse] How to do use "!" (Error on fail - cut)

[Mike C. F. <mcf...@ro...>]

Martin d'Anjou wrote:

>Mike,
>
>Both your answers helped me. I re-wrote my BNF a few times, learning more 
>at each step, and making less and less mistakes. My next step is to learn 
>to process the result tree.
>
>Is SimpleParse going to be around for a while? I mean, the mailing list 
>suggest there are only very few users. I wonder if I should switch to 
>SPARK...
>
SimpleParse is going to be around for a long while.  I only work on it 
when I get time, and particularly, when there are user requests for 
features/bugs or I find something I myself need, but I consider it a 
core part of my toolbox, so I'm not likely to let it die off.

Re size: I'd guess there's around 20 direct users at the moment, but 
there's considerably more secondary users  (e.g. every OpenGLContext 
user tends to be one, as does every mcf.vrml user, and there are other 
projects using it as well).  Basically, it's a developer's tool with a 
very specialised functionality not an end-user's tool, so it's user 
count is never going to be too high (do note that some users never post 
to the list as well).  That said, it's just been covered in Text 
Processing in Python, so maybe we'll get a huge influx now :) .

Have fun,
Mike

_______________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/

Re: [SimpleParse] How to do use "!" (Error on fail - cut)

[Martin d'A. <Mar...@s2...>]

Mike,

Both your answers helped me. I re-wrote my BNF a few times, learning more 
at each step, and making less and less mistakes. My next step is to learn 
to process the result tree.

Is SimpleParse going to be around for a while? I mean, the mailing list 
suggest there are only very few users. I wonder if I should switch to 
SPARK...

> >help!
> >
> section       := builddecl,!, elementdecl,elementdecl*

This has changed quite a bit in my re-writing, but at least I understand 
why it was wrong.

> You're probably also wanting this:
> 
> file          := ts, section, (ts, section)*, ts

Actually, my infinite loop was because I had (ts/section)* rather than 
(ts, section)*

Thanks a lot!
Martin

Re: [SimpleParse] how to debug SimpleParse

[Mike C. F. <mcf...@ro...>]

Hi Martin,

Not really, as the parser is just producing (very complex) mxTextTools 
tagging tables.  Normally what I do is write test frameworks that parse 
with the lower-level productions, working up to the highest-level 
productions.  Sprinkling cuts (!) throughout (where there is an actual 
"this must be here if you got this far" constraint) can also make it 
easier.  The parser will then complain when it can't get past that point.

Will respond to previous message in a sec...
Mike

Martin d'Anjou wrote:

>Hi,
>
>Is there an option for printing what the parser does, I seem to be quite 
>good at creating infinite loops without knowing, so I can at least figure 
>out what my bad productions are?
>
>Thanks,
>Martin
>  
>
 
_______________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/

[SimpleParse] how to debug SimpleParse

[Martin d'A. <Mar...@s2...>]

Hi,

Is there an option for printing what the parser does, I seem to be quite 
good at creating infinite loops without knowing, so I can at least figure 
out what my bad productions are?

Thanks,
Martin

[SimpleParse] How to do use "!" (Error on fail - cut)

[Martin d'A. <Mar...@s2...>]

Hi,

There is not much activity here, I hope there is someone watching! I am
really struggling with the "!" Error on fail (cut), I hope someone can
help me.

I keep getting an 'Expected syntax: builddecl' error when I run the 
following. I know I am getting it because I introduced a "!" after 
'builddecl', but I don't understand why it would be an error. Maybe my 
EBNF is all wrong. I have supplied the shortest example I could create 
that reproduces the problem below. Thanks in advance to anyone who can 
help!

I have the following EBNF definition in a file called biz.def:

file          := ts, section, ts, section*
<ts>          := ( [ \011-\015]+ / hash_comment )*
<ws>          := [ \t]
section       := builddecl!,elementdecl!,elementdecl*
builddecl     := ws*,'build',ws+,name,ws+,'on',ws+,planet,ts
elementdecl   := ws*,'element',ws*,'=',ws*,item,ws+,'is',ws+,quality,ts
name          := identifier
planet        := identifier
item          := identifier
quality       := identifier
identifier    := [a-zA-Z_],[a-zA-Z0-9_]*

I have the following text I try to parse with it:

# this is the file I try to parse
build cars on pluto
element = door is red
element = seats is leather

And here is the program to parse it:

import os
from sys import stdin, stdout, stderr
from simpleparse import generator
import simpleparse.common.comments
import simpleparse.common.numbers
from mx.TextTools import TextTools
import pprint

input = stdin.read()
decl = open('biz.def').read()
parser = generator.buildParser(decl).parserbyname('file')
taglist = TextTools.tag(input, parser)
pprint.pprint(taglist)

stderr.write('parsed %s chars of %s\n' %  (taglist[-1], len(input)))
if taglist[-1] != len(input):
   print "ERROR near line",input[:taglist[-1]].count('\n')



Thanks,
Martin d'Anjou

[SimpleParse] SimpleParse 2.0.1a1 released

[Mike C. F. <mcf...@ro...>]

A new SimpleParse release, which adds optimisations to reduce terminal 
productions (and thereby speed up certain types of very formally 
specified grammars) is now available.  For affected grammars, it's quite 
possible to see speedups of 2x compared with v 2.0.0 of the package.  It 
is strongly recommended that you use mxTextTools 2.1.0 with the 2.0.1 
release of SimpleParse.

Download and documentation:

    http://simpleparse.sourceforge.net/

What is SimpleParse:

    SimpleParse is a BSD-licensed
    <http://simpleparse.sourceforge.net/#License> Python package
    providing a simple parser generator for use with the mxTextTools
    <http://www.lemburg.com/files/python/mxTextTools.html> text-tagging
    engine. SimpleParse allows you to generate tagging tables for use
    with the text-tagging engine directly from your EBNF grammar.

Enjoy,
Mike

_______________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/

[SimpleParse] SimpleParse tag-table optimizations checked in...

[Mike C. F. <mcf...@ro...>]

Just a FYI: I have just checked in changes to the simpleparse engine 
which produce the same parsing tables with the following two grammars:

    a    := (table1 / table2 / any_line)*
    <any_line> := ANY*, EOL
    <ANY>      := -EOL
    <EOL>      := '\n'
    table1 := 'a'
    table2 := 'b'

vs.

    a    := (table1 / table2 / (-'\n'*, '\n'))*
    table1 := 'a'
    table2 := 'b'

Similarly:
    a:=b <b>:= -c* c:='this'
becomes:
    a := -'this'*

and

    a:=b >b<:= c c:= 'this'
becomes:
    a := c c:= 'this'

There are a number of other minor optimizations to the generated 
mxTextTools tables (primarily getting rid of spurious table/sub-table 
elements in some common areas).  If you are particularly interested, 
feel free to check out the code from the CVS repository.  I haven't done 
any empirical testing yet to see if the overall impact of the 
optimization calculations make this approach worthwhile, but I would 
imagine that, if the analysis stage isn't killing us entirely the parser 
should be somewhat faster, particularly on grammars which use lots of 
un-reporting or expanded productions.

Enjoy yourselves,
Mike

_______________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/

Re: [SimpleParse] Having SimpleParser ignore whitespaces

[Mike C. F. <mcf...@ro...>]

At the moment, there's no such mechanism.  This is largely a historic 
artefact of the parsing mechanism that SimpleParse uses.  "Normal" EBNF 
tools are 2-stage processors.  You define a set of tokens, a set of 
punctuation, and a set of whitespace, then tokenise into discreet tokens 
using those low-level definitions (lexing).  You then just deal with the 
resulting tokens.  The definitions are a requirement for the engine and 
if they happen to make certain things more convenient for the user, 
well, that's okay, as long as they aren't too happy about it :) ;) .

Since SimpleParse never had the need to require those 3 definitions (it 
doesn't lex), it's never grown a way to specify or use them.

If you wanted to add this functionality, you'd have lots of ways to do 
it (here's three off the top of my head):
	modify objectgenerator.Range and Literal to always add a generic 
"consume whitespace after parse" tag to the tag-table they produce.

	modify objectgenerator.SequentialGroup to insert a whitespace-consumer 
between each pair.

	add a new group-type to the SimpleParse EBNF format (e.g. using just a 
space between the element tokens) which defines a white-space-seperated 
group.  You will then need to make sure that  a := b c d := e is 
un-ambiguous (basically a name is now a name iff it is not followed by a 
:= token, so use a negative look-ahead check).

In all of those cases, you need to declare the composition of 
"whitespace" somewhere and make it available to your objectgenerator 
classes (likely in the generator object).  In all save the last, you'd 
need a way to differentiate when you do/do-not want the whitespace 
consumption.

BTW: Manually altering the tag-tables SimpleParse produces is probably 
one of the most painful ways to have your brain explode :o) .  I don't 
even look at them 99% of the time I'm working with the system.  The only 
real reason to do it is to debug an error in SimpleParse or to try to 
optimise the tables it generates.

Feel free to shout if this was unclear,
Mike

Karl Trygve Kalleberg wrote:
> Hi fellow parsists.
> 
> I notice that all of the example grammars include whitespaces in the
> productions explicitly. Is there any simple way to tell SimpleParse that
> the charset "[ \t\n\r]+" is considered a generic token separator, as is
> customary with other EBNF tools ? 
> 
> funcall := id, '(', arglist, ')', ';'
> 
> is most definitely easier to read and reason about than
> 
> funcall := id, ws, '(', ws, arglist, ws, ')', ws, ';'
> 
> I tried modifying the resultant tuple returned by generator.buildParser
> thusly;
> 
> parser = generator.buildParser(decl).parserbyname('root')
> parser = ((None,TextTools.AllInSet,TextTools.set(' \r\n\t'),+1),) + parser
> pprint.pprint( TextTools.tag( input, parser ))
> 
> but that does not seem to have any effect.
> 
> Any suggestions/pointers to solutions are most welcome.
> 
> 
> Kind regards,
> 
> Karl T
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by: OSDN - Tired of that same old
> cell phone?  Get a new here for FREE!
> https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390
> _______________________________________________
> SimpleParse-users mailing list
> Sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simpleparse-users
> 

-- 
_______________________________________
   Mike C. Fletcher
   Designer, VR Plumber, Coder
   http://members.rogers.com/mcfletch/