Re: [Pyparsing] Parsing large files using SkipTo is slow.

[Paul M. <pt...@au...>]

First off, you have some mixed spaces/tabs in your file, you should clean
that up.

By using '^' operators inside a Forward recursive expression, your parser is
doing a lot of extra work.

I got some better response by changing the commented lines below:

#~ value = Word(alphas) + ( ( sls + SkipTo(']',include=False)+srs ) ^
Word(alphas +"_") )
value = Word(alphas) + ( ( sls + SkipTo(']',include=False)+srs ) |
Word(alphas +"_") )
value.setParseAction(p_val)

named_block = Forward()
#~ block_content = OneOrMore( value ^ named_block )
block_content = OneOrMore( named_block | value )
named_block << OneOrMore(Word(alphas)) + slc + block_content + src
named_block.setParseAction( p_nb )
named_block.ignore(pythonStyleComment)
#~ named_block.enablePackrat()


There were 3.2 million calls to "hash", most likely due to the packrat
parsing. By changing Or's to MatchFirst's, the packratting was not
necessary, and was in fact costing extra performance.

-- Paul


-----Original Message-----
From: Max...@dl... [mailto:Max...@dl...] 
Sent: Monday, September 12, 2011 7:10 AM
To: pyp...@li...
Subject: [Pyparsing] Parsing large files using SkipTo is slow.

Hello,

 I am trying to parse a simple CAD file using pyparsing, where the
 grammar is quite simple and most of the lines of the file are supposed
 to be skipped. However this is still slower than I would expect it to
 be.

 The parser is in the parse_iv.py file,  the CAD file is the
 screwdriver_2.iv and the CProfile result is th err file.

 It seems like parseString is called for each line. I don't understand
 the internal workings of pyparsing but would it be possible to use
 something like file.index( skiptostr ) to speed things up?

ftp://ftp.robotic.dlr.de/outgoing/Argus/


 BR, Max


----------------------------------------------------------------------------
--
Doing More with Less: The Next Generation Virtual Desktop 
What are the key obstacles that have prevented many mid-market businesses
from deploying virtual desktops?   How do next-generation virtual desktops
provide companies an easier-to-deploy, easier-to-manage and more affordable
virtual desktop model.http://www.accelacomm.com/jaw/sfnl/114/51426474/
_______________________________________________
Pyparsing-users mailing list
Pyp...@li...
https://lists.sourceforge.net/lists/listinfo/pyparsing-users