Re: [Cscope-devel] Should we drop support for lex?

[Petr S. <pe...@sc...>]

Hi,


> [ Please use some mailer that wraps lines at ~72 characters.  Your
>   message is coming across as unreadably long lines.  ]
heh. Sorry was using "vi" and forgot to press enter.

> Petr Sorfa <pe...@us...> wrote:
> 
> > After looking at the problems with YYLMAX several users reported
> > (YYLMAX is reached and falls over). I realized that the problem is
> > with the -l option passed to flex. With the -l option, flex forces
> > yytext to be the size of YYLMAX, which due to a flex mishap is always
> > set to the size of 8192 bytes (this is against the AT&T standard, that
> > allows YYLMAX to be changed).
> 
>      What version of flex are you using?  Mine (2.5.4) has the "#define
> YYLMAX" within "#ifndef YYLMAX", which allows for easy redefinition of
> YYLMAX.
> 
> [ Even if YYLMAX wasn't within #ifndef, a sed script could be used to
>   change the value of YYLMAX.  ]
Correct, but if you actually look at the generated code, you realize
that the #ifndef YYLMAX is inserted before the user defined stuff, so
our #define of YYLMAX get ignored all the time. Try changing the value
for YYLMAX in constants.h and do an nm on scanner.o and you'll see that
it will always be set to 0x2000 (flex default size).

> 
> > Arguments against:
> > 1) Will lose AT&T support which seems to give us much better performance for
> > those platforms that do have AT&T lex
> 
>      That's not correct.  This is:
> 
> 1) Will lose AT&T support which ALWAYS gives us much better performance
>    for ALL platforms.  (OK, for those platforms that don't have lex, we
>    would have to provide pre-lex'ed source files.)
> 
> Unfortunately, this is a big issue, as flex is an incredible pig when it
> comes to cscope.  See my old message -- we're talking 5-6 times slower.
> It wouldn't be bad if flex was within, oh, 30% of lex, but it's not.
> 
>      Right now, using lex, cscope can rebuild the database on a
> large-ish project in under 30 seconds.  With flex, the rebuilding slows
> down to an unusable 2.5-3 minutes.  It's a big difference.
My guess may be your 6) solution for this.

> 
> > My two cents:
> > 1) is extremely attractive, since the -l option to flex is mentioned to reduc
> > e performance. Also flex is available for all platforms, whereas lex is not.
> > Our largest user base is Linux which doesn't use lex.
> 
>      This is good, as it's pretty easy.  [ And it should be done, if
> nothing else is.  I tested this during my flex/lex benchmarking, and the
> flex performance went from ~6X slower than lex, to ~5X.  ]
> 
>      There are also more choices:
> 
> 4. Convince SCO to open-source lex.  ;-)
Well, we will have to wait till we become Caldera, before that happens.
However, flex will remain the no.1 choice as lex has not moved forward
for several years.

> 5. Use sed script to edit the value of YYLMAX.
Err, or setup configure.in to pass a definition of YYLMAX when building
scanner.c. Not a autoconf expert so my attempt was dismal. Anybody want
to give a try? This will be the easiest solution, and will allow users
to fiddle with the value.

> 6. Completely rewrite scanner.l for flex.  Basically, all uses of REJECT
>    (one) and arbitrary trailing context (lots) -- and possibly more --
>    need to be eliminated.  For more information, see the flex man page,
>    under "PERFORMANCE CONSIDERATIONS".
> 
>    [ No, I'm not volunteering.  ]
Comes to a point of considering: "if it's not broken, don't fix it".

Thanks for your comments Darryl, they were really great. Lets see if we
get any other input from the other folks, before deciding.

Petr
> 
> --
>         Darryl Okahata
>         da...@so...
> 
> DISCLAIMER: this message is the author's personal opinion and does not
> constitute the support, opinion, or policy of Agilent Technologies, or
> of the little green men that have been following him all day.

-- 
--------------------------------------------------------
Petr Sorfa                  Software Engineer
Santa Cruz Operation (SCO) 
Murray Hill 07974
NJ, USA
--------------------------------------------------------
Disclaimer: All my comments are my own and nobody else's
----------------------------------------------------------