Re: [Flex-help] flex-2.5.37 testsuite clean up on Solaris

[Hugh S. <hg...@dm...>]

On Thu, 25 Oct 2012, Dennis Clarke wrote:

> [ warning - long ranty email ] 
        [...]
> I think bison 2.6.4 is fine.  Really. This issue seems to be in two tests only now.  Let me detail that below.
> 
> > For building flex.pdf, you just need the texinfo package. My memory is
> > vague on how far down the TeX stack that requires you to go. I've
> > generally used tetex as my distribution of choice when I've done it by
> > scratch, but I'm not sure if that is still the state of the art.
> 
> I went and built the whole damn thing.  
> Didn't really need it but it can't hurt to have it around right? 
> Well surprise, the full install of TeXlive takes 2.86GB on the filesystem and most people don't set aside that sort of space for stuff in /usr/local 
> 
> yikes 

That is big for Suns.  It seems a bit much to ask for lexical analyser.
Anyway, I'm beginning to understand AC_PATH_PROGS_FEATURE_CHECK so I may
be able to contribute to configure.in later to test for a working tex2pdf
which would allow us to turn off PDF generation.  
> 
> Please know that I am just "sharing" here and you don't have to do anything but hopefully you get a picture of how far out of whack the Solaris world is from the rest of the world that runs Linux. The gap is getting wider and wider it seems and I don't even know of a single software vendor ( orther than Oracle ) the releases a damn thing for Solaris.  Do you?  I sure don't. Seriously, it is a dead platform in a very very small niche market but a lot of big banks and such are stuck on it for years as they can not afford to migrate their ssoftware systems. Yet. 

The big packages we use don't seem to be supported on there now.
> 
> Personally I live in front of Red Hat Enterprise Linux and Debian these days with ssh access to old Sun/Solaris clunkers for service reasons. 
> 
> [ end of rant ] 
> 
> want to see something completely weird/bizarre/freaky ? 
> 
> I extract and then run ./configure within the texinfo-4.13a source directory. No problems. Run gmake ( GNU make is not Solaris XPG4 make ) and then run gmake check. I see this : 
> 
        [lots of hard won output trimmed]
> 
> So perfect right ? 
> 
> run gmake install and then : 
> 
> What I get was these : 
> 
        [...]
> 
> We have a nice shiney new perfect looking flex binary here now : 
> 
> $ file flex
> flex: ELF 64-bit MSB executable SPARCV9 Version 1, UltraSPARC3 Extensions Required, dynamically linked, not stripped
> $ elfdump -d flex 
> 
> Dynamic Section:  .dynamic
>      index  tag                value
>        [0]  NEEDED            0x1cd4              libintl.so.8
>        [1]  NEEDED            0x1ce1              libiconv.so.2
>        [2]  NEEDED            0x1ca5              libc.so.1
>        [3]  NEEDED            0x1cca              libm.so.2
        [...]
> 
> However .. let's fire off the testsuite : 
> 
> $ date -u ; gmake check 
> Thu Oct 25 17:44:49 GMT 2012
> gmake  check-recursive
> gmake[1]: Entering directory `/usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2'
        [....]
> Making check in doc
> gmake[2]: Entering directory `/usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/doc'
> restore=: && backupdir=".am$$" && \
> am__cwd=`pwd` && CDPATH="${ZSH_VERSION+.}:" && cd . && \
> rm -rf $backupdir && mkdir $backupdir && \
> if (/bin/bash /usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/missing --run makeinfo --version) >/dev/null 2>&1; then \
>   for f in flex.info flex.info-[0-9] flex.info-[0-9][0-9] flex.i[0-9] flex.i[0-9][0-9]; do \
>     if test -f $f; then mv $f $backupdir; restore=mv; else :; fi; \
>   done; \
> else :; fi && \
> cd "$am__cwd"; \
> if /bin/bash /usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/missing --run makeinfo   -I . \
>  -o flex.info flex.texi; \
> then \
>   rc=0; \
>   CDPATH="${ZSH_VERSION+.}:" && cd .; \
> else \
>   rc=$?; \
>   CDPATH="${ZSH_VERSION+.}:" && cd . && \
>   $restore $backupdir/* `echo "./flex.info" | sed 's|[^/]*$||'`; \
> fi; \
> rm -rf $backupdir; exit $rc
> TEXINPUTS=".:$TEXINPUTS" \
> MAKEINFO='/bin/bash /usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/missing --run makeinfo   -I .' \
> texi2dvi --pdf --batch  --clean flex.texi 
> /usr/local/bin/texi2dvi: local: not found
> /usr/local/bin/texi2dvi: BSTINPUTS: not found
> gmake[2]: *** [flex.pdf] Error 1
> gmake[2]: Leaving directory `/usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/doc'
> gmake[1]: *** [check-recursive] Error 1
> gmake[1]: Leaving directory `/usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2'
> gmake: *** [check] Error 2
> $ 
> 
> life just gets worse with this  :-(
> 
> $ which texi2dvi
> /usr/local/bin/texi2dvi
> $ file /usr/local/bin/texi2dvi
> /usr/local/bin/texi2dvi: executable shell script
> 
> Turns out that the script has a shell definition of  #! /bin/sh  at the top but really it wants bash and not really a real bourne shell from Solaris UNIX land. 

Always got bitten by that.  #! lines are way better than file extensions
for determining type, but quite often it was the path that bit me.

> 
> of course .. so I edit that and change the first line to  #!/bin/bash 
> 
> then run gamke check again and it runs along neatly ...
> 
> $ date -u ; gmake check 
> Thu Oct 25 17:47:07 GMT 2012
> gmake  check-recursive
        [...]
> Making check in doc
> gmake[2]: Entering directory `/usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/doc'
> TEXINPUTS=".:$TEXINPUTS" \
> MAKEINFO='/bin/bash /usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/missing --run makeinfo   -I .' \
> texi2dvi --pdf --batch  --clean flex.texi 
> This is pdfTeX, Version 3.1415926-2.4-1.40.13 (TeX Live 2012)
>  restricted \write18 enabled.
> entering extended mode
> 
> (./flex.texi
> (/usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/doc/texinfo.tex
> Loading texinfo [version 2009-08-14.15]: pdf, fonts, markup, glyphs,
        [...]
> .
> .
> .  piles of TeX goodness going on 
> .
        :-)
        [...]
> Seven tests fail ? 
> 
> turns out my CXXFLAGS are off.  argggg  ...
> 
> fix up CXXFLAGS and then : 
> 
        [...]
> $ 
> 
> better .. lets see what the problem is : 
> 
> $ cat tests/test-extended/OUTPUT
> gmake[5]: Entering directory `/usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/tests/test-extended'
> ../../flex  scanner.l
> /opt/solarisstudio12.3/bin/cc -c -o scanner.o -I. -I. -I../.. -I../.. -I/usr/local/include -errfmt=error -erroff=%none -errshort=full -xstrconst -xildoff -m64 -xmemalign=8s -xnolibmil -Xa -xcode=pic32 -xregs=no%appl -xlibmieee -mc -g -xs -ftrap=%none -Qy -xbuiltin=%none -xdebugformat=dwarf -xunroll=1 -xtarget=ultraT2 -xcache=8/16/4:4096/64/16 -D_TS_ERRNO -D_POSIX_PTHREAD_SEMANTICS -D_LARGEFILE64_SOURCE scanner.c
> /opt/solarisstudio12.3/bin/cc -errfmt=error -erroff=%none -errshort=full -xstrconst -xildoff -m64 -xmemalign=8s -xnolibmil -Xa -xcode=pic32 -xregs=no%appl -xlibmieee -mc -g -xs -ftrap=%none -Qy -xbuiltin=%none -xdebugformat=dwarf -xunroll=1 -xtarget=ultraT2 -xcache=8/16/4:4096/64/16 -D_TS_ERRNO -D_POSIX_PTHREAD_SEMANTICS -D_LARGEFILE64_SOURCE -o test-extended  scanner.o 
> ./test-extended < ./test.input | diff -q ./test.input -

diff -q means:
  -q, --brief                   report only when files differ
So this expects the input to be the same as the output?
Then why not use cmp for this?  Or at least use it first to see
if diff is needed?  I was thinking of rolling a configure test
to see if a diff -q was available, and I found this to borrow from:

http://gitorious.org/ratfor-revived/c-ratfor/commit/234b01e/diffs

but it is less portable.

        [local diff info and point about configure trimmed]
> 
> Here we go again as I reach for the bottle of scotch : 

#include <stdairplanequotes.h>

        [...]
> 
> Well that deserves that we raise a glass eh?  ( sorry, I'm Canadian ) 
> 
> So why did those two tests fail ? 
> 
> Let's look : 
> 
> $ 
> $ cat tests/test-bison-yylloc/OUTPUT
> gmake[5]: Entering directory `/usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/tests/test-bison-yylloc'
> ../../flex scanner.l
> /usr/local/bin/bison --defines --output=parser.c --name-prefix="test" parser.y
> /opt/solarisstudio12.3/bin/cc -c -o scanner.o -I. -I../.. -I../.. -I. -I/usr/local/include -errfmt=error -erroff=%none -errshort=full -xstrconst -xildoff -m64 -xmemalign=8s -xnolibmil -Xa -xcode=pic32 -xregs=no%appl -xlibmieee -mc -g -xs -ftrap=%none -Qy -xbuiltin=%none -xdebugformat=dwarf -xunroll=1 -xtarget=ultraT2 -xcache=8/16/4:4096/64/16 -D_TS_ERRNO -D_POSIX_PTHREAD_SEMANTICS -D_LARGEFILE64_SOURCE scanner.c
> /opt/solarisstudio12.3/bin/cc -c -o parser.o -I. -I../.. -I../.. -I. -I/usr/local/include -errfmt=error -erroff=%none -errshort=full -xstrconst -xildoff -m64 -xmemalign=8s -xnolibmil -Xa -xcode=pic32 -xregs=no%appl -xlibmieee -mc -g -xs -ftrap=%none -Qy -xbuiltin=%none -xdebugformat=dwarf -xunroll=1 -xtarget=ultraT2 -xcache=8/16/4:4096/64/16 -D_TS_ERRNO -D_POSIX_PTHREAD_SEMANTICS -D_LARGEFILE64_SOURCE parser.c
> "parser.c", line 1389: warning: implicit function declaration: testlex
> "parser.c", line 1566: warning: argument #1 is incompatible with prototype:
>         prototype: pointer to char : "parser.y", line 38
>         argument : pointer to const char
> /opt/solarisstudio12.3/bin/cc -c -o main.o -I. -I../.. -I../.. -I. -I/usr/local/include -errfmt=error -erroff=%none -errshort=full -xstrconst -xildoff -m64 -xmemalign=8s -xnolibmil -Xa -xcode=pic32 -xregs=no%appl -xlibmieee -mc -g -xs -ftrap=%none -Qy -xbuiltin=%none -xdebugformat=dwarf -xunroll=1 -xtarget=ultraT2 -xcache=8/16/4:4096/64/16 -D_TS_ERRNO -D_POSIX_PTHREAD_SEMANTICS -D_LARGEFILE64_SOURCE main.c
> "main.c", line 27: error: identifier redeclared: testparse
>         current : function(pointer to void) returning int
>         previous: function(void) returning int : "parser.h", line 98
> cc: acomp failed for main.c
> gmake[5]: *** [main.o] Error 2
> gmake[5]: Leaving directory `/usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/tests/test-bison-yylloc'
> $ 
> $ cat tests/test-bison-yylval/OUTPUT
> gmake[5]: Entering directory `/usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/tests/test-bison-yylval'
> /usr/local/bin/bison --defines --output=parser.c --name-prefix="test" parser.y
> /opt/solarisstudio12.3/bin/cc -c -o parser.o -I. -I../.. -I../.. -I. -I/usr/local/include -errfmt=error -erroff=%none -errshort=full -xstrconst -xildoff -m64 -xmemalign=8s -xnolibmil -Xa -xcode=pic32 -xregs=no%appl -xlibmieee -mc -g -xs -ftrap=%none -Qy -xbuiltin=%none -xdebugformat=dwarf -xunroll=1 -xtarget=ultraT2 -xcache=8/16/4:4096/64/16 -D_TS_ERRNO -D_POSIX_PTHREAD_SEMANTICS -D_LARGEFILE64_SOURCE parser.c
> "parser.c", line 1333: warning: implicit function declaration: testlex
> "parser.c", line 1514: warning: argument #1 is incompatible with prototype:
>         prototype: pointer to char : "parser.y", line 38
>         argument : pointer to const char
> ../../flex scanner.l
> /opt/solarisstudio12.3/bin/cc -c -o scanner.o -I. -I../.. -I../.. -I. -I/usr/local/include -errfmt=error -erroff=%none -errshort=full -xstrconst -xildoff -m64 -xmemalign=8s -xnolibmil -Xa -xcode=pic32 -xregs=no%appl -xlibmieee -mc -g -xs -ftrap=%none -Qy -xbuiltin=%none -xdebugformat=dwarf -xunroll=1 -xtarget=ultraT2 -xcache=8/16/4:4096/64/16 -D_TS_ERRNO -D_POSIX_PTHREAD_SEMANTICS -D_LARGEFILE64_SOURCE scanner.c
> /opt/solarisstudio12.3/bin/cc -c -o main.o -I. -I../.. -I../.. -I. -I/usr/local/include -errfmt=error -erroff=%none -errshort=full -xstrconst -xildoff -m64 -xmemalign=8s -xnolibmil -Xa -xcode=pic32 -xregs=no%appl -xlibmieee -mc -g -xs -ftrap=%none -Qy -xbuiltin=%none -xdebugformat=dwarf -xunroll=1 -xtarget=ultraT2 -xcache=8/16/4:4096/64/16 -D_TS_ERRNO -D_POSIX_PTHREAD_SEMANTICS -D_LARGEFILE64_SOURCE main.c
> "main.c", line 27: error: identifier redeclared: testparse
>         current : function(pointer to void) returning int
>         previous: function(void) returning int : "parser.h", line 85
> cc: acomp failed for main.c
> gmake[5]: *** [main.o] Error 2
> gmake[5]: Leaving directory `/usr/local/build/flex-2.5.37_SunOS5.10_sparcv9-try2/tests/test-bison-yylval'
> $ 
> 
> 
> Looks like a small code issue to me. 
> 
> "main.c", line 27: error: identifier redeclared: testparse
>         current : function(pointer to void) returning int
>         previous: function(void) returning int : "parser.h", line 98
> 
> $ diff main.c_backup main.c
> 27,28d26
> < extern int testparse(yyscan_t);
> < 
> 
> Same problem with test-bison-yylval.
> 
> However my real issue is with parser.h which has this in it at line 89 onwards : 
> 
> #ifdef YYPARSE_PARAM
> #if defined __STDC__ || defined __cplusplus
> int testparse (void *YYPARSE_PARAM);
> #else
> int testparse ();
> #endif
> #else /* ! YYPARSE_PARAM */
> #if defined __STDC__ || defined __cplusplus
> int testparse (void);
> #else
> int testparse ();
> #endif
> #endif /* ! YYPARSE_PARAM */
> 
> well this causes the compiler to barf because : 
> 
> "main.c", line 33: error: prototype mismatch: 1 arg passed, 0 expected
> cc: acomp failed for main.c
> 
> 
> Well scanner.h has this : 
> 
> /* An opaque pointer. */
> #ifndef YY_TYPEDEF_YY_SCANNER_T
> #define YY_TYPEDEF_YY_SCANNER_T
> typedef void* yyscan_t;
> #endif
> 
> And in main.c we have a call to testparse ( scanner ) with a arg of type yyscan_t however the code in parser.h has these : 
> 
> #ifdef YYPARSE_PARAM
> #if defined __STDC__ || defined __cplusplus
> int testparse (void *YYPARSE_PARAM);
> #else
> int testparse ();
> #endif
> #else /* ! YYPARSE_PARAM */
> #if defined __STDC__ || defined __cplusplus
> int testparse (void);
> #else
> int testparse ();
> #endif
> #endif /* ! YYPARSE_PARAM */
> 
> so for whatever reasons we have definitions for testparse with a valid void *YYPARSE_PARAM and then we also have a wrong int testparse (); and another wrong int testparse (void);. 
> 

Thank you.  That sort of thing is pretty tricky to spot.

> I'll have to dig into these and fix them. 
> 
> So what have I figured out here ? 
> 
> I sent you a pile of Makefile.am files all hand edited with love toensure we get proper CFLAGS and CXXFLAGS as required in the compile stages of the tests. I see that configure doesn't look for GNU diff but needs it. I see we have a small code issue in two tests and that yes, bison is needed but NEW bison is not an issue here. 
> 
> thoughts ? 

A vast amount of work, for which, thanks.  If we fire up the Solaris box
lurking somewhere we'll need this.

I can't see a simple way to test for Gnu diff when that is needed,
but I don't think that it is in this case; I think the test is a
PASS/FAIL, and the diff collects the output, for which a cmp
pre-test would do, albeit more slowly because of having to then 
collect the difference with diff in the failing cases.

Is it sensible to do an AC_PROG_CHECK for gdiff to check for a GNU diff,
or is that too blunt an instrument?  Is the proper thing to do a FEATURE
test with --version and a test for Gnu in the string?   I found one 
of those sort of patches  testing for coreutils programs.

http://www.mail-archive.com/pt...@pe.../msg05228.html

> 
> Dennis 
> 
> 
        Hugh