infomap-nlp-users

[infomap-nlp-users] Problem executing infomap-build : core dumped after initializing wordlist

[Tonio W. <ton...@un...>]

Hello,

when running a build on a rather small corpus (14 M words), infomap=20
stopped the execution, telling me "Error while dumping state (probably=20
corrupted stack)".
I work under cygwin and I have never had any problems before. I've done=20
exactly the same before using much larger corpora, and they ran=20
smoothly. Do you have any idea of what has gone wrong?

Thanks !

Tonio

P.S. Here's the shell output:

$ nice infomap-build -w /cygdrive/f/IMmodels/ -p=20
/cygdrive/f/IMmodels/params_de1.txt -s /cygdrive/f/IMmodels/taz.wsp TAZ
Sourcing param file "/cygdrive/f/IMmodels/params_de1.txt"
Sourcing extra param file "/tmp/infomap-build.kt3160"
Contents are:
Removing extra param file
WORKING_DATA_DIR =3D "/cygdrive/f/IMmodels//Test2"
CORPUS_DIR =3D "/cygdrive/f/IMmodels"
CORPUS_FILE =3D "/cygdrive/f/IMmodels/test.wsp"
FNAMES_FILE =3D ""
ROWS =3D "80000"
COLUMNS =3D "2999"
SINGVALS =3D "150"
SVD_ITER =3D "450"
PRE_CONTEXT_SIZE =3D "100"
POST_CONTEXT_SIZE =3D "100"
WRITE_MATLAB_FORMAT =3D "0"
VALID_CHARS_FILE =3D "valid_chars.de"
STOPLIST_FILE =3D "stopDEucf.txt"
COL_LABELS_FROM_FILE =3D "0"
COL_LABEL_FILE =3D ""
echo "Making datadir"
Making datadir
mkdir -p /cygdrive/f/IMmodels//Test2

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Building target: /cygdrive/f/IMmodels//Test2/wordlist
Prerequisites: /cygdrive/f/IMmodels/test.wsp
Wed Nov 1 15:49:31 WEST 2006
..................................................
prepare_corpus \
                 -cdir "/cygdrive/f/IMmodels" \
                 -mdir "/cygdrive/f/IMmodels//Test2" \
                 -cfile "/cygdrive/f/IMmodels/test.wsp" \
                 -fnfile "" \
                 -chfile "valid_chars.de" \
                 -slfile "stopDEucf.txt" \
                 -rptfile ""
Locale set to (null).
Opening File for "r":
         "valid_chars.de"
Opening File for "r":
         ""
my_fopen: No such file or directory
Opening File for "r":
         "stopDEucf.txt"
Opening File for "w":
         "/cygdrive/f/IMmodels//Test2/wordlist"
Opening File for "r":
         "/cygdrive/f/IMmodels/test.wsp"
Opening File for "w":
         "/cygdrive/f/IMmodels//Test2/numDocs"
Typecount =3D 499545
Preparing to sort ... Sorting ... Done.
Opening File for "w":
         "/cygdrive/f/IMmodels//Test2/dic"
..................................................
Finishing target: /cygdrive/f/IMmodels//Test2/wordlist
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D


=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Building target: /cygdrive/f/IMmodels//Test2/coll
Prerequisites: /cygdrive/f/IMmodels//Test2/wordlist=20
/cygdrive/f/IMmodels//Test2/dic /cygdrive/f/IMmodels//Test2/numDocs
Wed Nov 1 15:50:26 WEST 2006
..................................................
count_wordvec \
                 -mdir /cygdrive/f/IMmodels//Test2 \
                 -matlab 0 \
                 -precontext 100 \
                 -postcontext 100 \
                 -rows 80000 \
                 -columns 2999 \
                 -col_labels_from_file 0 \
                 -col_label_file ""
model data dir is "/cygdrive/f/IMmodels//Test2".
count_wordvec.c: looking for 80000 rows
which had better match 80000
Reading the dictionary... Opening File for "r":
         "/cygdrive/f/IMmodels//Test2/dic"
Opening File for "r":
         "/cygdrive/f/IMmodels//Test2/numDocs"
Initializing row indices...Done.
Initializing column indices...Done.
Allocating matrix memory...done.
Initializing matrix...done.
model data dir is "/cygdrive/f/IMmodels//Test2".
count_wordvec.c: about to call process_wordlist
Entering process_wordlist.
About to call initialize_wordlist.
Opening File for "r":
         "/cygdrive/f/IMmodels//Test2/wordlist"
Returned from initialize_wordlist.
     264 [main] count_wordvec 3864 _cygtls::handle_exceptions: Error=20
while dumping state (probably corrupted stack)
make: *** [/cygdrive/f/IMmodels//Test2/coll] Segmentation fault (core=20
dumped)


--=20
Tonio Wandmacher
Wiss. Mitarbeiter
Institut f=FCr Kognitionswissenschaft
AG K=FCnstliche Intelligenz
Universit=E4t Osnabr=FCck
Albrechtstra=DFe 28
D-49076 Osnabr=FCck

Raum 31/450c
Tel +49 (0)541 969-3391
E-Mail: ton...@un...

Re: [infomap-nlp-users] Problem executing infomap-build : core dumped after initializing wordlist

[Scott C. <ced...@gm...>]

Hi Tonio,

    Well, I'm not at all sure what is causing this.  A couple of
things you might want to try:

        1. Rebuild the software (configure/make) and try again.
        2. Run the command that failed count_wordvec command that
failed (shown below)
within gdb, and do a backtrace to find out where the segmentation
fault happened.  Report back.

      Sorry I can't be any more helpful... good luck.


                        Scott

count_wordvec \
                  -mdir /cygdrive/f/IMmodels//Test2 \
                  -matlab 0 \
                  -precontext 100 \
                  -postcontext 100 \
                  -rows 80000 \
                  -columns 2999 \
                  -col_labels_from_file 0 \
                  -col_label_file ""

On 11/1/06, Tonio Wandmacher <ton...@un...> wrote:
> Hello,
>
> when running a build on a rather small corpus (14 M words), infomap
> stopped the execution, telling me "Error while dumping state (probably
> corrupted stack)".
> I work under cygwin and I have never had any problems before. I've done
> exactly the same before using much larger corpora, and they ran
> smoothly. Do you have any idea of what has gone wrong?
>
> Thanks !
>
> Tonio
>
> P.S. Here's the shell output:
>
> $ nice infomap-build -w /cygdrive/f/IMmodels/ -p
> /cygdrive/f/IMmodels/params_de1.txt -s /cygdrive/f/IMmodels/taz.wsp TAZ
> Sourcing param file "/cygdrive/f/IMmodels/params_de1.txt"
> Sourcing extra param file "/tmp/infomap-build.kt3160"
> Contents are:
> Removing extra param file
> WORKING_DATA_DIR =3D "/cygdrive/f/IMmodels//Test2"
> CORPUS_DIR =3D "/cygdrive/f/IMmodels"
> CORPUS_FILE =3D "/cygdrive/f/IMmodels/test.wsp"
> FNAMES_FILE =3D ""
> ROWS =3D "80000"
> COLUMNS =3D "2999"
> SINGVALS =3D "150"
> SVD_ITER =3D "450"
> PRE_CONTEXT_SIZE =3D "100"
> POST_CONTEXT_SIZE =3D "100"
> WRITE_MATLAB_FORMAT =3D "0"
> VALID_CHARS_FILE =3D "valid_chars.de"
> STOPLIST_FILE =3D "stopDEucf.txt"
> COL_LABELS_FROM_FILE =3D "0"
> COL_LABEL_FILE =3D ""
> echo "Making datadir"
> Making datadir
> mkdir -p /cygdrive/f/IMmodels//Test2
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
> Building target: /cygdrive/f/IMmodels//Test2/wordlist
> Prerequisites: /cygdrive/f/IMmodels/test.wsp
> Wed Nov 1 15:49:31 WEST 2006
> ..................................................
> prepare_corpus \
>                  -cdir "/cygdrive/f/IMmodels" \
>                  -mdir "/cygdrive/f/IMmodels//Test2" \
>                  -cfile "/cygdrive/f/IMmodels/test.wsp" \
>                  -fnfile "" \
>                  -chfile "valid_chars.de" \
>                  -slfile "stopDEucf.txt" \
>                  -rptfile ""
> Locale set to (null).
> Opening File for "r":
>          "valid_chars.de"
> Opening File for "r":
>          ""
> my_fopen: No such file or directory
> Opening File for "r":
>          "stopDEucf.txt"
> Opening File for "w":
>          "/cygdrive/f/IMmodels//Test2/wordlist"
> Opening File for "r":
>          "/cygdrive/f/IMmodels/test.wsp"
> Opening File for "w":
>          "/cygdrive/f/IMmodels//Test2/numDocs"
> Typecount =3D 499545
> Preparing to sort ... Sorting ... Done.
> Opening File for "w":
>          "/cygdrive/f/IMmodels//Test2/dic"
> ..................................................
> Finishing target: /cygdrive/f/IMmodels//Test2/wordlist
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
>
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
> Building target: /cygdrive/f/IMmodels//Test2/coll
> Prerequisites: /cygdrive/f/IMmodels//Test2/wordlist
> /cygdrive/f/IMmodels//Test2/dic /cygdrive/f/IMmodels//Test2/numDocs
> Wed Nov 1 15:50:26 WEST 2006
> ..................................................
> count_wordvec \
>                  -mdir /cygdrive/f/IMmodels//Test2 \
>                  -matlab 0 \
>                  -precontext 100 \
>                  -postcontext 100 \
>                  -rows 80000 \
>                  -columns 2999 \
>                  -col_labels_from_file 0 \
>                  -col_label_file ""
> model data dir is "/cygdrive/f/IMmodels//Test2".
> count_wordvec.c: looking for 80000 rows
> which had better match 80000
> Reading the dictionary... Opening File for "r":
>          "/cygdrive/f/IMmodels//Test2/dic"
> Opening File for "r":
>          "/cygdrive/f/IMmodels//Test2/numDocs"
> Initializing row indices...Done.
> Initializing column indices...Done.
> Allocating matrix memory...done.
> Initializing matrix...done.
> model data dir is "/cygdrive/f/IMmodels//Test2".
> count_wordvec.c: about to call process_wordlist
> Entering process_wordlist.
> About to call initialize_wordlist.
> Opening File for "r":
>          "/cygdrive/f/IMmodels//Test2/wordlist"
> Returned from initialize_wordlist.
>      264 [main] count_wordvec 3864 _cygtls::handle_exceptions: Error
> while dumping state (probably corrupted stack)
> make: *** [/cygdrive/f/IMmodels//Test2/coll] Segmentation fault (core
> dumped)
>
>
> --
> Tonio Wandmacher
> Wiss. Mitarbeiter
> Institut f=FCr Kognitionswissenschaft
> AG K=FCnstliche Intelligenz
> Universit=E4t Osnabr=FCck
> Albrechtstra=DFe 28
> D-49076 Osnabr=FCck
>
> Raum 31/450c
> Tel +49 (0)541 969-3391
> E-Mail: ton...@un...
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job ea=
sier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronim=
o
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=
=3D121642
> _______________________________________________
> infomap-nlp-users mailing list
> inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infomap-nlp-users
>