Menu
▾
▴
infomap-nlp-users — General questions and discussion about how to use the software.
You can subscribe to this list here.
| 2004 |
Jan
(1) |
Feb
|
Mar
(8) |
Apr
(6) |
May
(6) |
Jun
(1) |
Jul
|
Aug
(2) |
Sep
|
Oct
(7) |
Nov
(1) |
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2005 |
Jan
(9) |
Feb
(13) |
Mar
(2) |
Apr
(2) |
May
(14) |
Jun
(9) |
Jul
|
Aug
(1) |
Sep
|
Oct
(15) |
Nov
(1) |
Dec
(1) |
| 2006 |
Jan
(2) |
Feb
(9) |
Mar
|
Apr
(1) |
May
(3) |
Jun
(3) |
Jul
(8) |
Aug
(8) |
Sep
(3) |
Oct
(6) |
Nov
(4) |
Dec
|
| 2007 |
Jan
(6) |
Feb
(5) |
Mar
(5) |
Apr
(15) |
May
(4) |
Jun
|
Jul
(4) |
Aug
(20) |
Sep
(14) |
Oct
(7) |
Nov
(11) |
Dec
(5) |
| 2008 |
Jan
(4) |
Feb
(5) |
Mar
(34) |
Apr
(35) |
May
(10) |
Jun
(14) |
Jul
(35) |
Aug
(15) |
Sep
(17) |
Oct
(21) |
Nov
(43) |
Dec
(40) |
| 2009 |
Jan
(37) |
Feb
(22) |
Mar
(45) |
Apr
(48) |
May
(123) |
Jun
(103) |
Jul
(71) |
Aug
(25) |
Sep
(19) |
Oct
(42) |
Nov
(12) |
Dec
(22) |
| 2010 |
Jan
(12) |
Feb
(18) |
Mar
(39) |
Apr
(59) |
May
(67) |
Jun
(65) |
Jul
(37) |
Aug
(39) |
Sep
(20) |
Oct
(3) |
Nov
|
Dec
(1) |
| 2011 |
Jan
(3) |
Feb
(6) |
Mar
(7) |
Apr
(1) |
May
(1) |
Jun
(4) |
Jul
(7) |
Aug
(4) |
Sep
(4) |
Oct
|
Nov
(4) |
Dec
(1) |
| 2012 |
Jan
(4) |
Feb
(5) |
Mar
(3) |
Apr
(4) |
May
(2) |
Jun
(1) |
Jul
(1) |
Aug
(5) |
Sep
(2) |
Oct
(4) |
Nov
(4) |
Dec
(1) |
| 2013 |
Jan
(7) |
Feb
(4) |
Mar
(2) |
Apr
(2) |
May
(3) |
Jun
(5) |
Jul
(9) |
Aug
(6) |
Sep
(6) |
Oct
(5) |
Nov
(7) |
Dec
(3) |
| 2014 |
Jan
(7) |
Feb
(4) |
Mar
(8) |
Apr
(7) |
May
(7) |
Jun
(2) |
Jul
(6) |
Aug
(4) |
Sep
(7) |
Oct
(9) |
Nov
(3) |
Dec
(1) |
| 2015 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(1) |
Dec
(1) |
| 2016 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2017 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Dominic W. <wi...@ma...> - 2005-10-04 14:51:21
|
Dear Arooj, Two questions I can think of that affect your question is i. is space a concern? and ii. how reliable do you want the results to be? The second question is affected crucially by the token frequency of the=20= types that are being indexed. In my (ad hoc) experience, by the time=20 you have about 20 or 30 occurrences of a word type, things are looking=20= reasonably stable. If you have fewer than 10, things are looking very=20 hit and miss. Those who have experimented with the King James Bible=20 corpus will probably have seen for themselves that infrequent words=20 like "kirioth" seem to crop up in very odd places. This suggests rephrasing the question as follows. In a corpus with m=20 tokens belonging to n < m types, how many of the types do you expect to=20= occur with frequency greater than some "stability threshold"? I believe=20= that the range 10 to 30 is a pretty good guess for this threshold, but=20= a guess it remains. One should be able to do a bit of counting and=20 comparing with Zipf's law to firm up the mathematics of this=20 suggestion. Am I answering the right question? Best wishes, Dominic On Oct 4, 2005, at 2:46 AM, Arooj Asghar wrote: > Hi, > =A0 > I=92ve wanted to index a large corpus, with almost 170K words. The=20 > default 20K doesn=92t work well for this. Is there a way of telling = what=20 > the optimal number of rows for a corpus of =91n=92 words should be? > =A0 > Thanks, > Arooj > =A0 > =A0 > > Express yourself instantly with MSN Messenger! MSN Messenger Download=20= > today it's FREE! =20 > ------------------------------------------------------- This SF.Net=20 > email is sponsored by: Power Architecture Resource Center: Free=20 > content, downloads, discussions, and more.=20 > http://solutions.newsforge.com/ibmarch.tmpl=20 > _______________________________________________ infomap-nlp-users=20 > mailing list inf...@li...=20 > https://lists.sourceforge.net/lists/listinfo/infomap-nlp-users= |
|
From: Arooj A. <aro...@ho...> - 2005-10-04 06:47:13
|
<html><div style='background-color:'><DIV class=RTE> <P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Hi,<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /><o:p></o:p></SPAN></P> <P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></P> <P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Ive wanted to index a large corpus, with almost 170K words. The default 20K doesnt work well for this. Is there a way of telling what the optimal number of rows for a corpus of n words should be?<o:p></o:p></SPAN></P> <P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></P> <P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Thanks,<o:p></o:p></SPAN></P> <P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Arooj<o:p></o:p></SPAN></P> <P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></P> <P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></P></DIV></div><br clear=all><hr>Express yourself instantly with MSN Messenger! <a href="http://g.msn.com/8HMBEN/2743??PS=47575" target="_top">MSN Messenger</a> Download today it's FREE!</html> |
|
From: Alan J. S. <sa...@gm...> - 2005-10-03 18:17:11
|
Hi Dominic and Scott, Well infomap works on OS X (10.4.2) now (haven't tried Ubuntu yet, but I cannot see any reason why it would not), so thanks very much for your help. There were three problems to be dealt with: 1) As Scott mentioned, the <DOC><TEXT> tags had to be before each text (rather than just one text - /me stupid!); 2) As Dominic said, the tags had to be in uppercase (quick regexp in Emacs and all's sorted out); 3) I had to set 'SVD_ITER=3D400' when building the model; The last one wasn't necessary, but without it I had the warning of 'too few iterations'. And now it seems to work - well, the associate command with 'god' works (when god is in lower case) giving me a nice list of associated words. Brilliant! I'm off to play with my new toy. Thanks very much again to both of you for your help (and for the software). Alan. On 03/10/05, Dominic Widdows <wi...@ma...> wrote: > Hi Alan, > > Another thing to check is whether anything is getting written to your > model directory. Running "ls -l" on the output files can give you a > good idea of how far the processing got while still producing useful > data. If you want to trun this on your files and send us the output > that might help. > > However, since you have the line "typecount =3D 0" in the output to > "prepare_corpus" below, I'm guessing that even the wordlists are not > being properly computed. > > One frustrating but possible suggestion is that the tags are > case-sensitive, and you should try <DOC> instead of <doc>, etc. > > Best wishes, > Dominic > > On Oct 3, 2005, at 12:23 AM, Scott Cederberg wrote: > > > On 10/1/05, Alan J. Salmoni <sa...@gm...> wrote: > >> Hi. > >> > >> I'm using infomap 0.8.6 to build a matrix but nothing I do seems to > >> work. > >> > >> I've formatted the King James' Bible using the doc and text tags thus: > >> > >> <doc> > >> <text> > >> The Old Testament of... > >> [...missing text...] > >> End of the Project Gutenberg Edition of the King James Bible > >> </text> > >> </doc> > >> > > > > Do you have only a single set of <doc>/</doc> tags and a single set of > > <text>/</text> tags surrounding the entire document? (It seems like > > that's what's suggested above, but I'm not sure.) If so, the system > > is interpreting the entire Bible as one long document. I'm not sure > > how this would lead to the error below (there should still be word > > vectors), but it'd be worth retrying with a set of <doc> and <text> > > tags around, say, every chapter. > > > > Let me know if that helps... > > > > > > Scott > > > >> and tried the infomap-build command thus: > >> > >> :infomap-build -s /Users/alan/Projects/infomap/corpus/kjv10.txt test1 > >> > >> The output is below. However, when I run the associate command thus: > >> > >> :associate -t -c test1 God > >> > >> I get: > >> > >> :No word/document vector for "God". > >> > >> Or "GOD" or "god". I've tried this on OS X, Ubuntu and Mandrake Linux > >> all with the same results. I've checked the KJ Bible to ensure that no > >> errant tags are contained (they are not) and followed the examples in > >> the user manual to the letter and still nothing. I've also tried > >> changing the SVD_ITER value to 400 thus: > >> > >> :infomap-build -D SVD_ITER=3D400 -s > >> /Users/alan/Projects/infomap/corpus/kjv10.txt test1 > >> > >> which took the new value into account, but again the associate command > >> doesn't work. Can anyone shed any light on what is going wrong or on > >> what I should be doing but don't yet know about? > >> > >> Rgds, > >> > >> Alan J. Salmoni. > >> > >> This is the output of the infomap-build command (with SVD-ITER > >> changed to 400): > >> > >> Sourcing param file "/usr/local/share/infomap-nlp/default-params" > >> Sourcing extra param file "/tmp/infomap-build.kyInQJ" > >> Contents are: > >> SVD_ITER=3D400 > >> Removing extra param file > >> WORKING_DATA_DIR =3D "/Users/alan/Projects/infomap/corpus/models/test1= " > >> CORPUS_DIR =3D "/Users/alan/Projects/infomap/corpus" > >> CORPUS_FILE =3D "/Users/alan/Projects/infomap/corpus/kjv10.txt" > >> FNAMES_FILE =3D "" > >> ROWS =3D "20000" > >> COLUMNS =3D "1000" > >> SINGVALS =3D "100" > >> SVD_ITER =3D "400" > >> PRE_CONTEXT_SIZE =3D "15" > >> POST_CONTEXT_SIZE =3D "15" > >> WRITE_MATLAB_FORMAT =3D "0" > >> VALID_CHARS_FILE =3D "/usr/local/share/infomap-nlp/valid_chars.en" > >> STOPLIST_FILE =3D "/usr/local/share/infomap-nlp/stop.list" > >> COL_LABELS_FROM_FILE =3D "0" > >> COL_LABEL_FILE =3D "" > >> echo "Making datadir" > >> Making datadir > >> mkdir -p /Users/alan/Projects/infomap/corpus/models/test1 > >> > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> Building target: > >> /Users/alan/Projects/infomap/corpus/models/test1/wordlist > >> Prerequisites: /Users/alan/Projects/infomap/corpus/kjv10.txt > >> Sat Oct 1 15:49:47 BST 2005 > >> .................................................. > >> prepare_corpus \ > >> -cdir "/Users/alan/Projects/infomap/corpus" \ > >> -mdir "/Users/alan/Projects/infomap/corpus/models/test1" \ > >> -cfile "/Users/alan/Projects/infomap/corpus/kjv10.txt" \ > >> -fnfile "" \ > >> -chfile "/usr/local/share/infomap-nlp/valid_chars.en" \ > >> -slfile "/usr/local/share/infomap-nlp/stop.list" \ > >> -rptfile "" > >> Locale set to en_US. > >> Opening File for "r": > >> "/usr/local/share/infomap-nlp/valid_chars.en" > >> Opening File for "r": > >> "" > >> my_fopen: No such file or directory > >> Opening File for "r": > >> "/usr/local/share/infomap-nlp/stop.list" > >> Opening File for "w": > >> "/Users/alan/Projects/infomap/corpus/models/test1/wordlist" > >> Opening File for "r": > >> "/Users/alan/Projects/infomap/corpus/kjv10.txt" > >> Opening File for "w": > >> "/Users/alan/Projects/infomap/corpus/models/test1/numDocs" > >> Typecount =3D 0 > >> Preparing to sort ... Sorting ... Done. > >> Opening File for "w": > >> "/Users/alan/Projects/infomap/corpus/models/test1/dic" > >> .................................................. > >> Finishing target: > >> /Users/alan/Projects/infomap/corpus/models/test1/wordlist > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> > >> > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> Building target: /Users/alan/Projects/infomap/corpus/models/test1/coll > >> Prerequisites: > >> /Users/alan/Projects/infomap/corpus/models/test1/wordlist > >> /Users/alan/Projects/infomap/corpus/models/test1/dic > >> /Users/alan/Projects/infomap/corpus/models/test1/numDocs > >> Sat Oct 1 15:49:48 BST 2005 > >> .................................................. > >> count_wordvec \ > >> -mdir /Users/alan/Projects/infomap/corpus/models/test1 \ > >> -matlab 0 \ > >> -precontext 15 \ > >> -postcontext 15 \ > >> -rows 20000 \ > >> -columns 1000 \ > >> -col_labels_from_file 0 \ > >> -col_label_file "" > >> model data dir is "/Users/alan/Projects/infomap/corpus/models/test1". > >> count_wordvec.c: looking for 0 rows > >> which had better match 0 > >> Reading the dictionary... Opening File for "r": > >> "/Users/alan/Projects/infomap/corpus/models/test1/dic" > >> Opening File for "r": > >> "/Users/alan/Projects/infomap/corpus/models/test1/numDocs" > >> Initializing row indices...Done. > >> Initializing column indices...Done. > >> Allocating matrix memory...done. > >> Initializing matrix...done. > >> model data dir is "/Users/alan/Projects/infomap/corpus/models/test1". > >> count_wordvec.c: about to call process_wordlist > >> Entering process_wordlist. > >> About to call initialize_wordlist. > >> Opening File for "r": > >> "/Users/alan/Projects/infomap/corpus/models/test1/wordlist" > >> Returned from initialize_wordlist. > >> Writing the co-occurrence matrix. > >> Entering write_matrix_svd; rows =3D 0 and columns =3D 1000. > >> Opening File for "w": > >> "/Users/alan/Projects/infomap/corpus/models/test1/coll" > >> Opening File for "w": > >> "/Users/alan/Projects/infomap/corpus/models/test1/indx" > >> .................................................. > >> Finishing target: > >> /Users/alan/Projects/infomap/corpus/models/test1/coll > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> > >> > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> Building target: /Users/alan/Projects/infomap/corpus/models/test1/left > >> Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/coll > >> /Users/alan/Projects/infomap/corpus/models/test1/indx > >> Sat Oct 1 15:49:48 BST 2005 > >> .................................................. > >> cd /Users/alan/Projects/infomap/corpus/models/test1 && rm -f svd_diag > >> left \ > >> rght sing > >> cd /Users/alan/Projects/infomap/corpus/models/test1 && svdinterface \ > >> -singvals 100 \ > >> -iter 400 > >> > >> This is svdinterface. > >> > >> Writing to: left > >> Writing to: rght > >> Writing to: sing > >> Writing to: svd_diag > >> Reading: indx > >> Reading: indx > >> Reading: coll > >> > >> FEWER THAN EXPECTED SINGULAR VALUES > >> .................................................. > >> Finishing target: > >> /Users/alan/Projects/infomap/corpus/models/test1/left > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> > >> > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> Building target: > >> /Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin > >> Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/left > >> /Users/alan/Projects/infomap/corpus/models/test1/dic > >> Sat Oct 1 15:49:49 BST 2005 > >> .................................................. > >> encode_wordvec \ > >> -m /Users/alan/Projects/infomap/corpus/models/test1 > >> Opening File for "r": > >> "/Users/alan/Projects/infomap/corpus/models/test1/left" > >> Opening File for "w": > >> "/Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin" > >> Reading the dictionary... > >> Opening File for "r": > >> "/Users/alan/Projects/infomap/corpus/models/test1/dic" > >> Initializing row indices...Done. > >> .................................................. > >> Finishing target: > >> /Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> > >> > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> Building target: > >> /Users/alan/Projects/infomap/corpus/models/test1/artvec.bin > >> Prerequisites: > >> /Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin > >> /Users/alan/Projects/infomap/corpus/models/test1/wordlist > >> /Users/alan/Projects/infomap/corpus/models/test1/dic > >> /Users/alan/Projects/infomap/corpus/models/test1/numDocs > >> Sat Oct 1 15:49:49 BST 2005 > >> .................................................. > >> count_artvec -m /Users/alan/Projects/infomap/corpus/models/test1 > >> Opening File for "w": > >> "/Users/alan/Projects/infomap/corpus/models/test1/artvec.bin" > >> Opening File for "r": > >> "/Users/alan/Projects/infomap/corpus/models/test1/numDocs" > >> Reading the dictionary... Opening File for "r": > >> "/Users/alan/Projects/infomap/corpus/models/test1/dic" > >> Initializing row indices...Done. > >> Allocating matrix memory...done. > >> Initializing matrix...done. > >> Opening File for "r": > >> "/Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin" > >> count_artvec.c: about to read 0 rows from wordvector file. > >> Entering process_wordlist. > >> About to call initialize_wordlist. > >> Opening File for "r": > >> "/Users/alan/Projects/infomap/corpus/models/test1/wordlist" > >> Returned from initialize_wordlist. > >> .................................................. > >> Finishing target: > >> /Users/alan/Projects/infomap/corpus/models/test1/artvec.bin > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> > >> > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> Building target: > >> /Users/alan/Projects/infomap/corpus/models/test1/model_params.txt > >> Prerequisites: > >> /Users/alan/Projects/infomap/corpus/models/test1/model_params.bin > >> /Users/alan/Projects/infomap/corpus/models/test1/model_info.bin > >> /Users/alan/Projects/infomap/corpus/models/test1/corpus_format.bin > >> Sat Oct 1 15:49:49 BST 2005 > >> .................................................. > >> write_text_params -mdir > >> /Users/alan/Projects/infomap/corpus/models/test1 > >> .................................................. > >> Finishing target: > >> /Users/alan/Projects/infomap/corpus/models/test1/model_params.txt > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > >> > >> > >> ------------------------------------------------------- > >> This SF.Net email is sponsored by: > >> Power Architecture Resource Center: Free content, downloads, > >> discussions, > >> and more. http://solutions.newsforge.com/ibmarch.tmpl > >> _______________________________________________ > >> infomap-nlp-users mailing list > >> inf...@li... > >> https://lists.sourceforge.net/lists/listinfo/infomap-nlp-users > >> > > > > > > ------------------------------------------------------- > > This SF.Net email is sponsored by: > > Power Architecture Resource Center: Free content, downloads, > > discussions, > > and more. http://solutions.newsforge.com/ibmarch.tmpl > > _______________________________________________ > > infomap-nlp-users mailing list > > inf...@li... > > https://lists.sourceforge.net/lists/listinfo/infomap-nlp-users > > > > -- Alan James Salmoni |
|
From: Dominic W. <wi...@ma...> - 2005-10-03 16:12:16
|
Hi Alan, Another thing to check is whether anything is getting written to your model directory. Running "ls -l" on the output files can give you a good idea of how far the processing got while still producing useful data. If you want to trun this on your files and send us the output that might help. However, since you have the line "typecount = 0" in the output to "prepare_corpus" below, I'm guessing that even the wordlists are not being properly computed. One frustrating but possible suggestion is that the tags are case-sensitive, and you should try <DOC> instead of <doc>, etc. Best wishes, Dominic On Oct 3, 2005, at 12:23 AM, Scott Cederberg wrote: > On 10/1/05, Alan J. Salmoni <sa...@gm...> wrote: >> Hi. >> >> I'm using infomap 0.8.6 to build a matrix but nothing I do seems to >> work. >> >> I've formatted the King James' Bible using the doc and text tags thus: >> >> <doc> >> <text> >> The Old Testament of... >> [...missing text...] >> End of the Project Gutenberg Edition of the King James Bible >> </text> >> </doc> >> > > Do you have only a single set of <doc>/</doc> tags and a single set of > <text>/</text> tags surrounding the entire document? (It seems like > that's what's suggested above, but I'm not sure.) If so, the system > is interpreting the entire Bible as one long document. I'm not sure > how this would lead to the error below (there should still be word > vectors), but it'd be worth retrying with a set of <doc> and <text> > tags around, say, every chapter. > > Let me know if that helps... > > > Scott > >> and tried the infomap-build command thus: >> >> :infomap-build -s /Users/alan/Projects/infomap/corpus/kjv10.txt test1 >> >> The output is below. However, when I run the associate command thus: >> >> :associate -t -c test1 God >> >> I get: >> >> :No word/document vector for "God". >> >> Or "GOD" or "god". I've tried this on OS X, Ubuntu and Mandrake Linux >> all with the same results. I've checked the KJ Bible to ensure that no >> errant tags are contained (they are not) and followed the examples in >> the user manual to the letter and still nothing. I've also tried >> changing the SVD_ITER value to 400 thus: >> >> :infomap-build -D SVD_ITER=400 -s >> /Users/alan/Projects/infomap/corpus/kjv10.txt test1 >> >> which took the new value into account, but again the associate command >> doesn't work. Can anyone shed any light on what is going wrong or on >> what I should be doing but don't yet know about? >> >> Rgds, >> >> Alan J. Salmoni. >> >> This is the output of the infomap-build command (with SVD-ITER >> changed to 400): >> >> Sourcing param file "/usr/local/share/infomap-nlp/default-params" >> Sourcing extra param file "/tmp/infomap-build.kyInQJ" >> Contents are: >> SVD_ITER=400 >> Removing extra param file >> WORKING_DATA_DIR = "/Users/alan/Projects/infomap/corpus/models/test1" >> CORPUS_DIR = "/Users/alan/Projects/infomap/corpus" >> CORPUS_FILE = "/Users/alan/Projects/infomap/corpus/kjv10.txt" >> FNAMES_FILE = "" >> ROWS = "20000" >> COLUMNS = "1000" >> SINGVALS = "100" >> SVD_ITER = "400" >> PRE_CONTEXT_SIZE = "15" >> POST_CONTEXT_SIZE = "15" >> WRITE_MATLAB_FORMAT = "0" >> VALID_CHARS_FILE = "/usr/local/share/infomap-nlp/valid_chars.en" >> STOPLIST_FILE = "/usr/local/share/infomap-nlp/stop.list" >> COL_LABELS_FROM_FILE = "0" >> COL_LABEL_FILE = "" >> echo "Making datadir" >> Making datadir >> mkdir -p /Users/alan/Projects/infomap/corpus/models/test1 >> >> ================================================== >> Building target: >> /Users/alan/Projects/infomap/corpus/models/test1/wordlist >> Prerequisites: /Users/alan/Projects/infomap/corpus/kjv10.txt >> Sat Oct 1 15:49:47 BST 2005 >> .................................................. >> prepare_corpus \ >> -cdir "/Users/alan/Projects/infomap/corpus" \ >> -mdir "/Users/alan/Projects/infomap/corpus/models/test1" \ >> -cfile "/Users/alan/Projects/infomap/corpus/kjv10.txt" \ >> -fnfile "" \ >> -chfile "/usr/local/share/infomap-nlp/valid_chars.en" \ >> -slfile "/usr/local/share/infomap-nlp/stop.list" \ >> -rptfile "" >> Locale set to en_US. >> Opening File for "r": >> "/usr/local/share/infomap-nlp/valid_chars.en" >> Opening File for "r": >> "" >> my_fopen: No such file or directory >> Opening File for "r": >> "/usr/local/share/infomap-nlp/stop.list" >> Opening File for "w": >> "/Users/alan/Projects/infomap/corpus/models/test1/wordlist" >> Opening File for "r": >> "/Users/alan/Projects/infomap/corpus/kjv10.txt" >> Opening File for "w": >> "/Users/alan/Projects/infomap/corpus/models/test1/numDocs" >> Typecount = 0 >> Preparing to sort ... Sorting ... Done. >> Opening File for "w": >> "/Users/alan/Projects/infomap/corpus/models/test1/dic" >> .................................................. >> Finishing target: >> /Users/alan/Projects/infomap/corpus/models/test1/wordlist >> ================================================== >> >> >> ================================================== >> Building target: /Users/alan/Projects/infomap/corpus/models/test1/coll >> Prerequisites: >> /Users/alan/Projects/infomap/corpus/models/test1/wordlist >> /Users/alan/Projects/infomap/corpus/models/test1/dic >> /Users/alan/Projects/infomap/corpus/models/test1/numDocs >> Sat Oct 1 15:49:48 BST 2005 >> .................................................. >> count_wordvec \ >> -mdir /Users/alan/Projects/infomap/corpus/models/test1 \ >> -matlab 0 \ >> -precontext 15 \ >> -postcontext 15 \ >> -rows 20000 \ >> -columns 1000 \ >> -col_labels_from_file 0 \ >> -col_label_file "" >> model data dir is "/Users/alan/Projects/infomap/corpus/models/test1". >> count_wordvec.c: looking for 0 rows >> which had better match 0 >> Reading the dictionary... Opening File for "r": >> "/Users/alan/Projects/infomap/corpus/models/test1/dic" >> Opening File for "r": >> "/Users/alan/Projects/infomap/corpus/models/test1/numDocs" >> Initializing row indices...Done. >> Initializing column indices...Done. >> Allocating matrix memory...done. >> Initializing matrix...done. >> model data dir is "/Users/alan/Projects/infomap/corpus/models/test1". >> count_wordvec.c: about to call process_wordlist >> Entering process_wordlist. >> About to call initialize_wordlist. >> Opening File for "r": >> "/Users/alan/Projects/infomap/corpus/models/test1/wordlist" >> Returned from initialize_wordlist. >> Writing the co-occurrence matrix. >> Entering write_matrix_svd; rows = 0 and columns = 1000. >> Opening File for "w": >> "/Users/alan/Projects/infomap/corpus/models/test1/coll" >> Opening File for "w": >> "/Users/alan/Projects/infomap/corpus/models/test1/indx" >> .................................................. >> Finishing target: >> /Users/alan/Projects/infomap/corpus/models/test1/coll >> ================================================== >> >> >> ================================================== >> Building target: /Users/alan/Projects/infomap/corpus/models/test1/left >> Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/coll >> /Users/alan/Projects/infomap/corpus/models/test1/indx >> Sat Oct 1 15:49:48 BST 2005 >> .................................................. >> cd /Users/alan/Projects/infomap/corpus/models/test1 && rm -f svd_diag >> left \ >> rght sing >> cd /Users/alan/Projects/infomap/corpus/models/test1 && svdinterface \ >> -singvals 100 \ >> -iter 400 >> >> This is svdinterface. >> >> Writing to: left >> Writing to: rght >> Writing to: sing >> Writing to: svd_diag >> Reading: indx >> Reading: indx >> Reading: coll >> >> FEWER THAN EXPECTED SINGULAR VALUES >> .................................................. >> Finishing target: >> /Users/alan/Projects/infomap/corpus/models/test1/left >> ================================================== >> >> >> ================================================== >> Building target: >> /Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin >> Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/left >> /Users/alan/Projects/infomap/corpus/models/test1/dic >> Sat Oct 1 15:49:49 BST 2005 >> .................................................. >> encode_wordvec \ >> -m /Users/alan/Projects/infomap/corpus/models/test1 >> Opening File for "r": >> "/Users/alan/Projects/infomap/corpus/models/test1/left" >> Opening File for "w": >> "/Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin" >> Reading the dictionary... >> Opening File for "r": >> "/Users/alan/Projects/infomap/corpus/models/test1/dic" >> Initializing row indices...Done. >> .................................................. >> Finishing target: >> /Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin >> ================================================== >> >> >> ================================================== >> Building target: >> /Users/alan/Projects/infomap/corpus/models/test1/artvec.bin >> Prerequisites: >> /Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin >> /Users/alan/Projects/infomap/corpus/models/test1/wordlist >> /Users/alan/Projects/infomap/corpus/models/test1/dic >> /Users/alan/Projects/infomap/corpus/models/test1/numDocs >> Sat Oct 1 15:49:49 BST 2005 >> .................................................. >> count_artvec -m /Users/alan/Projects/infomap/corpus/models/test1 >> Opening File for "w": >> "/Users/alan/Projects/infomap/corpus/models/test1/artvec.bin" >> Opening File for "r": >> "/Users/alan/Projects/infomap/corpus/models/test1/numDocs" >> Reading the dictionary... Opening File for "r": >> "/Users/alan/Projects/infomap/corpus/models/test1/dic" >> Initializing row indices...Done. >> Allocating matrix memory...done. >> Initializing matrix...done. >> Opening File for "r": >> "/Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin" >> count_artvec.c: about to read 0 rows from wordvector file. >> Entering process_wordlist. >> About to call initialize_wordlist. >> Opening File for "r": >> "/Users/alan/Projects/infomap/corpus/models/test1/wordlist" >> Returned from initialize_wordlist. >> .................................................. >> Finishing target: >> /Users/alan/Projects/infomap/corpus/models/test1/artvec.bin >> ================================================== >> >> >> ================================================== >> Building target: >> /Users/alan/Projects/infomap/corpus/models/test1/model_params.txt >> Prerequisites: >> /Users/alan/Projects/infomap/corpus/models/test1/model_params.bin >> /Users/alan/Projects/infomap/corpus/models/test1/model_info.bin >> /Users/alan/Projects/infomap/corpus/models/test1/corpus_format.bin >> Sat Oct 1 15:49:49 BST 2005 >> .................................................. >> write_text_params -mdir >> /Users/alan/Projects/infomap/corpus/models/test1 >> .................................................. >> Finishing target: >> /Users/alan/Projects/infomap/corpus/models/test1/model_params.txt >> ================================================== >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: >> Power Architecture Resource Center: Free content, downloads, >> discussions, >> and more. http://solutions.newsforge.com/ibmarch.tmpl >> _______________________________________________ >> infomap-nlp-users mailing list >> inf...@li... >> https://lists.sourceforge.net/lists/listinfo/infomap-nlp-users >> > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, > discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > infomap-nlp-users mailing list > inf...@li... > https://lists.sourceforge.net/lists/listinfo/infomap-nlp-users > |
|
From: Scott C. <ced...@gm...> - 2005-10-03 04:24:07
|
On 10/1/05, Alan J. Salmoni <sa...@gm...> wrote:
> Hi.
>
> I'm using infomap 0.8.6 to build a matrix but nothing I do seems to work.
>
> I've formatted the King James' Bible using the doc and text tags thus:
>
> <doc>
> <text>
> The Old Testament of...
> [...missing text...]
> End of the Project Gutenberg Edition of the King James Bible
> </text>
> </doc>
>
Do you have only a single set of <doc>/</doc> tags and a single set of
<text>/</text> tags surrounding the entire document? (It seems like
that's what's suggested above, but I'm not sure.) If so, the system
is interpreting the entire Bible as one long document. I'm not sure
how this would lead to the error below (there should still be word
vectors), but it'd be worth retrying with a set of <doc> and <text>
tags around, say, every chapter.
Let me know if that helps...
=20
Scott
> and tried the infomap-build command thus:
>
> :infomap-build -s /Users/alan/Projects/infomap/corpus/kjv10.txt test1
>
> The output is below. However, when I run the associate command thus:
>
> :associate -t -c test1 God
>
> I get:
>
> :No word/document vector for "God".
>
> Or "GOD" or "god". I've tried this on OS X, Ubuntu and Mandrake Linux
> all with the same results. I've checked the KJ Bible to ensure that no
> errant tags are contained (they are not) and followed the examples in
> the user manual to the letter and still nothing. I've also tried
> changing the SVD_ITER value to 400 thus:
>
> :infomap-build -D SVD_ITER=3D400 -s
> /Users/alan/Projects/infomap/corpus/kjv10.txt test1
>
> which took the new value into account, but again the associate command
> doesn't work. Can anyone shed any light on what is going wrong or on
> what I should be doing but don't yet know about?
>
> Rgds,
>
> Alan J. Salmoni.
>
> This is the output of the infomap-build command (with SVD-ITER changed to=
400):
>
> Sourcing param file "/usr/local/share/infomap-nlp/default-params"
> Sourcing extra param file "/tmp/infomap-build.kyInQJ"
> Contents are:
> SVD_ITER=3D400
> Removing extra param file
> WORKING_DATA_DIR =3D "/Users/alan/Projects/infomap/corpus/models/test1"
> CORPUS_DIR =3D "/Users/alan/Projects/infomap/corpus"
> CORPUS_FILE =3D "/Users/alan/Projects/infomap/corpus/kjv10.txt"
> FNAMES_FILE =3D ""
> ROWS =3D "20000"
> COLUMNS =3D "1000"
> SINGVALS =3D "100"
> SVD_ITER =3D "400"
> PRE_CONTEXT_SIZE =3D "15"
> POST_CONTEXT_SIZE =3D "15"
> WRITE_MATLAB_FORMAT =3D "0"
> VALID_CHARS_FILE =3D "/usr/local/share/infomap-nlp/valid_chars.en"
> STOPLIST_FILE =3D "/usr/local/share/infomap-nlp/stop.list"
> COL_LABELS_FROM_FILE =3D "0"
> COL_LABEL_FILE =3D ""
> echo "Making datadir"
> Making datadir
> mkdir -p /Users/alan/Projects/infomap/corpus/models/test1
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
> Building target: /Users/alan/Projects/infomap/corpus/models/test1/wordlis=
t
> Prerequisites: /Users/alan/Projects/infomap/corpus/kjv10.txt
> Sat Oct 1 15:49:47 BST 2005
> ..................................................
> prepare_corpus \
> -cdir "/Users/alan/Projects/infomap/corpus" \
> -mdir "/Users/alan/Projects/infomap/corpus/models/test1" \
> -cfile "/Users/alan/Projects/infomap/corpus/kjv10.txt" \
> -fnfile "" \
> -chfile "/usr/local/share/infomap-nlp/valid_chars.en" \
> -slfile "/usr/local/share/infomap-nlp/stop.list" \
> -rptfile ""
> Locale set to en_US.
> Opening File for "r":
> "/usr/local/share/infomap-nlp/valid_chars.en"
> Opening File for "r":
> ""
> my_fopen: No such file or directory
> Opening File for "r":
> "/usr/local/share/infomap-nlp/stop.list"
> Opening File for "w":
> "/Users/alan/Projects/infomap/corpus/models/test1/wordlist"
> Opening File for "r":
> "/Users/alan/Projects/infomap/corpus/kjv10.txt"
> Opening File for "w":
> "/Users/alan/Projects/infomap/corpus/models/test1/numDocs"
> Typecount =3D 0
> Preparing to sort ... Sorting ... Done.
> Opening File for "w":
> "/Users/alan/Projects/infomap/corpus/models/test1/dic"
> ..................................................
> Finishing target: /Users/alan/Projects/infomap/corpus/models/test1/wordli=
st
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
>
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
> Building target: /Users/alan/Projects/infomap/corpus/models/test1/coll
> Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/wordlist
> /Users/alan/Projects/infomap/corpus/models/test1/dic
> /Users/alan/Projects/infomap/corpus/models/test1/numDocs
> Sat Oct 1 15:49:48 BST 2005
> ..................................................
> count_wordvec \
> -mdir /Users/alan/Projects/infomap/corpus/models/test1 \
> -matlab 0 \
> -precontext 15 \
> -postcontext 15 \
> -rows 20000 \
> -columns 1000 \
> -col_labels_from_file 0 \
> -col_label_file ""
> model data dir is "/Users/alan/Projects/infomap/corpus/models/test1".
> count_wordvec.c: looking for 0 rows
> which had better match 0
> Reading the dictionary... Opening File for "r":
> "/Users/alan/Projects/infomap/corpus/models/test1/dic"
> Opening File for "r":
> "/Users/alan/Projects/infomap/corpus/models/test1/numDocs"
> Initializing row indices...Done.
> Initializing column indices...Done.
> Allocating matrix memory...done.
> Initializing matrix...done.
> model data dir is "/Users/alan/Projects/infomap/corpus/models/test1".
> count_wordvec.c: about to call process_wordlist
> Entering process_wordlist.
> About to call initialize_wordlist.
> Opening File for "r":
> "/Users/alan/Projects/infomap/corpus/models/test1/wordlist"
> Returned from initialize_wordlist.
> Writing the co-occurrence matrix.
> Entering write_matrix_svd; rows =3D 0 and columns =3D 1000.
> Opening File for "w":
> "/Users/alan/Projects/infomap/corpus/models/test1/coll"
> Opening File for "w":
> "/Users/alan/Projects/infomap/corpus/models/test1/indx"
> ..................................................
> Finishing target: /Users/alan/Projects/infomap/corpus/models/test1/coll
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
>
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
> Building target: /Users/alan/Projects/infomap/corpus/models/test1/left
> Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/coll
> /Users/alan/Projects/infomap/corpus/models/test1/indx
> Sat Oct 1 15:49:48 BST 2005
> ..................................................
> cd /Users/alan/Projects/infomap/corpus/models/test1 && rm -f svd_diag lef=
t \
> rght sing
> cd /Users/alan/Projects/infomap/corpus/models/test1 && svdinterface \
> -singvals 100 \
> -iter 400
>
> This is svdinterface.
>
> Writing to: left
> Writing to: rght
> Writing to: sing
> Writing to: svd_diag
> Reading: indx
> Reading: indx
> Reading: coll
>
> FEWER THAN EXPECTED SINGULAR VALUES
> ..................................................
> Finishing target: /Users/alan/Projects/infomap/corpus/models/test1/left
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
>
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
> Building target: /Users/alan/Projects/infomap/corpus/models/test1/wordvec=
.bin
> Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/left
> /Users/alan/Projects/infomap/corpus/models/test1/dic
> Sat Oct 1 15:49:49 BST 2005
> ..................................................
> encode_wordvec \
> -m /Users/alan/Projects/infomap/corpus/models/test1
> Opening File for "r":
> "/Users/alan/Projects/infomap/corpus/models/test1/left"
> Opening File for "w":
> "/Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin"
> Reading the dictionary...
> Opening File for "r":
> "/Users/alan/Projects/infomap/corpus/models/test1/dic"
> Initializing row indices...Done.
> ..................................................
> Finishing target: /Users/alan/Projects/infomap/corpus/models/test1/wordve=
c.bin
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
>
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
> Building target: /Users/alan/Projects/infomap/corpus/models/test1/artvec.=
bin
> Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/wordvec.b=
in
> /Users/alan/Projects/infomap/corpus/models/test1/wordlist
> /Users/alan/Projects/infomap/corpus/models/test1/dic
> /Users/alan/Projects/infomap/corpus/models/test1/numDocs
> Sat Oct 1 15:49:49 BST 2005
> ..................................................
> count_artvec -m /Users/alan/Projects/infomap/corpus/models/test1
> Opening File for "w":
> "/Users/alan/Projects/infomap/corpus/models/test1/artvec.bin"
> Opening File for "r":
> "/Users/alan/Projects/infomap/corpus/models/test1/numDocs"
> Reading the dictionary... Opening File for "r":
> "/Users/alan/Projects/infomap/corpus/models/test1/dic"
> Initializing row indices...Done.
> Allocating matrix memory...done.
> Initializing matrix...done.
> Opening File for "r":
> "/Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin"
> count_artvec.c: about to read 0 rows from wordvector file.
> Entering process_wordlist.
> About to call initialize_wordlist.
> Opening File for "r":
> "/Users/alan/Projects/infomap/corpus/models/test1/wordlist"
> Returned from initialize_wordlist.
> ..................................................
> Finishing target: /Users/alan/Projects/infomap/corpus/models/test1/artvec=
.bin
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
>
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
> Building target:
> /Users/alan/Projects/infomap/corpus/models/test1/model_params.txt
> Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/model_par=
ams.bin
> /Users/alan/Projects/infomap/corpus/models/test1/model_info.bin
> /Users/alan/Projects/infomap/corpus/models/test1/corpus_format.bin
> Sat Oct 1 15:49:49 BST 2005
> ..................................................
> write_text_params -mdir /Users/alan/Projects/infomap/corpus/models/test1
> ..................................................
> Finishing target:
> /Users/alan/Projects/infomap/corpus/models/test1/model_params.txt
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> infomap-nlp-users mailing list
> inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infomap-nlp-users
>
|
|
From: Alan J. S. <sa...@gm...> - 2005-10-02 22:14:48
|
Hi.
I'm using infomap 0.8.6 to build a matrix but nothing I do seems to work.
I've formatted the King James' Bible using the doc and text tags thus:
<doc>
<text>
The Old Testament of...
[...missing text...]
End of the Project Gutenberg Edition of the King James Bible
</text>
</doc>
and tried the infomap-build command thus:
:infomap-build -s /Users/alan/Projects/infomap/corpus/kjv10.txt test1
The output is below. However, when I run the associate command thus:
:associate -t -c test1 God
I get:
:No word/document vector for "God".
Or "GOD" or "god". I've tried this on OS X, Ubuntu and Mandrake Linux
all with the same results. I've checked the KJ Bible to ensure that no
errant tags are contained (they are not) and followed the examples in
the user manual to the letter and still nothing. I've also tried
changing the SVD_ITER value to 400 thus:
:infomap-build -D SVD_ITER=3D400 -s
/Users/alan/Projects/infomap/corpus/kjv10.txt test1
which took the new value into account, but again the associate command
doesn't work. Can anyone shed any light on what is going wrong or on
what I should be doing but don't yet know about?
Rgds,
Alan J. Salmoni.
This is the output of the infomap-build command (with SVD-ITER changed to 4=
00):
Sourcing param file "/usr/local/share/infomap-nlp/default-params"
Sourcing extra param file "/tmp/infomap-build.kyInQJ"
Contents are:
SVD_ITER=3D400
Removing extra param file
WORKING_DATA_DIR =3D "/Users/alan/Projects/infomap/corpus/models/test1"
CORPUS_DIR =3D "/Users/alan/Projects/infomap/corpus"
CORPUS_FILE =3D "/Users/alan/Projects/infomap/corpus/kjv10.txt"
FNAMES_FILE =3D ""
ROWS =3D "20000"
COLUMNS =3D "1000"
SINGVALS =3D "100"
SVD_ITER =3D "400"
PRE_CONTEXT_SIZE =3D "15"
POST_CONTEXT_SIZE =3D "15"
WRITE_MATLAB_FORMAT =3D "0"
VALID_CHARS_FILE =3D "/usr/local/share/infomap-nlp/valid_chars.en"
STOPLIST_FILE =3D "/usr/local/share/infomap-nlp/stop.list"
COL_LABELS_FROM_FILE =3D "0"
COL_LABEL_FILE =3D ""
echo "Making datadir"
Making datadir
mkdir -p /Users/alan/Projects/infomap/corpus/models/test1
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Building target: /Users/alan/Projects/infomap/corpus/models/test1/wordlist
Prerequisites: /Users/alan/Projects/infomap/corpus/kjv10.txt
Sat Oct 1 15:49:47 BST 2005
..................................................
prepare_corpus \
-cdir "/Users/alan/Projects/infomap/corpus" \
-mdir "/Users/alan/Projects/infomap/corpus/models/test1" \
-cfile "/Users/alan/Projects/infomap/corpus/kjv10.txt" \
-fnfile "" \
-chfile "/usr/local/share/infomap-nlp/valid_chars.en" \
-slfile "/usr/local/share/infomap-nlp/stop.list" \
-rptfile ""
Locale set to en_US.
Opening File for "r":
"/usr/local/share/infomap-nlp/valid_chars.en"
Opening File for "r":
""
my_fopen: No such file or directory
Opening File for "r":
"/usr/local/share/infomap-nlp/stop.list"
Opening File for "w":
"/Users/alan/Projects/infomap/corpus/models/test1/wordlist"
Opening File for "r":
"/Users/alan/Projects/infomap/corpus/kjv10.txt"
Opening File for "w":
"/Users/alan/Projects/infomap/corpus/models/test1/numDocs"
Typecount =3D 0
Preparing to sort ... Sorting ... Done.
Opening File for "w":
"/Users/alan/Projects/infomap/corpus/models/test1/dic"
..................................................
Finishing target: /Users/alan/Projects/infomap/corpus/models/test1/wordlist
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Building target: /Users/alan/Projects/infomap/corpus/models/test1/coll
Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/wordlist
/Users/alan/Projects/infomap/corpus/models/test1/dic
/Users/alan/Projects/infomap/corpus/models/test1/numDocs
Sat Oct 1 15:49:48 BST 2005
..................................................
count_wordvec \
-mdir /Users/alan/Projects/infomap/corpus/models/test1 \
-matlab 0 \
-precontext 15 \
-postcontext 15 \
-rows 20000 \
-columns 1000 \
-col_labels_from_file 0 \
-col_label_file ""
model data dir is "/Users/alan/Projects/infomap/corpus/models/test1".
count_wordvec.c: looking for 0 rows
which had better match 0
Reading the dictionary... Opening File for "r":
"/Users/alan/Projects/infomap/corpus/models/test1/dic"
Opening File for "r":
"/Users/alan/Projects/infomap/corpus/models/test1/numDocs"
Initializing row indices...Done.
Initializing column indices...Done.
Allocating matrix memory...done.
Initializing matrix...done.
model data dir is "/Users/alan/Projects/infomap/corpus/models/test1".
count_wordvec.c: about to call process_wordlist
Entering process_wordlist.
About to call initialize_wordlist.
Opening File for "r":
"/Users/alan/Projects/infomap/corpus/models/test1/wordlist"
Returned from initialize_wordlist.
Writing the co-occurrence matrix.
Entering write_matrix_svd; rows =3D 0 and columns =3D 1000.
Opening File for "w":
"/Users/alan/Projects/infomap/corpus/models/test1/coll"
Opening File for "w":
"/Users/alan/Projects/infomap/corpus/models/test1/indx"
..................................................
Finishing target: /Users/alan/Projects/infomap/corpus/models/test1/coll
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Building target: /Users/alan/Projects/infomap/corpus/models/test1/left
Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/coll
/Users/alan/Projects/infomap/corpus/models/test1/indx
Sat Oct 1 15:49:48 BST 2005
..................................................
cd /Users/alan/Projects/infomap/corpus/models/test1 && rm -f svd_diag left =
\
rght sing
cd /Users/alan/Projects/infomap/corpus/models/test1 && svdinterface \
-singvals 100 \
-iter 400
This is svdinterface.
Writing to: left
Writing to: rght
Writing to: sing
Writing to: svd_diag
Reading: indx
Reading: indx
Reading: coll
FEWER THAN EXPECTED SINGULAR VALUES
..................................................
Finishing target: /Users/alan/Projects/infomap/corpus/models/test1/left
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Building target: /Users/alan/Projects/infomap/corpus/models/test1/wordvec.b=
in
Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/left
/Users/alan/Projects/infomap/corpus/models/test1/dic
Sat Oct 1 15:49:49 BST 2005
..................................................
encode_wordvec \
-m /Users/alan/Projects/infomap/corpus/models/test1
Opening File for "r":
"/Users/alan/Projects/infomap/corpus/models/test1/left"
Opening File for "w":
"/Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin"
Reading the dictionary...
Opening File for "r":
"/Users/alan/Projects/infomap/corpus/models/test1/dic"
Initializing row indices...Done.
..................................................
Finishing target: /Users/alan/Projects/infomap/corpus/models/test1/wordvec.=
bin
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Building target: /Users/alan/Projects/infomap/corpus/models/test1/artvec.bi=
n
Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin
/Users/alan/Projects/infomap/corpus/models/test1/wordlist
/Users/alan/Projects/infomap/corpus/models/test1/dic
/Users/alan/Projects/infomap/corpus/models/test1/numDocs
Sat Oct 1 15:49:49 BST 2005
..................................................
count_artvec -m /Users/alan/Projects/infomap/corpus/models/test1
Opening File for "w":
"/Users/alan/Projects/infomap/corpus/models/test1/artvec.bin"
Opening File for "r":
"/Users/alan/Projects/infomap/corpus/models/test1/numDocs"
Reading the dictionary... Opening File for "r":
"/Users/alan/Projects/infomap/corpus/models/test1/dic"
Initializing row indices...Done.
Allocating matrix memory...done.
Initializing matrix...done.
Opening File for "r":
"/Users/alan/Projects/infomap/corpus/models/test1/wordvec.bin"
count_artvec.c: about to read 0 rows from wordvector file.
Entering process_wordlist.
About to call initialize_wordlist.
Opening File for "r":
"/Users/alan/Projects/infomap/corpus/models/test1/wordlist"
Returned from initialize_wordlist.
..................................................
Finishing target: /Users/alan/Projects/infomap/corpus/models/test1/artvec.b=
in
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Building target:
/Users/alan/Projects/infomap/corpus/models/test1/model_params.txt
Prerequisites: /Users/alan/Projects/infomap/corpus/models/test1/model_param=
s.bin
/Users/alan/Projects/infomap/corpus/models/test1/model_info.bin
/Users/alan/Projects/infomap/corpus/models/test1/corpus_format.bin
Sat Oct 1 15:49:49 BST 2005
..................................................
write_text_params -mdir /Users/alan/Projects/infomap/corpus/models/test1
..................................................
Finishing target:
/Users/alan/Projects/infomap/corpus/models/test1/model_params.txt
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
|
|
From: Dominic W. <wi...@ma...> - 2005-08-22 01:08:22
|
Dear Infomap Users, We are pleased to announce the long-awaited (even if not much-awaited!) version 0.8.6 of the Infomap NLP software. The main changes from version 0.8.5 are greater flexibility to handle different character sets (thanks to Viktor Tron), and a few bug fixes and enhancements that should improve overall performance and stability. In more detail, the changes are as follows: - To extend the coverage of different character sets, infomap-build now reads a "valid chars" file, which contains all the characters to be treated as alphabetic by this model. There is a default "valid_chars.en" file for English shipped with the release. - The vector output functionality (associate -f FILENAME) has been fixed, and so the option to output search results to a file should work now. - In the svd interface, myutils.c has been changed so that the memory allocation works for 32 and 64 bit machines. - The default admin/install.control file has been altered so that infomap-install works automatically for more db formats (even though this means it gives more error warnings as well). The version has been tested for single-file corpora on Linux, Solaris, BSD / OSX, and Windows / Cygwin platforms. infomap-build works on all of these, infomap-install still appeasr to have problems on Solaris. There are several further enhancements that we would like to make, in particular, adding the web interface and some of the other tools that are described in Infomap papers and used in the online demos, but never made it into the release. Also, it would be great to build in some of the GUI developments that other list members have made. Hopefully, getting a new more stable release out there will be a big step in the right direction, especially since it's forced Scott and myself to get to grips once more with autoconf, automake, and some of the other gruesome incantations that go into releasing new versions of software. Finally, I'm generally much more responsive to this MAYA e-mail address (wi...@ma...), so those of you who have my CSLI old address in your address books may wish to update. Much graitude to everyone who has helped find bugs, fix bugs, and keep the Infomap software alive and circulating and useful to people. Just from my own contacts, we now have users from as far afield as medical informatics, the intelligence community, and cultural anthropology, as well as a good bunch of computational linguistics stalwarts. Thanks, folks. Best wishes, Dominic |
|
From: Scott C. <ced...@gm...> - 2005-06-30 03:21:03
|
Hi Robert,
On 6/29/05, Robert Batesole <ha...@be...> wrote:
> Thanks for such a quick response. The faster I can get Infomap working i=
n
> Cygwin, the sooner I can get my portion of the project underway! *grin*
>=20
Well, I hope you can get there soon.
> The contents of my search/.deps are as follows:
>=20
> 06/29/2005 04:20p 2,741 arrayoutput.Po
> 06/29/2005 04:20p 2,822 associate.Po
> 06/29/2005 04:20p 2,581 files.Po
> 06/29/2005 04:20p 2,576 list.Po
> 06/29/2005 04:20p 2,087 model_params.Po
> 06/29/2005 04:20p 2,763 neighbors.Po
> 06/29/2005 04:20p 2,822 print_doc.Po
> 06/29/2005 04:20p 2,723 query.Po
> 06/29/2005 04:20p 1,961 search_utils.Po
> 06/29/2005 04:20p 1,359 toc.Po
> 06/29/2005 04:20p 2,607 utils.Po
>=20
So you seem to have everything you need. The question is why make
can't find these files.
> [Robert@robbiebee /tmp/infomap-nlp-0.8.5]$ make
> Makefile:200: arrayoutput.Po: No such file or directory
> Makefile:201: associate.Po: No such file or directory
> Makefile:202: files.Po: No such file or directory
> Makefile:203: list.Po: No such file or directory
> Makefile:204: model_params.Po: No such file or directory
> Makefile:205: neighbors.Po: No such file or directory
> Makefile:206: print_doc.Po: No such file or directory
> Makefile:207: query.Po: No such file or directory
> Makefile:208: search_utils.Po: No such file or directory
> Makefile:209: toc.Po: No such file or directory
> Makefile:210: utils.Po: No such file or directory
So the problem is at lines 200--210 of search/Makefile. What do these
lines say? Actually, could you send me that entire Makefile?
If the lines in question (or any other lines that refer to the *.Po
files; you should try searching through the file to find them) refer
to these files as being in a location *other* than search/.deps/, you
should change them.
In my makefile, for instance, there are lines that say
include ./$(DEPDIR)arrayoutput.Po
include ./$(DEPDIR)/associate.Po
include ./$(DEPDIR)/files.Po
etc. $(DEPDIR) is defined elsewhere in the file:
DEPDIR =3D .deps
My best guess at this point is that the "configure" system has somehow
incorrectly defined one or the other of these items in your Makefile
(or a similar problem has happened somewhere else in the file).
So please send me your Makefile and I'll take a look at it, but that's
the kind of problem I recommend you try to track down and fix in the
meantime.
=20
Scott
|
|
From: Robert B. <ha...@be...> - 2005-06-30 00:56:00
|
Thanks for such a quick response. The faster I can get Infomap working in
Cygwin, the sooner I can get my portion of the project underway! *grin*
The contents of my search/.deps are as follows:
06/29/2005 04:20p 2,741 arrayoutput.Po
06/29/2005 04:20p 2,822 associate.Po
06/29/2005 04:20p 2,581 files.Po
06/29/2005 04:20p 2,576 list.Po
06/29/2005 04:20p 2,087 model_params.Po
06/29/2005 04:20p 2,763 neighbors.Po
06/29/2005 04:20p 2,822 print_doc.Po
06/29/2005 04:20p 2,723 query.Po
06/29/2005 04:20p 1,961 search_utils.Po
06/29/2005 04:20p 1,359 toc.Po
06/29/2005 04:20p 2,607 utils.Po
The output of the "make clean" is as follows:
[Robert@robbiebee /tmp/infomap-nlp-0.8.5]$ make clean
Making clean in doc
make[1]: Entering directory `/tmp/infomap-nlp-0.8.5/doc'
Making clean in html
make[2]: Entering directory `/tmp/infomap-nlp-0.8.5/doc/html'
Making clean in tutorial
make[3]: Entering directory `/tmp/infomap-nlp-0.8.5/doc/html/tutorial'
make[3]: Nothing to be done for `clean'.
make[3]: Leaving directory `/tmp/infomap-nlp-0.8.5/doc/html/tutorial'
Making clean in .
make[3]: Entering directory `/tmp/infomap-nlp-0.8.5/doc/html'
make[3]: Nothing to be done for `clean-am'.
make[3]: Leaving directory `/tmp/infomap-nlp-0.8.5/doc/html'
make[2]: Leaving directory `/tmp/infomap-nlp-0.8.5/doc/html'
Making clean in man
make[2]: Entering directory `/tmp/infomap-nlp-0.8.5/doc/man'
Making clean in man5
make[3]: Entering directory `/tmp/infomap-nlp-0.8.5/doc/man/man5'
make[3]: Nothing to be done for `clean'.
make[3]: Leaving directory `/tmp/infomap-nlp-0.8.5/doc/man/man5'
Making clean in man1
make[3]: Entering directory `/tmp/infomap-nlp-0.8.5/doc/man/man1'
make[3]: Nothing to be done for `clean'.
make[3]: Leaving directory `/tmp/infomap-nlp-0.8.5/doc/man/man1'
Making clean in .
make[3]: Entering directory `/tmp/infomap-nlp-0.8.5/doc/man'
make[3]: Nothing to be done for `clean-am'.
make[3]: Leaving directory `/tmp/infomap-nlp-0.8.5/doc/man'
make[2]: Leaving directory `/tmp/infomap-nlp-0.8.5/doc/man'
Making clean in .
make[2]: Entering directory `/tmp/infomap-nlp-0.8.5/doc'
make[2]: Nothing to be done for `clean-am'.
make[2]: Leaving directory `/tmp/infomap-nlp-0.8.5/doc'
make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5/doc'
Making clean in svd
make[1]: Entering directory `/tmp/infomap-nlp-0.8.5/svd'
Making clean in svdinterface
make[2]: Entering directory `/tmp/infomap-nlp-0.8.5/svd/svdinterface'
Makefile:182: las2.Po: No such file or directory
Makefile:183: model_params.Po: No such file or directory
Makefile:184: myutils.Po: No such file or directory
Makefile:185: svdinterface.Po: No such file or directory
make[2]: *** No rule to make target `svdinterface.Po'. Stop.
make[2]: Leaving directory `/tmp/infomap-nlp-0.8.5/svd/svdinterface'
make[1]: *** [clean-recursive] Error 1
make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5/svd'
make: *** [clean-recursive] Error 1
[Robert@robbiebee /tmp/infomap-nlp-0.8.5]$ make
Making all in .
make[1]: Entering directory `/tmp/infomap-nlp-0.8.5'
make[1]: Nothing to be done for `all-am'.
make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5'
Making all in admin
make[1]: Entering directory `/tmp/infomap-nlp-0.8.5/admin'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5/admin'
Making all in lib
make[1]: Entering directory `/tmp/infomap-nlp-0.8.5/lib'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5/lib'
Making all in search
make[1]: Entering directory `/tmp/infomap-nlp-0.8.5/search'
Makefile:200: arrayoutput.Po: No such file or directory
Makefile:201: associate.Po: No such file or directory
Makefile:202: files.Po: No such file or directory
Makefile:203: list.Po: No such file or directory
Makefile:204: model_params.Po: No such file or directory
Makefile:205: neighbors.Po: No such file or directory
Makefile:206: print_doc.Po: No such file or directory
Makefile:207: query.Po: No such file or directory
Makefile:208: search_utils.Po: No such file or directory
Makefile:209: toc.Po: No such file or directory
Makefile:210: utils.Po: No such file or directory
make[1]: *** No rule to make target `utils.Po'. Stop.
make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5/search'
make: *** [all-recursive] Error 1
[Robert@robbiebee /tmp/infomap-nlp-0.8.5]$
----- Original Message -----
From: "Scott Cederberg" <ced...@gm...>
To: "Robert Batesole" <ha...@be...>
Cc: <inf...@li...>
Sent: Wednesday, June 29, 2005 4:58 PM
Subject: Re: [infomap-nlp-users] InfoMap and Cygwin Compiling Problems
Hi Robert,
On 6/29/05, Robert Batesole <ha...@be...> wrote:
> However, your original instructions identified the declaration on
> line 167 of myutils.c at the beginning of the myutils() function, but line
> 167 falls under the mymalloc() function, which I assumed was the reference
> you were indicating.
You're right; sorry for the mistake
>
> I removed the declaration "char *malloc();"
That's right.
> Makefile:200: arrayoutput.Po: No such file or directory
> Makefile:201: associate.Po: No such file or directory
> Makefile:202: files.Po: No such file or directory
> Makefile:203: list.Po: No such file or directory
> Makefile:204: model_params.Po: No such file or directory
> Makefile:205: neighbors.Po: No such file or directory
> Makefile:206: print_doc.Po: No such file or directory
> Makefile:207: query.Po: No such file or directory
> Makefile:208: search_utils.Po: No such file or directory
> Makefile:209: toc.Po: No such file or directory
> Makefile:210: utils.Po: No such file or directory
> make[1]: *** No rule to make target `utils.Po'. Stop.
> make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5/search'
> make: *** [all-recursive] Error 1
>
Hmm... these .Po files shouldn't have to be built; they are part of
the distribution package. Can you tell me whether you have a
search/.deps directory, and if so what its contents are? The .Po
files should live in .deps subdirectories of search/, preprocessing/,
svd/svdinterface/, etc.
Scott
|
|
From: Scott C. <ced...@gm...> - 2005-06-29 22:07:13
|
One thing that may be worth trying is running "make clean" and then trying "make" again. |
|
From: Scott C. <ced...@gm...> - 2005-06-29 21:59:05
|
Hi Robert,
On 6/29/05, Robert Batesole <ha...@be...> wrote:
> However, your original instructions identified the declaration on
> line 167 of myutils.c at the beginning of the myutils() function, but lin=
e
> 167 falls under the mymalloc() function, which I assumed was the referenc=
e
> you were indicating.
You're right; sorry for the mistake
>=20
> I removed the declaration "char *malloc();"
That's right.
> Makefile:200: arrayoutput.Po: No such file or directory
> Makefile:201: associate.Po: No such file or directory
> Makefile:202: files.Po: No such file or directory
> Makefile:203: list.Po: No such file or directory
> Makefile:204: model_params.Po: No such file or directory
> Makefile:205: neighbors.Po: No such file or directory
> Makefile:206: print_doc.Po: No such file or directory
> Makefile:207: query.Po: No such file or directory
> Makefile:208: search_utils.Po: No such file or directory
> Makefile:209: toc.Po: No such file or directory
> Makefile:210: utils.Po: No such file or directory
> make[1]: *** No rule to make target `utils.Po'. Stop.
> make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5/search'
> make: *** [all-recursive] Error 1
>
Hmm... these .Po files shouldn't have to be built; they are part of
the distribution package. Can you tell me whether you have a
search/.deps directory, and if so what its contents are? The .Po
files should live in .deps subdirectories of search/, preprocessing/,
svd/svdinterface/, etc.
=20
Scott
|
|
From: Robert B. <ha...@be...> - 2005-06-29 21:40:16
|
Good afternoon Scott,
Due to limited experience with the UNIX environment, I decided to manually
delete the line containing the malloc declaration as opposed to the CVS
route. However, your original instructions identified the declaration on
line 167 of myutils.c at the beginning of the myutils() function, but line
167 falls under the mymalloc() function, which I assumed was the reference
you were indicating.
I removed the declaration "char *malloc();" from myutils.c and reattempted
the "./configure" and "make" commands, but received error messages when
executing the "make" command. The following text below is the output of both
commands. Any assistance you can provide is appreciated!
-Robert Batesole
Robert@robbiebee /tmp/infomap-nlp-0.8.5
$ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking for C compiler default output file name... a.exe
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... .exe
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking for a BSD-compatible install... /usr/bin/install -c
checking whether make sets $(MAKE)... (cached) yes
checking for sqrt in -lm... yes
checking for dbm_open in -lgdbm... no
checking for dbm_open in -lgdbm_compat... yes
checking for egrep... grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking locale.h usability... yes
checking locale.h presence... yes
checking for locale.h... yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking sys/file.h usability... yes
checking sys/file.h presence... yes
checking for sys/file.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for unistd.h... (cached) yes
checking ndbm.h usability... yes
checking ndbm.h presence... yes
checking for ndbm.h... yes
checking for an ANSI C-conforming const... yes
checking for off_t... yes
checking for size_t... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible malloc... yes
checking whether lstat dereferences a symlink specified with a trailing
slash...
yes
checking whether stat accepts an empty string... no
checking for getcwd... yes
checking for memset... yes
checking for setlocale... yes
checking for sqrt... yes
checking for strchr... yes
checking for strdup... yes
checking for strstr... yes
checking for dbm_open... yes
checking for dbm_close... yes
checking for dbm_fetch... yes
checking for dbm_store... yes
configure: creating ./config.status
./config.status: line 305: NLP config.status 0.8.5
configured by ./configure, generated by GNU Autoconf 2.59,
with options ""
Copyright (C) 2003 Free Software Foundation, Inc.
This config.status script is free software; the Free Software Foundation
gives unlimited permission to copy, distribute and modify it.: No such file
or d
irectory
config.status: creating Makefile
config.status: creating preprocessing/Makefile
config.status: creating search/Makefile
config.status: creating svd/Makefile
config.status: creating svd/svdinterface/Makefile
config.status: creating lib/Makefile
config.status: creating admin/Makefile
config.status: creating admin/infomap-build
config.status: creating admin/infomap-install
config.status: creating admin/Makefile.data
config.status: creating admin/default-params
config.status: creating doc/Makefile
config.status: creating doc/man/Makefile
config.status: creating doc/man/man1/Makefile
config.status: creating doc/man/man5/Makefile
config.status: creating doc/html/Makefile
config.status: creating doc/html/tutorial/Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
Robert@robbiebee /tmp/infomap-nlp-0.8.5
$ make
Making all in .
make[1]: Entering directory `/tmp/infomap-nlp-0.8.5'
make[1]: Nothing to be done for `all-am'.
make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5'
Making all in admin
make[1]: Entering directory `/tmp/infomap-nlp-0.8.5/admin'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5/admin'
Making all in lib
make[1]: Entering directory `/tmp/infomap-nlp-0.8.5/lib'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5/lib'
Making all in search
make[1]: Entering directory `/tmp/infomap-nlp-0.8.5/search'
Makefile:200: arrayoutput.Po: No such file or directory
Makefile:201: associate.Po: No such file or directory
Makefile:202: files.Po: No such file or directory
Makefile:203: list.Po: No such file or directory
Makefile:204: model_params.Po: No such file or directory
Makefile:205: neighbors.Po: No such file or directory
Makefile:206: print_doc.Po: No such file or directory
Makefile:207: query.Po: No such file or directory
Makefile:208: search_utils.Po: No such file or directory
Makefile:209: toc.Po: No such file or directory
Makefile:210: utils.Po: No such file or directory
make[1]: *** No rule to make target `utils.Po'. Stop.
make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5/search'
make: *** [all-recursive] Error 1
Robert@robbiebee /tmp/infomap-nlp-0.8.5
$
----- Original Message -----
From: "Scott Cederberg" <ced...@gm...>
To: "Robert Batesole" <ha...@be...>
Cc: <inf...@li...>
Sent: Tuesday, June 28, 2005 9:25 AM
Subject: Re: [infomap-nlp-users] InfoMap and Cygwin Compiling Problems
Hi Robert,
This is a bug that has been fixed in the CVS version of the
software. We're planning to release the fixed version shortly, but
until then you have two options:
1. Check out the latest CVS version and use that. (You can find
instructions on how to obtain this version by following the "CVS
Repository" link near the bottom of the Infomap SourceForge project
info page (http://sourceforge.net/projects/infomap-nlp/) and reading
the section entitled "Anonymous CVS Access".)
2. Edit the svd/svdinterface/myutils.c file. Go to line 167 in
myutils.c, and remove the obsolete line where "malloc" is declared,
right at the beginning of the myutils() function. After that
everything should work.
Please let me know if this works out for you. I'll try to get
the new release out by the end of the week, if you'd rather wait for
that.
Scott
|
|
From: Scott C. <ced...@gm...> - 2005-06-28 14:25:24
|
Hi Robert,
This is a bug that has been fixed in the CVS version of the
software. We're planning to release the fixed version shortly, but
until then you have two options:
1. Check out the latest CVS version and use that. (You can find
instructions on how to obtain this version by following the "CVS
Repository" link near the bottom of the Infomap SourceForge project
info page (http://sourceforge.net/projects/infomap-nlp/) and reading
the section entitled "Anonymous CVS Access".)
2. Edit the svd/svdinterface/myutils.c file. Go to line 167 in
myutils.c, and remove the obsolete line where "malloc" is declared,
right at the beginning of the myutils() function. After that
everything should work.
Please let me know if this works out for you. I'll try to get
the new release out by the end of the week, if you'd rather wait for
that.
=20
Scott
On 6/27/05, Robert Batesole <ha...@be...> wrote:
> Making all in svdinterface=20
>=20
> myutils.c: In function `mymalloc':=20
>=20
> myutils.c:167: error: conflicting types for 'malloc'=20
>=20
> make[3]: *** [myutils.o] Error 1=20
>=20
> make[3]: Leaving directory `/tmp/infomap-nlp-0.8.5/svd/svdinterface'=20
>
|
|
From: Robert B. <ha...@be...> - 2005-06-28 00:31:20
|
Good evening. My name is Robert Batesole and I am working on a senior = project for the Univ. of Southern Mississippi. I've been working with = the InfoMap software, attempting to correctly compile and run the = software under a Cygwin environment, but have encountered difficulties = when running the 'make' program after './configure'. Below is the output produced by both the make and ./configure files. = Any chance the problem can be identified? Thanks! -Robert Batesole OUTPUT OF 'MAKE' AND './CONFIGURE': Robert@robbiebee /tmp/infomap-nlp-0.8.5 $ ./configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking for C compiler default output file name... a.exe checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... .exe checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ANSI C... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking how to run the C preprocessor... gcc -E checking for a BSD-compatible install... /usr/bin/install -c checking whether make sets $(MAKE)... (cached) yes checking for sqrt in -lm... yes checking for dbm_open in -lgdbm... no checking for dbm_open in -lgdbm_compat... yes checking for egrep... grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking fcntl.h usability... yes checking fcntl.h presence... yes checking for fcntl.h... yes checking locale.h usability... yes checking locale.h presence... yes checking for locale.h... yes checking for stdlib.h... (cached) yes checking for string.h... (cached) yes checking sys/file.h usability... yes checking sys/file.h presence... yes checking for sys/file.h... yes checking sys/time.h usability... yes checking sys/time.h presence... yes checking for sys/time.h... yes checking for unistd.h... (cached) yes checking ndbm.h usability... yes checking ndbm.h presence... yes checking for ndbm.h... yes checking for an ANSI C-conforming const... yes checking for off_t... yes checking for size_t... yes checking for stdlib.h... (cached) yes checking for GNU libc compatible malloc... yes checking whether lstat dereferences a symlink specified with a trailing = slash... yes checking whether stat accepts an empty string... no checking for getcwd... yes checking for memset... yes checking for setlocale... yes checking for sqrt... yes checking for strchr... yes checking for strdup... yes checking for strstr... yes checking for dbm_open... yes checking for dbm_close... yes checking for dbm_fetch... yes checking for dbm_store... yes configure: creating ./config.status ./config.status: line 305: NLP config.status 0.8.5 configured by ./configure, generated by GNU Autoconf 2.59, with options "" =20 Copyright (C) 2003 Free Software Foundation, Inc. This config.status script is free software; the Free Software Foundation gives unlimited permission to copy, distribute and modify it.: No such = file or d irectory config.status: creating Makefile config.status: creating preprocessing/Makefile config.status: creating search/Makefile config.status: creating svd/Makefile config.status: creating svd/svdinterface/Makefile config.status: creating lib/Makefile config.status: creating admin/Makefile config.status: creating admin/infomap-build config.status: creating admin/infomap-install config.status: creating admin/Makefile.data config.status: creating admin/default-params config.status: creating doc/Makefile config.status: creating doc/man/Makefile config.status: creating doc/man/man1/Makefile config.status: creating doc/man/man5/Makefile config.status: creating doc/html/Makefile config.status: creating doc/html/tutorial/Makefile config.status: creating config.h config.status: config.h is unchanged config.status: executing depfiles commands =20 Robert@robbiebee /tmp/infomap-nlp-0.8.5 $ make make all-recursive make[1]: Entering directory `/tmp/infomap-nlp-0.8.5' Making all in . make[2]: Entering directory `/tmp/infomap-nlp-0.8.5' make[2]: Leaving directory `/tmp/infomap-nlp-0.8.5' Making all in admin make[2]: Entering directory `/tmp/infomap-nlp-0.8.5/admin' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/tmp/infomap-nlp-0.8.5/admin' Making all in lib make[2]: Entering directory `/tmp/infomap-nlp-0.8.5/lib' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/tmp/infomap-nlp-0.8.5/lib' Making all in search make[2]: Entering directory `/tmp/infomap-nlp-0.8.5/search' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/tmp/infomap-nlp-0.8.5/search' Making all in preprocessing make[2]: Entering directory `/tmp/infomap-nlp-0.8.5/preprocessing' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/tmp/infomap-nlp-0.8.5/preprocessing' Making all in svd make[2]: Entering directory `/tmp/infomap-nlp-0.8.5/svd' Making all in svdinterface make[3]: Entering directory `/tmp/infomap-nlp-0.8.5/svd/svdinterface' if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -I../../lib -I../../admin -g = -O2 -MT m yutils.o -MD -MP -MF ".deps/myutils.Tpo" -c -o myutils.o myutils.c; \ then mv -f ".deps/myutils.Tpo" ".deps/myutils.Po"; else rm -f = ".deps/myutils.Tpo "; exit 1; fi myutils.c: In function `mymalloc': myutils.c:167: error: conflicting types for 'malloc' make[3]: *** [myutils.o] Error 1 make[3]: Leaving directory `/tmp/infomap-nlp-0.8.5/svd/svdinterface' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/tmp/infomap-nlp-0.8.5/svd' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/tmp/infomap-nlp-0.8.5' make: *** [all] Error 2 =20 Robert@robbiebee /tmp/infomap-nlp-0.8.5 |
|
From: Dominic W. <dwi...@cs...> - 2005-06-02 15:09:12
|
> Thanks very much for your bug report! I fixed the problem, committed the > new code and the "-f" option of associate should now write the correct > word vectors to the file specified. That's great, Beate, thanks. Scott and I were working on this last night and failed to figure it out, but your fix works fine for me. I made the 32 / 64 bit malloc fix as well, and committed that a few days ago. I'm still haveing trouble with the infoma-install program, like a few of the others seem to be doing. We should make a new release soon with these fixes, though. Thanks a lot, Dominic |
|
From: Beate D. <do...@im...> - 2005-06-02 08:20:24
|
Hi!
Thanks very much for your bug report! I fixed the problem, committed the
new code and the "-f" option of associate should now write the correct
word vectors to the file specified.
Best,
Beate
On Tue, 31 May 2005, Linuxer Wang wrote:
> Hello, all
>
> I read part of the source codes, I have a problem with function
> find_neighbors() in file neighbors.c.
>
> I marked the problems lines as @1 and @2 as follows:
>
> if( neighbor_item.score > threshold) {
> if ( (neighbor_item.vector = malloc(vector_size)) == NULL ) { <---------@1
> fprintf( stderr, "neighbors.c: can't allocate vector memory.\n" );
> free_tail( list, neighbor_item_free );
> return 0;
> }
> list_insert( list, &last, depth, <---------@2
> (void *) &neighbor_item, sizeof( NEIGHBOR_ITEM ),
> neighbor_item_cmp, neighbor_item_free );
>
> list_length++;
>
> /* Set the threshold to the lowest value in the list */
> if( (last != NULL) && (list_length >= depth))
> threshold = ((NEIGHBOR_ITEM *) (last->data))->score;
> }
>
> Note that, @1 clears the vector of neighbor_item, so when @2 tries to
> insert neighbor_item to the list, the vector of neighbor_item is already
> zero array.
> So the problem is that the order of @1 and @2 should be conversed. The
> current codes cause the output to file contains only zeros.
>
> Can anyone verify it?
> Yours,
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by Yahoo.
> Introducing Yahoo! Search Developer Network - Create apps using Yahoo!
> Search APIs Find out how you can build Yahoo! directly into your own
> Applications - visit http://developer.yahoo.net/?fr=offad-ysdn-ostg-q22005
> _______________________________________________
> infomap-nlp-devel mailing list
> inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infomap-nlp-devel
>
|
|
From: Mike W. <mf...@ma...> - 2005-05-31 21:24:11
|
Hi, I realized that gdbms needed to be compiled in ndbm mode (you should probably include that in the docs). Regardless, I found that I couldn't run autoconf with gcc 3.4 anyways. |
|
From: Mike W. <mf...@ma...> - 2005-05-31 20:51:31
|
Hi, I can't get infomap to see my gdbm installation on my Mandrake Linux
10.1 box.
Here's the configuration output:
checking for dbm_open... no
checking for dbm_close... no
checking for dbm_fetch... no
checking for dbm_store... no
configure: WARNING: configure could not find all of the necessary DBM
functions.
(dbm_open, dbm_close, dbm_fetch, dbm_store). This probably
means that configure could not find your DBM library, or
that you do not have a DBM library installed. (See above
output for the results of library checks.) Please install
a DBM-compatible library (such as GNU DBM) or write to
inf...@li... for advice.
configure: error: One or more errors found (see output above for details).
Configuration unsuccessful.
I have the latest gdbm installed from the gnu site, and it doesn't seem
to matter.
One problem I've noticed is that the symbols inside the gdbm .so's are
gdbm_open, gdbm_close, etc...
Anyways, any help you could give me would be appreciated.
|
|
From: Linuxer W. <lin...@gm...> - 2005-05-31 18:25:36
|
Hello, all
I read part of the source codes, I have a problem with function
find_neighbors() in file neighbors.c.
I marked the problems lines as @1 and @2 as follows:
if( neighbor_item.score > threshold) {
if ( (neighbor_item.vector = malloc(vector_size)) == NULL ) { <---------@1
fprintf( stderr, "neighbors.c: can't allocate vector memory.\n" );
free_tail( list, neighbor_item_free );
return 0;
}
list_insert( list, &last, depth, <---------@2
(void *) &neighbor_item, sizeof( NEIGHBOR_ITEM ),
neighbor_item_cmp, neighbor_item_free );
list_length++;
/* Set the threshold to the lowest value in the list */
if( (last != NULL) && (list_length >= depth))
threshold = ((NEIGHBOR_ITEM *) (last->data))->score;
}
Note that, @1 clears the vector of neighbor_item, so when @2 tries to
insert neighbor_item to the list, the vector of neighbor_item is already
zero array.
So the problem is that the order of @1 and @2 should be conversed. The
current codes cause the output to file contains only zeros.
Can anyone verify it?
Yours,
|
|
From: Rahul J. <rj...@ya...> - 2005-05-23 17:15:45
|
Hi Michiel, I have tried building the many type model and it works as expected. According to the Infomap documentation, the installed models is the right way to save models and that way some temperory files which are no longer needed (but were needed during the build phase) will be removed. This was the reason I thought I should try to install models although for all practical purposes building the models works for me. Any more information will be appreciated... Thanks, Rahul. --- Spapé, Michiel <MS...@FS...> wrote: > Hi Rahul, > > Finally someone with a similar setup! I am also > running Infomap on a winxp machine using cygwin, but > somehow, I don't see much of a resemblance further > than that. However, I can say: > > - I have never been able to succesfully build 'many' > type models, although the single (tagged) corpusfile > works quite well. > - have you tried just using infomap-build instead of > install? I have yet to understand why anyone would > 'install' a model, instead of just building it; if > you want to have it saved more securely, why not > just copy the model to another folder? > - If you have it installed, maybe you would like to > have a look at my little front-end that also > incorporates infomap... > > Cheers, > > Mich > > > -----Original Message----- > From: inf...@li... > [mailto:inf...@li...] > On Behalf Of Rahul Joshi > Sent: zondag 22 mei 2005 4:05 > To: inf...@li... > Subject: [infomap-nlp-users] "infomap-install" Issue > > Hi, > > I have successfully built models of 'many' type. But > I want the models to be persistent. So I am trying > to install the models. > > I am using cygwin on Windows XP with the > INFOMAP_WORKING_DIR and INFOAMP_MODEL_PATH correctly > set. > > On running infomap-install, I get the following but > no models are created at the expected location. > > ************************** > > rahul@myPC /cygdrive/c > $ infomap-install myModel > Using install control file > "/usr/local/share/infomap-nlp/install.control" > > ************************** > > Can you please tell me if I am doing any mistake? > > Thanks, > Rahul. > > > > __________________________________ > Yahoo! Mail Mobile > Take Yahoo! Mail with you! Check email on your > mobile phone. > http://mobile.yahoo.com/learn/mail > > > ------------------------------------------------------- > This SF.Net email is sponsored by Oracle Space > Sweepstakes Want to be the first software developer > in space? > Enter now for the Oracle Space Sweepstakes! > http://ads.osdn.com/?ad_id=7412&alloc_id=16344&op=click > _______________________________________________ > infomap-nlp-users mailing list > inf...@li... > https://lists.sourceforge.net/lists/listinfo/infomap-nlp-users > > ********************************************************************** > This email and any files transmitted with it are > confidential and > intended solely for the use of the individual or > entity to whom they > are addressed. If you have received this email in > error please notify > the system manager. > ********************************************************************** > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by Oracle Space > Sweepstakes > Want to be the first software developer in space? > Enter now for the Oracle Space Sweepstakes! > http://ads.osdn.com/?ad_idt12&alloc_id344&op=click > _______________________________________________ > infomap-nlp-users mailing list > inf...@li... > https://lists.sourceforge.net/lists/listinfo/infomap-nlp-users > Discover Yahoo! Find restaurants, movies, travel and more fun for the weekend. Check it out! http://discover.yahoo.com/weekend.html |
|
From: <MS...@FS...> - 2005-05-23 09:23:31
|
Hi Rahul, Finally someone with a similar setup! I am also running Infomap on a winxp = machine using cygwin, but somehow, I don't see much of a resemblance furthe= r than that. However, I can say: - I have never been able to succesfully build 'many' type models, although = the single (tagged) corpusfile works quite well. - have you tried just using infomap-build instead of install? I have yet to= understand why anyone would 'install' a model, instead of just building it= ; if you want to have it saved more securely, why not just copy the model t= o another folder? - If you have it installed, maybe you would like to have a look at my littl= e front-end that also incorporates infomap... Cheers, Mich -----Original Message----- From: inf...@li... [mailto:infomap-nlp-use= rs-...@li...] On Behalf Of Rahul Joshi Sent: zondag 22 mei 2005 4:05 To: inf...@li... Subject: [infomap-nlp-users] "infomap-install" Issue Hi,=20 I have successfully built models of 'many' type. But I want the models to b= e persistent. So I am trying to install the models.=20 I am using cygwin on Windows XP with the INFOMAP_WORKING_DIR and INFOAMP_MO= DEL_PATH correctly set.=20 On running infomap-install, I get the following but no models are created a= t the expected location. ************************** rahul@myPC /cygdrive/c $ infomap-install myModel Using install control file "/usr/local/share/infomap-nlp/install.control" ************************** Can you please tell me if I am doing any mistake?=20 Thanks, Rahul. =09 __________________________________ Yahoo! Mail Mobile Take Yahoo! Mail with you! Check email on your mobile phone.=20 http://mobile.yahoo.com/learn/mail=20 ------------------------------------------------------- This SF.Net email is sponsored by Oracle Space Sweepstakes Want to be the f= irst software developer in space? Enter now for the Oracle Space Sweepstakes! http://ads.osdn.com/?ad_id=3D7412&alloc_id=3D16344&op=3Dclick _______________________________________________ infomap-nlp-users mailing list inf...@li... https://lists.sourceforge.net/lists/listinfo/infomap-nlp-users ********************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. ********************************************************************** |
|
From: Rahul J. <rj...@ya...> - 2005-05-22 02:05:41
|
Hi, I have successfully built models of 'many' type. But I want the models to be persistent. So I am trying to install the models. I am using cygwin on Windows XP with the INFOMAP_WORKING_DIR and INFOAMP_MODEL_PATH correctly set. On running infomap-install, I get the following but no models are created at the expected location. ************************** rahul@myPC /cygdrive/c $ infomap-install myModel Using install control file "/usr/local/share/infomap-nlp/install.control" ************************** Can you please tell me if I am doing any mistake? Thanks, Rahul. __________________________________ Yahoo! Mail Mobile Take Yahoo! Mail with you! Check email on your mobile phone. http://mobile.yahoo.com/learn/mail |
|
From: <lei...@gm...> - 2005-05-20 07:13:06
|
Hi,
maybe you have solved this already, but one thing I had to do was to fix=
=20
this function in myutils.c:
float **Allocate_Float_Matrix( int N, int M )
{
float *workp, **p;
int i, j;
if ((p =3D (float **) malloc((unsigned) N * (M + 1) *
sizeof(float))) =3D=3D NULL)
errormessage("allocate_float","",NON);
workp =3D ((float *) p) + N;
for (i =3D j =3D 0; i < N; i++, j +=3D M)
p[i] =3D &workp[j];
return p;
}
The array contains bot floats and float pointers, which will work in 32bit=
=20
even if it's ugly code...
This version works both in 64 and 32bit:
=20
float **Allocate_Float_Matrix( int N, int M ) {
float *workp, **p;
int i, j;
if(!(p =3D malloc(N * sizeof(float *))))=20
errormessage("allocate_float","",NON);
for(i=3D0; i<N; i++) {
if(!(p[i] =3D malloc(M*sizeof(float)))) errormessage("allocate_float","",NO=
N);=20
}
return p;
}
Regards,
Leif
On 5/19/05, Deane, Paul <pd...@et...> wrote:=20
>=20
> Yes, cc'ing this discussion to the infomap list would be excellent.=20
> Daniel,
> could you give them the breakdown on the crashes we've been experiencing?
>=20
>=20
> -----Original Message-----
> From: Scott Cederberg [mailto:ced...@gm...]
> Sent: Thursday, May 19, 2005 4:59 PM
> To: Deane, Paul
> Cc: Dominic Widdows; bea...@im...; Zuckerman, Daniel;
> inf...@li...
> Subject: Re: Infomap & 64-bit
>=20
> Hi Paul,
>=20
> I'd be happy to be put in touch with Daniel to see if we can
> replicate the problem for debugging. If you're comfortable carrying
> on this discussion on the infomap-nlp-users list, let's do that so it can
> be useful for other people. (I'd been ignoring the lists for a while but=
=20
> am
> now reading them again.)
>=20
> It's going to take me a little while to refresh my memory as to how
> all this stuff works, but I'd like to root out the problem so the softwar=
e
> can scale to 64-bit systems in the future.
>=20
>=20
> Scott
>=20
> On 5/18/05, Deane, Paul <pd...@et...> wrote:
> > Here's my original post to the infomap list on this issue. We're
> > running Gentoo linux, and one problem we encountered when trying to
> > get it to compile was that the paths for ndbm.h didn't match the setup
> > on our machine, so we had to create a symbolic link -- so that's a
> > possible compatibility issue. However, the crash we get on the 64 bit
> > machine's down in svdinterface.
> >
> > I'm cc'ing the programmer who's been working on this, Daniel
> > Zuckerman, to keep him in the loop. Thanks if you're able to help
> > here. Ideal would be if Daniel could send you the details of the
> > scaleup problems we've had and see if you could replicate them -- we
> > had a problem on 32-bit when we tried to increase the size of the=20
> default
> SVD analysis, also.
> >
> > ----------------------------------------------------------------------
> > ----
> >
> > Has anyone successfully compiled infomap on 64-bit linux? (AMD 64
> > running Debian with 64-bit libraries only).
> >
> > When I tried to compile 0.8.5 (using the 64-bit version of GCC), I get
> > the following error messages during compile:
> >
> > myutils.c: In function `mymalloc':
> > myutils.c:167: error: conflicting types for 'malloc'
> > make[3]: *** [myutils.o] Error 1
> > make[2]: *** [all-recursive] Error 1
> > make[1]: *** [all-recursive] Error 1
> >
> > When I comment out the offending line, it compiles, but I get memory
> > errors when I attempt to run the program, i.e., the end of the log look=
s
> like this:
> >
> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
> > Building target: /export/home/bragi2/pdeane/test/left
> > Prerequisites: /export/home/bragi2/pdeane/test/coll
> > /export/home/bragi2/pdeane/t
> > est/indx
> > Fri Apr 29 09:06:56 EDT 2005
> > ..................................................
> > cd /export/home/bragi2/pdeane/test && rm svd_diag left \
> > rght sing
> > rm: cannot remove `svd_diag': No such file or directory
> > rm: cannot remove `left': No such file or directory
> > rm: cannot remove `rght': No such file or directory
> > rm: cannot remove `sing': No such file or directory
> > make: [/export/home/bragi2/pdeane/test/left] Error 1 (ignored) cd
> > /export/home/bragi2/pdeane/test && svdinterface \
> > -singvals 100 \
> > -iter 100
> >
> > This is svdinterface.
> >
> > Writing to: left
> > Writing to: rght
> > Writing to: sing
> > Writing to: svd_diag
> > Reading: indx
> > Reading: indx
> > Reading: coll
> > make: *** [/export/home/bragi2/pdeane/test/left] Error 139
> > ----------------------------------------------------------------------
> > ------
> > -----------------
> >
> > When I check the directory the model was being created in, this is the
> > state of things, with the evidence suggesting that something when
> > nastily wrong in the svdinterface code that calls the mymalloc
> > function:
> >
> > -rw-r--r-- 1 pdeane nlp 2819650 Apr 29 09:06 coll
> > -rw-r--r-- 1 pdeane nlp 374 Apr 29 09:06 corpus_format.bin
> > -rw-r--r-- 1 pdeane nlp 157223 Apr 29 09:06 dic
> > -rw-r--r-- 1 pdeane nlp 3939 Apr 29 09:06 indx
> > -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 left
> > -rw-r--r-- 1 pdeane nlp 28 Apr 29 09:06 model_info.bin
> > -rw-r--r-- 1 pdeane nlp 16396 Apr 29 09:06 model_params.bin
> > -rw-r--r-- 1 pdeane nlp 4 Apr 29 09:06 numDocs
> > -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 rght
> > -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 sing
> > -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 svd_diag
> > -rw-r--r-- 1 pdeane nlp 540345 Apr 29 09:06 wordlist
> >
> > I tried changing the defines e.g. BIGINT, BIGFLOAT in fixed.h to
> > 64-bit values but this makes no difference, and that was the only
> > obvious 32-bit dependency I could see.
> >
> > Interestingly, the original SVDPACKC compiles and runs in my
> > environment, so whatever is going on looks like it must be a function
> > of the svdinterface code and associated changes. Also, btw, I have had
> > no problem running infomap on a 32-bit linux machine nor under cygwin o=
n=20
> a
> windows machine.
> >
> > Can anybody help me out here?
> >
> >
> >
> >
> > -----Original Message-----
> > From: Dominic Widdows [mailto:dwi...@cs...]
> > Sent: Wednesday, May 18, 2005 10:01 AM
> > To: Deane, Paul
> > Cc: ced...@gm...; bea...@im...
> > Subject: Re: Infomap & 64-bit
> >
> > Hi Scott,
> >
> > Sorry to bother you during your vacation, but do you have any idea why
> > the infomap software may fail to run on a 64-bit machine?
> >
> > BTW, Paul, I believe there is a "matlab file" output option in the
> > infomap code, which outputs the raw counts to a matlab-compatible
> > sparse matrix format. This would probably make it easy for you to
> > separate the problem into counting coocurrences and then performing the
> SVD in a separate stage.
> > I'm cc'ing Beate Dorow, since she worked on this a few years ago.
> >
> > Best wishes,
> > Dominic
> >
> > On Wed, 18 May 2005, Deane, Paul wrote:
> >
> > > Hi, seeing your post today on the infomap mailing list prompts me to
> > > repeat a query.
> > >
> > > I've been trying to get infomap working on a large 64-bit machine
> > > running Linux, and my post to this list outlined my problems. Reason
> > > I'm doing this is I want to run analyses on some very large corpora,
> > > and infomap doesn't seem to scale up well to these very large
> > > corpora when I run them on our 32-bit machines ... something always
> > > breaks and we get memory errors when we increase the number of words
> > > we want to cover and/or the size of the corpus up toward our target,
> > > which is 100,000 words of vocabulary on a 500-million-word corpus.
> > >
> > > Is there a chance of getting any help here? If we can't get past the
> > > scaleup problems, we'll have to do the analysis outside of infomap,
> > > i.e., compile the counts and do the SVD indepedently.
> > >
> > > Thanks,
> > >
> > > Paul Deane
> > >
> > >
> > >
> > > ********************************************************************
> > > **
> > > **** This e-mail and any files transmitted with it may contain
> > > privileged or confidential information. It is solely for use by the
> > > individual for whom it is intended, even if addressed incorrectly.
> > > If you received this e-mail in error, please notify the sender; do
> > > not disclose, copy, distribute, or take any action in reliance on
> > > the contents of this information; and delete it from your system.
> > > Any other use of this e-mail is prohibited. Thank you for your
> compliance.
> > >
> > >
> > >
> > >
> >
> >
> >
> > **********************************************************************
> > **** This e-mail and any files transmitted with it may contain
> > privileged or confidential information. It is solely for use by the
> > individual for whom it is intended, even if addressed incorrectly. If
> > you received this e-mail in error, please notify the sender; do not
> > disclose, copy, distribute, or take any action in reliance on the
> > contents of this information; and delete it from your system. Any
> > other use of this e-mail is prohibited. Thank you for your compliance.
> >
> >
> >
> >
>=20
>=20
>=20
> *************************************************************************=
*
> This e-mail and any files transmitted with it may contain privileged or
> confidential information. It is solely for use by the individual for whom
> it is intended, even if addressed incorrectly. If you received this e-mai=
l
> in error, please notify the sender; do not disclose, copy, distribute, or
> take any action in reliance on the contents of this information; and=20
> delete
> it from your system. Any other use of this e-mail is prohibited. Thank yo=
u
> for your compliance.
>=20
>=20
>=20
>=20
>=20
> -------------------------------------------------------
> This SF.Net <http://SF.Net> email is sponsored by Oracle Space Sweepstake=
s
> Want to be the first software developer in space?
> Enter now for the Oracle Space Sweepstakes!
> http://ads.osdn.com/?ad_id=3D7412&alloc_id=3D16344&op=3Dclick
> _______________________________________________
> infomap-nlp-users mailing list
> inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infomap-nlp-users
>=20
--=20
Leif Gr=F6nqvist, GSLT, le...@li...,
www.ling.gu.se/~leifg<http://www.ling.gu.se/~leifg>,
031-821515(home)
School of Mathematics and Systems Engineering, V=E4xj=F6 University=20
0707164380(mob)
Department of Linguistics, G=F6teborg University, +46 31 773 1177, 773=20
4853(fax)
|
|
From: Deane, P. <pd...@et...> - 2005-05-19 21:02:19
|
Yes, cc'ing this discussion to the infomap list would be excellent. Daniel,
could you give them the breakdown on the crashes we've been experiencing?
-----Original Message-----
From: Scott Cederberg [mailto:ced...@gm...]
Sent: Thursday, May 19, 2005 4:59 PM
To: Deane, Paul
Cc: Dominic Widdows; bea...@im...; Zuckerman, Daniel;
inf...@li...
Subject: Re: Infomap & 64-bit
Hi Paul,
I'd be happy to be put in touch with Daniel to see if we can
replicate the problem for debugging. If you're comfortable carrying
on this discussion on the infomap-nlp-users list, let's do that so it can
be useful for other people. (I'd been ignoring the lists for a while but am
now reading them again.)
It's going to take me a little while to refresh my memory as to how
all this stuff works, but I'd like to root out the problem so the software
can scale to 64-bit systems in the future.
Scott
On 5/18/05, Deane, Paul <pd...@et...> wrote:
> Here's my original post to the infomap list on this issue. We're
> running Gentoo linux, and one problem we encountered when trying to
> get it to compile was that the paths for ndbm.h didn't match the setup
> on our machine, so we had to create a symbolic link -- so that's a
> possible compatibility issue. However, the crash we get on the 64 bit
> machine's down in svdinterface.
>
> I'm cc'ing the programmer who's been working on this, Daniel
> Zuckerman, to keep him in the loop. Thanks if you're able to help
> here. Ideal would be if Daniel could send you the details of the
> scaleup problems we've had and see if you could replicate them -- we
> had a problem on 32-bit when we tried to increase the size of the default
SVD analysis, also.
>
> ----------------------------------------------------------------------
> ----
>
> Has anyone successfully compiled infomap on 64-bit linux? (AMD 64
> running Debian with 64-bit libraries only).
>
> When I tried to compile 0.8.5 (using the 64-bit version of GCC), I get
> the following error messages during compile:
>
> myutils.c: In function `mymalloc':
> myutils.c:167: error: conflicting types for 'malloc'
> make[3]: *** [myutils.o] Error 1
> make[2]: *** [all-recursive] Error 1
> make[1]: *** [all-recursive] Error 1
>
> When I comment out the offending line, it compiles, but I get memory
> errors when I attempt to run the program, i.e., the end of the log looks
like this:
>
> ==================================================
> Building target: /export/home/bragi2/pdeane/test/left
> Prerequisites: /export/home/bragi2/pdeane/test/coll
> /export/home/bragi2/pdeane/t
> est/indx
> Fri Apr 29 09:06:56 EDT 2005
> ..................................................
> cd /export/home/bragi2/pdeane/test && rm svd_diag left \
> rght sing
> rm: cannot remove `svd_diag': No such file or directory
> rm: cannot remove `left': No such file or directory
> rm: cannot remove `rght': No such file or directory
> rm: cannot remove `sing': No such file or directory
> make: [/export/home/bragi2/pdeane/test/left] Error 1 (ignored) cd
> /export/home/bragi2/pdeane/test && svdinterface \
> -singvals 100 \
> -iter 100
>
> This is svdinterface.
>
> Writing to: left
> Writing to: rght
> Writing to: sing
> Writing to: svd_diag
> Reading: indx
> Reading: indx
> Reading: coll
> make: *** [/export/home/bragi2/pdeane/test/left] Error 139
> ----------------------------------------------------------------------
> ------
> -----------------
>
> When I check the directory the model was being created in, this is the
> state of things, with the evidence suggesting that something when
> nastily wrong in the svdinterface code that calls the mymalloc
> function:
>
> -rw-r--r-- 1 pdeane nlp 2819650 Apr 29 09:06 coll
> -rw-r--r-- 1 pdeane nlp 374 Apr 29 09:06 corpus_format.bin
> -rw-r--r-- 1 pdeane nlp 157223 Apr 29 09:06 dic
> -rw-r--r-- 1 pdeane nlp 3939 Apr 29 09:06 indx
> -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 left
> -rw-r--r-- 1 pdeane nlp 28 Apr 29 09:06 model_info.bin
> -rw-r--r-- 1 pdeane nlp 16396 Apr 29 09:06 model_params.bin
> -rw-r--r-- 1 pdeane nlp 4 Apr 29 09:06 numDocs
> -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 rght
> -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 sing
> -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 svd_diag
> -rw-r--r-- 1 pdeane nlp 540345 Apr 29 09:06 wordlist
>
> I tried changing the defines e.g. BIGINT, BIGFLOAT in fixed.h to
> 64-bit values but this makes no difference, and that was the only
> obvious 32-bit dependency I could see.
>
> Interestingly, the original SVDPACKC compiles and runs in my
> environment, so whatever is going on looks like it must be a function
> of the svdinterface code and associated changes. Also, btw, I have had
> no problem running infomap on a 32-bit linux machine nor under cygwin on a
windows machine.
>
> Can anybody help me out here?
>
>
>
>
> -----Original Message-----
> From: Dominic Widdows [mailto:dwi...@cs...]
> Sent: Wednesday, May 18, 2005 10:01 AM
> To: Deane, Paul
> Cc: ced...@gm...; bea...@im...
> Subject: Re: Infomap & 64-bit
>
> Hi Scott,
>
> Sorry to bother you during your vacation, but do you have any idea why
> the infomap software may fail to run on a 64-bit machine?
>
> BTW, Paul, I believe there is a "matlab file" output option in the
> infomap code, which outputs the raw counts to a matlab-compatible
> sparse matrix format. This would probably make it easy for you to
> separate the problem into counting coocurrences and then performing the
SVD in a separate stage.
> I'm cc'ing Beate Dorow, since she worked on this a few years ago.
>
> Best wishes,
> Dominic
>
> On Wed, 18 May 2005, Deane, Paul wrote:
>
> > Hi, seeing your post today on the infomap mailing list prompts me to
> > repeat a query.
> >
> > I've been trying to get infomap working on a large 64-bit machine
> > running Linux, and my post to this list outlined my problems. Reason
> > I'm doing this is I want to run analyses on some very large corpora,
> > and infomap doesn't seem to scale up well to these very large
> > corpora when I run them on our 32-bit machines ... something always
> > breaks and we get memory errors when we increase the number of words
> > we want to cover and/or the size of the corpus up toward our target,
> > which is 100,000 words of vocabulary on a 500-million-word corpus.
> >
> > Is there a chance of getting any help here? If we can't get past the
> > scaleup problems, we'll have to do the analysis outside of infomap,
> > i.e., compile the counts and do the SVD indepedently.
> >
> > Thanks,
> >
> > Paul Deane
> >
> >
> >
> > ********************************************************************
> > **
> > **** This e-mail and any files transmitted with it may contain
> > privileged or confidential information. It is solely for use by the
> > individual for whom it is intended, even if addressed incorrectly.
> > If you received this e-mail in error, please notify the sender; do
> > not disclose, copy, distribute, or take any action in reliance on
> > the contents of this information; and delete it from your system.
> > Any other use of this e-mail is prohibited. Thank you for your
compliance.
> >
> >
> >
> >
>
>
>
> **********************************************************************
> **** This e-mail and any files transmitted with it may contain
> privileged or confidential information. It is solely for use by the
> individual for whom it is intended, even if addressed incorrectly. If
> you received this e-mail in error, please notify the sender; do not
> disclose, copy, distribute, or take any action in reliance on the
> contents of this information; and delete it from your system. Any
> other use of this e-mail is prohibited. Thank you for your compliance.
>
>
>
>
**************************************************************************
This e-mail and any files transmitted with it may contain privileged or
confidential information. It is solely for use by the individual for whom
it is intended, even if addressed incorrectly. If you received this e-mail
in error, please notify the sender; do not disclose, copy, distribute, or
take any action in reliance on the contents of this information; and delete
it from your system. Any other use of this e-mail is prohibited. Thank you
for your compliance.
|
|
From: Scott C. <ced...@gm...> - 2005-05-19 20:59:30
|
Hi Paul,
I'd be happy to be put in touch with Daniel to see if we can
replicate the problem for debugging. If you're comfortable carrying
on this discussion on the infomap-nlp-users list, let's do that so it
can be useful for other people. (I'd been ignoring the lists for a
while but am now reading them again.)
It's going to take me a little while to refresh my memory as to
how all this stuff works, but I'd like to root out the problem so the
software can scale to 64-bit systems in the future.
=20
Scott
On 5/18/05, Deane, Paul <pd...@et...> wrote:
> Here's my original post to the infomap list on this issue. We're running
> Gentoo linux, and one problem we encountered when trying to get it to
> compile was that the paths for ndbm.h didn't match the setup on our machi=
ne,
> so we had to create a symbolic link -- so that's a possible compatibility
> issue. However, the crash we get on the 64 bit machine's down in
> svdinterface.
>=20
> I'm cc'ing the programmer who's been working on this, Daniel Zuckerman, t=
o
> keep him in the loop. Thanks if you're able to help here. Ideal would be =
if
> Daniel could send you the details of the scaleup problems we've had and s=
ee
> if you could replicate them -- we had a problem on 32-bit when we tried t=
o
> increase the size of the default SVD analysis, also.
>=20
> -------------------------------------------------------------------------=
-
>=20
> Has anyone successfully compiled infomap on 64-bit linux? (AMD 64 running
> Debian with 64-bit libraries only).
>=20
> When I tried to compile 0.8.5 (using the 64-bit version of GCC), I get th=
e
> following error messages during compile:
>=20
> myutils.c: In function `mymalloc':
> myutils.c:167: error: conflicting types for 'malloc'
> make[3]: *** [myutils.o] Error 1
> make[2]: *** [all-recursive] Error 1
> make[1]: *** [all-recursive] Error 1
>=20
> When I comment out the offending line, it compiles, but I get memory erro=
rs
> when I attempt to run the program, i.e., the end of the log looks like th=
is:
>=20
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
> Building target: /export/home/bragi2/pdeane/test/left
> Prerequisites: /export/home/bragi2/pdeane/test/coll
> /export/home/bragi2/pdeane/t
> est/indx
> Fri Apr 29 09:06:56 EDT 2005
> ..................................................
> cd /export/home/bragi2/pdeane/test && rm svd_diag left \
> rght sing
> rm: cannot remove `svd_diag': No such file or directory
> rm: cannot remove `left': No such file or directory
> rm: cannot remove `rght': No such file or directory
> rm: cannot remove `sing': No such file or directory
> make: [/export/home/bragi2/pdeane/test/left] Error 1 (ignored)
> cd /export/home/bragi2/pdeane/test && svdinterface \
> -singvals 100 \
> -iter 100
>=20
> This is svdinterface.
>=20
> Writing to: left
> Writing to: rght
> Writing to: sing
> Writing to: svd_diag
> Reading: indx
> Reading: indx
> Reading: coll
> make: *** [/export/home/bragi2/pdeane/test/left] Error 139
> -------------------------------------------------------------------------=
---
> -----------------
>=20
> When I check the directory the model was being created in,
> this is the state of things, with the evidence suggesting that something
> when nastily wrong in the svdinterface code that calls the mymalloc
> function:
>=20
> -rw-r--r-- 1 pdeane nlp 2819650 Apr 29 09:06 coll
> -rw-r--r-- 1 pdeane nlp 374 Apr 29 09:06 corpus_format.bin
> -rw-r--r-- 1 pdeane nlp 157223 Apr 29 09:06 dic
> -rw-r--r-- 1 pdeane nlp 3939 Apr 29 09:06 indx
> -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 left
> -rw-r--r-- 1 pdeane nlp 28 Apr 29 09:06 model_info.bin
> -rw-r--r-- 1 pdeane nlp 16396 Apr 29 09:06 model_params.bin
> -rw-r--r-- 1 pdeane nlp 4 Apr 29 09:06 numDocs
> -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 rght
> -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 sing
> -rw-r--r-- 1 pdeane nlp 0 Apr 29 09:06 svd_diag
> -rw-r--r-- 1 pdeane nlp 540345 Apr 29 09:06 wordlist
>=20
> I tried changing the defines e.g. BIGINT, BIGFLOAT in fixed.h to 64-bit
> values but this makes no difference, and that was the only obvious 32-bit
> dependency I could see.
>=20
> Interestingly, the original SVDPACKC compiles and runs in my environment,=
so
> whatever is going on looks like it must be a function of the svdinterface
> code and associated changes. Also, btw, I have had no problem running
> infomap on a 32-bit linux machine nor under cygwin on a windows machine.
>=20
> Can anybody help me out here?
>=20
>=20
>=20
>=20
> -----Original Message-----
> From: Dominic Widdows [mailto:dwi...@cs...]
> Sent: Wednesday, May 18, 2005 10:01 AM
> To: Deane, Paul
> Cc: ced...@gm...; bea...@im...
> Subject: Re: Infomap & 64-bit
>=20
> Hi Scott,
>=20
> Sorry to bother you during your vacation, but do you have any idea why th=
e
> infomap software may fail to run on a 64-bit machine?
>=20
> BTW, Paul, I believe there is a "matlab file" output option in the infoma=
p
> code, which outputs the raw counts to a matlab-compatible sparse matrix
> format. This would probably make it easy for you to separate the problem
> into counting coocurrences and then performing the SVD in a separate stag=
e.
> I'm cc'ing Beate Dorow, since she worked on this a few years ago.
>=20
> Best wishes,
> Dominic
>=20
> On Wed, 18 May 2005, Deane, Paul wrote:
>=20
> > Hi, seeing your post today on the infomap mailing list prompts me to
> > repeat a query.
> >
> > I've been trying to get infomap working on a large 64-bit machine
> > running Linux, and my post to this list outlined my problems. Reason
> > I'm doing this is I want to run analyses on some very large corpora,
> > and infomap doesn't seem to scale up well to these very large corpora
> > when I run them on our 32-bit machines ... something always breaks and
> > we get memory errors when we increase the number of words we want to
> > cover and/or the size of the corpus up toward our target, which is
> > 100,000 words of vocabulary on a 500-million-word corpus.
> >
> > Is there a chance of getting any help here? If we can't get past the
> > scaleup problems, we'll have to do the analysis outside of infomap,
> > i.e., compile the counts and do the SVD indepedently.
> >
> > Thanks,
> >
> > Paul Deane
> >
> >
> >
> > **********************************************************************
> > **** This e-mail and any files transmitted with it may contain
> > privileged or confidential information. It is solely for use by the
> > individual for whom it is intended, even if addressed incorrectly. If
> > you received this e-mail in error, please notify the sender; do not
> > disclose, copy, distribute, or take any action in reliance on the
> > contents of this information; and delete it from your system. Any
> > other use of this e-mail is prohibited. Thank you for your compliance.
> >
> >
> >
> >
>=20
>=20
>=20
> *************************************************************************=
*
> This e-mail and any files transmitted with it may contain privileged or
> confidential information. It is solely for use by the individual for whom
> it is intended, even if addressed incorrectly. If you received this e-mai=
l
> in error, please notify the sender; do not disclose, copy, distribute, or
> take any action in reliance on the contents of this information; and dele=
te
> it from your system. Any other use of this e-mail is prohibited. Thank yo=
u
> for your compliance.
>=20
>=20
>=20
>
|
1213 messages has been excluded from this view by a project administrator.
