Recent changes to modelshttps://sourceforge.net/p/jobimtext/wiki/models/Recent changes to modelsenTue, 08 Apr 2014 11:53:34 -0000models modified by Chris Biemannhttps://sourceforge.net/p/jobimtext/wiki/models/<div class="markdown_content"><pre>--- v23 +++ v24 @@ -34,4 +34,4 @@ #Sense Clusters Dataset |Download| Holing System|Clustering ------|--------|--------|-------- -news120M|[I](http://sourceforge.net/projects/jobimtext/files/data/sensecluster/news120M_sense_cluster_cw.tgz/download)|Stanford Parser & Lemma|Chinese Whisperer +news120M|[I](http://sourceforge.net/projects/jobimtext/files/data/sensecluster/news120M_sense_cluster_cw.tgz/download)|Stanford Parser & Lemma|Chinese Whispers </pre> </div>Chris BiemannTue, 08 Apr 2014 11:53:34 -0000https://sourceforge.neteecf3edf26d2d7a806cc958fc5d1cadd9fab023emodels modified by Eugen Rupperthttps://sourceforge.net/p/jobimtext/wiki/models/<div class="markdown_content"><pre>--- v22 +++ v23 @@ -3,6 +3,8 @@ This page lists the datasets and models which are available at: https://sourceforge.net/projects/jobimtext/files/data/ + +We also computed Models for various time slices on [Google Books](http://sourceforge.net/p/jobimtext/wiki/LREC2014_Google_DT/) data. [TOC] </pre> </div>Eugen RuppertWed, 05 Mar 2014 15:02:24 -0000https://sourceforge.netec370006ec34361b10cc3aba91534d0d8dc21ab0models modified by Martin Riedlhttps://sourceforge.net/p/jobimtext/wiki/models/<div class="markdown_content"><pre>--- v21 +++ v22 @@ -31,5 +31,5 @@ #Sense Clusters Dataset |Download| Holing System|Clustering -------|--------|-------- +------|--------|--------|-------- news120M|[I](http://sourceforge.net/projects/jobimtext/files/data/sensecluster/news120M_sense_cluster_cw.tgz/download)|Stanford Parser & Lemma|Chinese Whisperer </pre> </div>Martin RiedlFri, 19 Jul 2013 17:47:07 -0000https://sourceforge.netc969dc9558199514476c7cf472406b43ce3f029fmodels modified by Martin Riedlhttps://sourceforge.net/p/jobimtext/wiki/models/<div class="markdown_content"><pre>--- v20 +++ v21 @@ -1,4 +1,4 @@ -Data and Models +Datasets and Precomputed Models ======= This page lists the datasets and models which are available at: @@ -13,7 +13,7 @@ - **[en_wikipedia](http://sourceforge.net/projects/jobimtext/files/data/dataset/dataset_wikipedia_en.tar.gz/download)**: This dataset is constructed using English Wikipedia. It consists of 35.9 million sentences. - **[en_google_books](http://commondatastorage.googleapis.com/books/syntactic-ngrams/index.html)**: This dataset is constructed by Yoav Goldberg ([A Dataset of Syntactc-Ngrams over Time from a Very Large Corpus of English Books, *SEM 2013](http://commondatastorage.googleapis.com/books/syntactic-ngrams/syntngrams.final.pdf)) and can be downloaded [here](http://commondatastorage.googleapis.com/books/syntactic-ngrams/index.html). -#Models +#Distributional Thesaurus Models Dataset |Download| Holing System|Word Count | Feature Count | Word Feature Count| Word Feature Significances | Similarities @@ -27,3 +27,9 @@ en_news120M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_3gram2_pruned.zip/download) | 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words de_news70M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/de_news70M_pruned.zip/download)| 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words google_books|[I](http://sourceforge.net/projects/jobimtext/files/data/models/google_books_top_1M_words.tar.gz/download)| dependency parses| all | none | none| top 1000/word for words with wordcount > 100| top 200 for words with wordcount > 100 + + +#Sense Clusters +Dataset |Download| Holing System|Clustering +------|--------|-------- +news120M|[I](http://sourceforge.net/projects/jobimtext/files/data/sensecluster/news120M_sense_cluster_cw.tgz/download)|Stanford Parser & Lemma|Chinese Whisperer </pre> </div>Martin RiedlFri, 19 Jul 2013 17:46:44 -0000https://sourceforge.net7a9a570bf7b302e18b24334a9fd9411e213f47f8models modified by Martin Riedlhttps://sourceforge.net/p/jobimtext/wiki/models/<div class="markdown_content"><pre>--- v19 +++ v20 @@ -26,4 +26,4 @@ en_news120M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_stanford_lemma_np_pruned.zip/download)| StanfordParser & Lemma | all |none |none|top 1000/word for 100k frequent words|top 200 for 100k frequent words en_news120M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_3gram2_pruned.zip/download) | 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words de_news70M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/de_news70M_pruned.zip/download)| 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words -google_books|[I](http://sourceforge.net/projects/jobimtext/files/data/models/google_books_top1M_words/download)| dependency parses| all | none | none| top 1000/word for words with wordcount > 100| top 200 for words with wordcount > 100 +google_books|[I](http://sourceforge.net/projects/jobimtext/files/data/models/google_books_top_1M_words.tar.gz/download)| dependency parses| all | none | none| top 1000/word for words with wordcount > 100| top 200 for words with wordcount > 100 </pre> </div>Martin RiedlTue, 09 Jul 2013 13:40:06 -0000https://sourceforge.netb0181c3adcf6e7df6bca9fc019ba59ffbd3b77f4models modified by Martin Riedlhttps://sourceforge.net/p/jobimtext/wiki/models/<div class="markdown_content"><pre>--- v18 +++ v19 @@ -26,4 +26,4 @@ en_news120M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_stanford_lemma_np_pruned.zip/download)| StanfordParser & Lemma | all |none |none|top 1000/word for 100k frequent words|top 200 for 100k frequent words en_news120M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_3gram2_pruned.zip/download) | 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words de_news70M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/de_news70M_pruned.zip/download)| 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words -google_books|[I]()| dependency parses| all | none | none| top 1000/word for words with wordcount > 100| top 200 for words with wordcount > 100 +google_books|[I](http://sourceforge.net/projects/jobimtext/files/data/models/google_books_top1M_words/download)| dependency parses| all | none | none| top 1000/word for words with wordcount > 100| top 200 for words with wordcount > 100 </pre> </div>Martin RiedlMon, 08 Jul 2013 13:47:12 -0000https://sourceforge.net4a011a3db484b1699aeb9a3dc0285b60489d5ac4models modified by Martin Riedlhttps://sourceforge.net/p/jobimtext/wiki/models/<div class="markdown_content"><pre>--- v17 +++ v18 @@ -7,11 +7,11 @@ [TOC] #Datasets -There are two datasets [available](http://sourceforge.net/projects/jobimtext/files/data/dataset) that can be used for calculating a new thesaurus. The format of the files is that each line contains one sentence. +Following [available](http://sourceforge.net/projects/jobimtext/files/data/dataset) that can be used for calculating a new thesaurus. The format of the files is that each line contains one sentence. - **[en_news10M](http://sourceforge.net/projects/jobimtext/files/data/dataset/dataset_news10M.gz/download)**: This dataset is taken from [LCC](http://corpora.uni-leipzig.de/download.html). It consists of 10 million English sentences taken from news web pages. - **[en_wikipedia](http://sourceforge.net/projects/jobimtext/files/data/dataset/dataset_wikipedia_en.tar.gz/download)**: This dataset is constructed using English Wikipedia. It consists of 35.9 million sentences. - +- **[en_google_books](http://commondatastorage.googleapis.com/books/syntactic-ngrams/index.html)**: This dataset is constructed by Yoav Goldberg ([A Dataset of Syntactc-Ngrams over Time from a Very Large Corpus of English Books, *SEM 2013](http://commondatastorage.googleapis.com/books/syntactic-ngrams/syntngrams.final.pdf)) and can be downloaded [here](http://commondatastorage.googleapis.com/books/syntactic-ngrams/index.html). #Models @@ -26,3 +26,4 @@ en_news120M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_stanford_lemma_np_pruned.zip/download)| StanfordParser & Lemma | all |none |none|top 1000/word for 100k frequent words|top 200 for 100k frequent words en_news120M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_3gram2_pruned.zip/download) | 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words de_news70M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/de_news70M_pruned.zip/download)| 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words +google_books|[I]()| dependency parses| all | none | none| top 1000/word for words with wordcount > 100| top 200 for words with wordcount > 100 </pre> </div>Martin RiedlMon, 08 Jul 2013 13:45:31 -0000https://sourceforge.netf948a31372eed08ade3563d9f810535452f23c1emodels modified by Martin Riedlhttps://sourceforge.net/p/jobimtext/wiki/models/<div class="markdown_content"><pre>--- v16 +++ v17 @@ -22,7 +22,7 @@ en_wikipedia | [I](http://sourceforge.net/projects/jobimtext/files/data/models/wikipedia_en_malt_demo.tar.gz/download) | Malt Parser & Lemmatized|none|none |none|top 1000/word|top 100 en_news10M |[I](http://sourceforge.net/projects/jobimtext/files/data/models/similarities-news10M-maltparser.tar.gz/download) | Malt Parser & Lemmatized| all|all |all|all|top 200 en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/similarities-news120M_stanford_lemma_np.tar.gz_1/download) [II](http://sourceforge.net/projects/jobimtext/files/data/models/similarities-news120M_stanford_lemma_np.tar.gz_2/download) [III](http://sourceforge.net/projects/jobimtext/files/data/models/similarities-news120M_stanford_lemma_np.tar.gz_3/download) | Stanford Parser & Lemmatized| all|all |all|all|top 200 -en_wikipedia -en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_stanford_lemma_np_pruned.zip/download)| StanfordParser & Lemma | all |none |none|top 1000/word for 100k frequent words|top 200 for 100k frequent words -en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_3gram2_pruned.zip/download) | 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words -de_news70M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/de_news70M_pruned.zip/download)| 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words +en_news120M pruned mysql | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_stanford_lemma_np_pruned.mysql.gz/download)| StanfordParser & Lemma | all |none |none|top 1000/word for 100k frequent words|top 200 for 100k frequent words +en_news120M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_stanford_lemma_np_pruned.zip/download)| StanfordParser & Lemma | all |none |none|top 1000/word for 100k frequent words|top 200 for 100k frequent words +en_news120M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_3gram2_pruned.zip/download) | 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words +de_news70M pruned | [I](http://sourceforge.net/projects/jobimtext/files/data/models/de_news70M_pruned.zip/download)| 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words </pre> </div>Martin RiedlMon, 03 Jun 2013 09:12:55 -0000https://sourceforge.net8aec6f9ef032bc61bc591f3e7ec46ee1adf56ef7models modified by Martin Riedlhttps://sourceforge.net/p/jobimtext/wiki/models/<div class="markdown_content"><pre>--- v15 +++ v16 @@ -16,19 +16,12 @@ #Models - -- **en_wikipedia_maltparser [I](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-wiki-maltparser-part-I.tar.gz/download) [II](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-wiki-maltparser-part-II.tar.gz/download)**: This model is computed using the from the English Wikipedia dataset. As "Holing System" dependencies extracted from the [MaltParser](http://www.maltparser.org/) are used. - -- **[en_news10M_maltparser](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news10M-maltparser.tar.gz/download)**: This model is computed using the from the en_news10M dataset. As "Holing System" dependencies extracted from the [MaltParser](http://www.maltparser.org/) are used. - -- **en_news120M_stanfordparser [I](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news120M_stanford_lemma_np.tar.gz_1/download) [II](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news120M_stanford_lemma_np.tar.gz_2/download) [III](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news120M_stanford_lemma_np.tar.gz_3/download)**: This thesaurus is calculated based on 120M sentences extracted from news web pages which are taken from [LCC](http://corpora.uni-leipzig.de/download.html). The similarities are computed using the dependencies from the [Stanford Parser](http://nlp.stanford.edu/software/lex-parser.shtml). - Dataset |Download| Holing System|Word Count | Feature Count | Word Feature Count| Word Feature Significances | Similarities ------|--------|--------|----------|----|----|-----|--------- -en_wikipedia |[I](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-wiki-maltparser-part-I.tar.gz/download) [II](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-wiki-maltparser-part-II.tar.gz/download) | Malt Parser & Lemmatized| all|all |all|all|top 200 +en_wikipedia |[I](http://sourceforge.net/projects/jobimtext/files/data/models/similarities-wiki-maltparser-part-I.tar.gz/download) [II](http://sourceforge.net/projects/jobimtext/files/data/models/similarities-wiki-maltparser-part-II.tar.gz/download) | Malt Parser & Lemmatized| all|all |all|all|top 200 en_wikipedia | [I](http://sourceforge.net/projects/jobimtext/files/data/models/wikipedia_en_malt_demo.tar.gz/download) | Malt Parser & Lemmatized|none|none |none|top 1000/word|top 100 -en_news10M |[I](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news10M-maltparser.tar.gz/download) | Malt Parser & Lemmatized| all|all |all|all|top 200 -en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news120M_stanford_lemma_np.tar.gz_1/download) [II](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news120M_stanford_lemma_np.tar.gz_2/download) [III](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news120M_stanford_lemma_np.tar.gz_3/download) | Stanford Parser & Lemmatized| all|all |all|all|top 200 +en_news10M |[I](http://sourceforge.net/projects/jobimtext/files/data/models/similarities-news10M-maltparser.tar.gz/download) | Malt Parser & Lemmatized| all|all |all|all|top 200 +en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/similarities-news120M_stanford_lemma_np.tar.gz_1/download) [II](http://sourceforge.net/projects/jobimtext/files/data/models/similarities-news120M_stanford_lemma_np.tar.gz_2/download) [III](http://sourceforge.net/projects/jobimtext/files/data/models/similarities-news120M_stanford_lemma_np.tar.gz_3/download) | Stanford Parser & Lemmatized| all|all |all|all|top 200 en_wikipedia en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_stanford_lemma_np_pruned.zip/download)| StanfordParser & Lemma | all |none |none|top 1000/word for 100k frequent words|top 200 for 100k frequent words en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_3gram2_pruned.zip/download) | 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words </pre> </div>Martin RiedlMon, 03 Jun 2013 09:09:56 -0000https://sourceforge.net602e6721c791f69ce3e827f7892113995842dee3models modified by Martin Riedlhttps://sourceforge.net/p/jobimtext/wiki/models/<div class="markdown_content"><pre>--- v14 +++ v15 @@ -30,6 +30,6 @@ en_news10M |[I](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news10M-maltparser.tar.gz/download) | Malt Parser & Lemmatized| all|all |all|all|top 200 en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news120M_stanford_lemma_np.tar.gz_1/download) [II](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news120M_stanford_lemma_np.tar.gz_2/download) [III](http://sourceforge.net/projects/jobimtext/files/data/model/similarities-news120M_stanford_lemma_np.tar.gz_3/download) | Stanford Parser & Lemmatized| all|all |all|all|top 200 en_wikipedia -en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_stanford_lemma_np_pruned.zip/download)| StanfordParser & Lemma | all |none |none|top 1000/word|top 200 for 100k frequent words -en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_3gram2_pruned.zip/download) | 3gram w. hole at pos. 2 | all |none|none|top 1000/word|top 200 for 100k frequent words -de_news70M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/de_news70M_pruned.zip/download)| 3gram w. hole at pos. 2 | all |none|none|top 1000/word|top 200 for 100k frequent words +en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_stanford_lemma_np_pruned.zip/download)| StanfordParser & Lemma | all |none |none|top 1000/word for 100k frequent words|top 200 for 100k frequent words +en_news120M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/news120M_3gram2_pruned.zip/download) | 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words +de_news70M | [I](http://sourceforge.net/projects/jobimtext/files/data/models/de_news70M_pruned.zip/download)| 3gram w. hole at pos. 2 | all |none|none|top 1000/word for 100k frequent words|top 200 for 100k frequent words </pre> </div>Martin RiedlThu, 04 Apr 2013 14:41:54 -0000https://sourceforge.net94fdcc98229bd9f8fc6f302ad1e5c6973cfd365d