Menu

#80 How to specify a stemmer to use at query time?

v3.x
closed
1
2018-06-26
2017-12-05
No

I built an index using both Krovetz and a custom Snowball Spanish stemmer (see build manifest below). Galago always uses the krovetz stemmer at query time.

If I build the index using only the snowball stemmer, Galago uses the Null stemmer at query time.

How can I tell galago to use the snowball stemmer?

{
  "corpus" : true,
  "corpusParameters" : {
    "blockSize" : 512,
    "filename" : "/media/michaelz/SCRATCH/index/trec_spanish_tmp/corpus",
    "mergerClass" : "org.lemurproject.galago.core.index.merge.CorpusMerger",
    "readerClass" : "org.lemurproject.galago.core.index.corpus.CorpusReader",
    "tokenizer" : {
      "fields" : [ ],
      "formats" : {

      }
    },
    "writerClass" : "org.lemurproject.galago.core.index.corpus.CorpusFolderWriter"
  },
  "fieldIndex" : false,
  "fieldIndexParameters" : {
    "nonStemmedPostings" : true,
    "stemmedPostings" : true,
    "stemmer" : [ "krovetz" , "snowball"]
  },
  "filetype" : "trectext",
  "galagoVersion" : "3.12",
  "galagoVersionBuildDateTime" : "2017-12-04 18:52",
  "indexBuildDateTime" : "2017-12-05 12:40",
  "indexPath" : "/media/michaelz/SCRATCH/index/trec_spanish_tmp",
  "inputPath" : [ "/media/michaelz/SCRATCH/data/trec/trec_spanish/file-infosel.list"],
  "nonStemmedPostings" : true,
  "skipDistance" : 500,
  "skipping" : true,
  "stemmedCounts" : false,
  "stemmedPostings" : true,
  "stemmer" : [ "krovetz" , "snowball"],
  "stemmerClass" : {
    "krovetz" : "org.lemurproject.galago.core.parse.stem.KrovetzStemmer",
    "snowball" : "org.lemurproject.galago.core.parse.stem.SnowballSpanishStemmer"
  },
  "tokenizer" : {
    "fields" : [ ],
    "formats" : {

    }
  }
}

Discussion

  • David Fisher

    David Fisher - 2017-12-06
    • status: open --> pending
    • assigned_to: David Fisher
     
  • Lemur Project

    Lemur Project - 2018-06-26
    • status: pending --> closed
     

Log in to post a comment.