Astro

Hi Michael, Thanks for helping out again, glad to have gotten to the bottom of the LM scores. If you would have any further clue on the BM25 calculation, that would be wonderful. Thanks again! Kind regards, Astro

Also, I am still not completely sure about the scoring mechanisms in Okapi and LM. I updated the Okapi scoring function on what I could identify on the reference given by @Michael Zarozinski (thanks!): the BM25TF function and the IDF function. I am still not sure what type of termweighting is done in the Okapi model, the current weighting scheme however does not do anything in the calculation since in all cases it comes down to 1 as we only have one query term. Okapi_score = log(termWeight * IDF...

Also, I am still not completely sure about the scoring mechanisms in Okapi and LM. I updated the Okapi scoring function on what I could identify on the reference given by @Michael Zarozinski (thanks!): the BM25TF function and the IDF function. I am still not sure what type of termweighting is done in the Okapi model, the current weighting scheme however does not do anything in the calculation since in all cases it comes down to 1 as we only have one query term. Okapi_score = log(termWeight * IDF...

Hi Michael, Thank you very much for your efforts. The workingSetDocno was a remainder of my non-toy example runs, by removing it indeed it does drop doc4 with a score of 0. Great! I have installed Indri 5.13. However, when I run the toy example on BM25 I currently get a failing assertion and therefore IndriRunQuery stops before producing output. After running `path\to\indri 5.13\bin\BuildIndex.exe param/toy/index_param` and `path\to\indri 5.11\bin\IndriRunquery.exe param/toy/bm25.indri input/toy/toy_query.txt...

Hi Michael, Thank you very much for your efforts. The workingSetDocno was a remainder of my non-toy example runs, by removing it indeed it does drop doc4 with a score of 0. Great! I have installed Indri 5.13. However, when I run the toy example on BM25 I currently get a failing assertion and therefore IndriRunQuery stops before producing output. After running `path\to\indri 5.13\bin\BuildIndex.exe param/toy/index_param` and `path\to\indri 5.11\bin\IndriRunquery.exe param/toy/bm25.indri input/toy/toy_query.txt...

Hi Michael, Thanks for your reply! I am using Indri 5.11. These are the files and commands I am using: Query file <parameters> <query> <number>1</number> <text>A</text> <workingSetDocno>1</workingSetDocno> <workingSetDocno>2</workingSetDocno> <workingSetDocno>3</workingSetDocno> <workingSetDocno>4</workingSetDocno> </query> </parameters> Parameter files for building the index <parameters> <index>index/toy</index> <corpus> <path>input/toy/toy_doc.txt</path> <class>trectext</class> </corpus> <memory>500m</memory>...

Hi Michael, Thanks for your reply! I am using Indri 5.11. These are the files and commands I am using: Query file <parameters> <query> <number>1</number> <text>A</text> <workingSetDocno>1</workingSetDocno> <workingSetDocno>2</workingSetDocno> <workingSetDocno>3</workingSetDocno> <workingSetDocno>4</workingSetDocno> </query> </parameters> Parameter files for building the index <parameters> <index>index/toy</index> <corpus> <path>input/toy/toy_doc.txt</path> <class>trectext</class> </corpus> <memory>500m</memory>...

Hi Michael, Thanks for your reply! I am using Indri 5.11. These are the files and commands I am using: Query file <parameters> <query> <number>1</number> <text>A</text> <workingSetDocno>1</workingSetDocno> <workingSetDocno>2</workingSetDocno> <workingSetDocno>3</workingSetDocno> <workingSetDocno>4</workingSetDocno> </query> </parameters> Parameter files for building the index <parameters> <index>index/toy</index> <corpus> <path>input/toy/toy_doc.txt</path> <class>trectext</class> </corpus> <memory>500m</memory>...