Surabhi Amit Chembra

From my analysis, it is the NDCG calculated based on the order of documents present in the test data for each query. If the rows within each query is sorted by relevancy label in the decsending order, it would be the maximum possible NDCG, which is, 1.0. So, if documents are in the order of BM25 score, then the baseline would indicate the NDCG score using BM25 alone and no ML algorithm.

Thanks Diego. Appreciate the response.

Thanks Diego, totally understand not recalling much details. Appreciate the response.

Thanks Diego, totally understand not recalling much details. If you remember anything about how you solved it, please let me know.

Thanks Diego, totally understand not recalling much details. If you remember about how you solved it, please let me know. I wanted to check if reducing dataset by limiting number of documents to 45k per query helped you.

Thanks Diego, so if the number of documents for each query is less than 45661, then the bug is not triggered even if there are thousands of queries? And is this fixed in new versions of RankLib?

Thanks Diego, so if the number of documents for each query is less than 45661, then the bug is not triggered even if there are thousands of queries?

For pairwise algorithms in RankLib, can the target value be a float? I assume for each qid, a pair of documents will be considered in pairwise algorithm and assigned a +1 or -1 label according to which has bigger relevance label(target value).