I am using AdaRank for training my data. I am trying several metrics to see the best result produced and I noticed that by using P@K and MAP, the output shows the P@k and MAP always 1.0. I am just wondering, are those metrics not supported to be used with AdaRank algorithm?
I look forward to your response.
Cheers
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I don't believe there are any metric use limitations for RankLib algorithms. Any of the metrics should work, within the limits of binary versus multiple levels of relevance labels.
Keep in mind that MAP and I believe NDCG require a full relevance judgments file for proper evaluation. You won't get good evaluations if input files don't contain all the relevant documents when using those metrics. Use the -qrel argument to bring in a full relevance judgment file for metrics requiring full relevance knowledge (or as best as can be determined).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I am using AdaRank for training my data. I am trying several metrics to see the best result produced and I noticed that by using P@K and MAP, the output shows the P@k and MAP always 1.0. I am just wondering, are those metrics not supported to be used with AdaRank algorithm?
I look forward to your response.
Cheers
I don't believe there are any metric use limitations for RankLib algorithms. Any of the metrics should work, within the limits of binary versus multiple levels of relevance labels.
Keep in mind that MAP and I believe NDCG require a full relevance judgments file for proper evaluation. You won't get good evaluations if input files don't contain all the relevant documents when using those metrics. Use the -qrel argument to bring in a full relevance judgment file for metrics requiring full relevance knowledge (or as best as can be determined).