Menu

#285 Data Order Shuffling Effects RankLib Model Evaluation Metric

v1.x
accepted
1
2016-05-17
2016-05-17
No

Many evaluation metrics over several RankLib LTR models shows either very good or very bad results that are seemingly affected by shuffling the input data. This is happening with a significantly large enough training data although feature counts are small.

The suspicion is that the problem may be related to scoring class function doing sorting before scoring.

See https://sourceforge.net/p/lemur/discussion/ranklib/thread/15d0a4e8/?limit=25#0b0d for more details on the problem.

I don't believe input data ordering should effect evaluation metric results.

Discussion


Log in to post a comment.