Menu
▾
▴
#287 RankLib - Feature Normalization
Hi gents,
I was verifying a little bit how normalization works in RankLib, and specifically for Sparse DataPoints.
First of all I noticed this :
ciir/umass/edu/learning/SparseDataPoint.java:113
if(pos >= 0)
fVals[pos] = fval;
else
{
System.err.println("Error in SparseDataPoint::setFeatureValue(): feature (id=" + fid + ") not found.");
System.exit(1);
}
}
This is in a SparseDataPoint but actually doesn't support sparse at all, because the else branch can absolutely happen during normalization and we basically kill the entire process.
So this is my first question :
are we sure we are ok with that ? basically it makes my sparse data set normalization broke .
Second point is a little bit more general :
The normalizer take in input a RankList at the time, normalizing intra RankList : ciir.umass.edu.features.LinearNormalizer#normalize(ciir.umass.edu.learning.RankList)
So far, so good but when we try to normalize the dataset :
We simply iterate each RankList and normalize it separetely !
Then the model as we know at training time, merges all the RankLists, this should build a very inconsistent dataset ( as the feature values were normalized per RankList with different max/min depending of the elements of the RankList)
Please correct me if I misunderstood anything,
In the other case I can provide a patch + tests
Cheers
It seems sparse data representations were added to RankLib as an afterthought, and only for a particular ranking algorithm (I forget which one).
As you noted, the sparse representation largely didn't even actually work.
I'd have to look more closely at the normalization code (I think all types of normalization would have issues), but I don't think it's a problem to normalize across all rank lists for a specific query.
The problem is how to handle missing features within a sparse data set.
Some people just treat the feature value as 0, which doesn't seem right. One could also just skip it and end up with feature min/max/average values that go across varying sample sizes, which also might not be a good thing.
In general, I'd favor normalization simply ignoring missing values from a sparse data point.
It might also be nice to allow the user to specify what a default value should be for each specific feature value (a default value for all features seems the wrong way to go).
I'd appreciate any further thoughts on how sparse data points are supposed to treat missing feature values.