See CoorAscent.java and in particular the getDistance() method which determines Euclidean distance between original and new weight vectors within an optimization step. The slack variable is the regularization parameter. The product of distance and slack is subtracted from the ranking score to give the regularized score.
By default, regularization is off, so to use it, you'll need to add a -reg <slack_value> parameter. Default regularization parameter value is 0.001.</slack_value>
Details of the algorithm are given in the Metzler and Croft, 2007 paper which is available on line.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The weight vector is initialized as you state, but it changes over the iterations and restarts. The distance combined with slack parameter is somewhat analagous to an L2 regularization.
This regularization process isn't really defined in the original Metzler and Croft paper, so may have been experimental or just added by the implementer. The idea appears to be to penalize weights that vary too much from uniform to reduce the likelihood of over-fitting.
Perhaps I'll hear back from the implementer of this algorithm that will more definitively state the intent of this code, but it appears to me to be a form of regularization.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Looks like this is the use of slack variables, more commonly applied to soft margin classification, where one adds/subtracts values to allow some errors, by moving weights to where they belong as costs/losses, all for the purpose of producing a more generalized model.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Could somebody please explain how regularization is implemented in the Linear feature model solved by Coordinate Ascent?
See CoorAscent.java and in particular the getDistance() method which determines Euclidean distance between original and new weight vectors within an optimization step. The slack variable is the regularization parameter. The product of distance and slack is subtracted from the ranking score to give the regularized score.
By default, regularization is off, so to use it, you'll need to add a -reg <slack_value> parameter. Default regularization parameter value is 0.001.</slack_value>
Details of the algorithm are given in the Metzler and Croft, 2007 paper which is available on line.
The getDistance() method is clear with me.
The paper also gives the notion of distance between two models.
However, how does getDistance(weight, regVector) give us regularization?
(The regVector is a uniform vector. [1/num_feats .. ... .. 1/num_feats].)
The weight vector is initialized as you state, but it changes over the iterations and restarts. The distance combined with slack parameter is somewhat analagous to an L2 regularization.
This regularization process isn't really defined in the original Metzler and Croft paper, so may have been experimental or just added by the implementer. The idea appears to be to penalize weights that vary too much from uniform to reduce the likelihood of over-fitting.
Perhaps I'll hear back from the implementer of this algorithm that will more definitively state the intent of this code, but it appears to me to be a form of regularization.
Looks like this is the use of slack variables, more commonly applied to soft margin classification, where one adds/subtracts values to allow some errors, by moving weights to where they belong as costs/losses, all for the purpose of producing a more generalized model.