I want to work with data more over than 20G and i tried but it failed in a system with 16G Ram.
how can i handle it?is there any way to add incrementally data for training ?
I used LambdaMart.
Thanks
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Sorry for the late response. Unfortunately, RankLib doesn't have the incremental training feature at the moment. I'll add that sometime in the future.
One thing you can do for now is to code that function yourself. Since you're only interested in LambdaMART, this should be simple. Go to the LambdaMART.init() class, add some hard-codes on loading some model trained on partial data (using the existing load() function in the same class). You can even paramaterize the model file using a static variable, which could be specified via cmd-line.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi guys
I want to work with data more over than 20G and i tried but it failed in a system with 16G Ram.
how can i handle it?is there any way to add incrementally data for training ?
I used LambdaMart.
Thanks
Reza,
Sorry for the late response. Unfortunately, RankLib doesn't have the incremental training feature at the moment. I'll add that sometime in the future.
One thing you can do for now is to code that function yourself. Since you're only interested in LambdaMART, this should be simple. Go to the LambdaMART.init() class, add some hard-codes on loading some model trained on partial data (using the existing load() function in the same class). You can even paramaterize the model file using a static variable, which could be specified via cmd-line.