If you have to estimate a distribution for which you have no information other than the possible outcomes which one should you use? The uniform distribution has the least bias towards a particular outcome. It may not be right but if you don't know, it's a reasonable default.
When you have trainging data there are many distributions which meet the criteria specified by the training data. Which one should you choose? The idea, again, is that you should choose the distribution which has the least bias with respect to items that you have no information about. The distribution which is closest wrt to Kulback Libner distance to the uniform distribution is this one.
If you know something about your distribution of outcomes then you can choose the distribution which is closest to that distribution and matches the criteria specified by the training data. But if you don't know anything about that distribution, then maxent suggests that the uniform distribution is the best choice. Hope this helps...Tom
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Why the most uniform (the most uncertainty, the highest entropy) distribution is good?
If you have to estimate a distribution for which you have no information other than the possible outcomes which one should you use? The uniform distribution has the least bias towards a particular outcome. It may not be right but if you don't know, it's a reasonable default.
When you have trainging data there are many distributions which meet the criteria specified by the training data. Which one should you choose? The idea, again, is that you should choose the distribution which has the least bias with respect to items that you have no information about. The distribution which is closest wrt to Kulback Libner distance to the uniform distribution is this one.
If you know something about your distribution of outcomes then you can choose the distribution which is closest to that distribution and matches the criteria specified by the training data. But if you don't know anything about that distribution, then maxent suggests that the uniform distribution is the best choice. Hope this helps...Tom