I see a lot of implementation of maximum entropy. Can someone explain why it is possible to estimate the parameter based on maximum entropy?
$\begingroup$ 
$\endgroup$
3 
- 3$\begingroup$ Answers to this question aren't likely to be helpful or constructive. It is possible because entropy is bounded above and almost always is a continuous function of the parameter. Thus, by varying the parameter withing a bounded region you can always find a maximum. A more penetrating avenue of investigation would be to ask what desirable properties a maximum-entropy estimator might have. $\endgroup$whuber– whuber ♦2017-05-12 21:42:37 +00:00Commented May 12, 2017 at 21:42
- 2$\begingroup$ I want to get some sense on maximum entropy. As you mentioned, the entropy is bounded above. And it's common to use the maximum entropy as good criteria to estimate the hidden parameters. Can you explain the reason why it is reasonable to pick up estimated parameter by virtue of reaching the peak on entropy plot? In other words, how to connect the maximum-entropy to other easily understood criteria together, like maximum likelihood estimation? Thx $\endgroup$Sirius Hou– Sirius Hou2017-05-13 02:58:38 +00:00Commented May 13, 2017 at 2:58
- $\begingroup$ Some ideas here: stats.stackexchange.com/questions/66186/… $\endgroup$kjetil b halvorsen– kjetil b halvorsen ♦2017-06-05 22:58:55 +00:00Commented Jun 5, 2017 at 22:58
Add a comment |