Mar 31, 2015

Solution to Student Puzzle Corner 8

Anirban DasGupta writes:

Well done to Yixin Wang (Columbia University) and Vivian Meng (McGill University), who provided correct solutions to the various parts of the problem.

Yixin Wang Vivian Meng

The problem asked was the following: suppose $X_1, X_2, \cdots , X_n$ are iid $N(\mu , \sigma ^2)$ where the mean $\mu $ is some unknown positive integer, and $\sigma $ is a completely unknown standard deviation. We are to find the unique MLE of $\mu $ and $\sigma ^2$, and show that the MLE of $\mu $ converges to the true $\mu $ exponentially fast and that the MLE of $\sigma ^2$ is consistent.

Denote the likelihood function by $l(\mu , \sigma )$. Then, directly, whenever $\bar{x} \geq \frac{1}{2}$, \[ l(\mu + 1, \sigma ) \geq l(\mu , \sigma ) \Leftrightarrow \mu \leq [\bar{x} – \frac{1}{2}], \] where $[z]$ stands for the integer part of $z$. Thus, for $\bar{x} \geq \frac{1}{2}$, the unique MLE of $\mu $ is $1+[\bar{x} – \frac{1}{2}] = [\bar{x} + \frac{1}{2}]$; this is the same as the integer closest to the unrestricted MLE $\bar{X}$, a very intuitive result. If $\bar{X} < \frac{1}{2}$, $l(\mu , \sigma )$ is monotone decreasing in $\mu $ over the set of positive integers, and in that case the MLE of $\mu $ is $1$. By a standard calculus argument, once we have found the MLE $\hat{\mu }$ of $\mu $, the MLE of $\sigma ^2$ is $\hat{\sigma ^2} = \frac{1}{n}\sum_{i = 1}^n (X_i - \hat{\mu })^2$. To complete the solution, if the true $\mu > 1$, \[ P_\mu (\hat{\mu } = \mu ) = P_\mu (\mu – \frac{1}{2} \leq \bar{X} \leq \mu + \frac{1}{2}).\]

The complementary probability, $P_\mu (\hat{\mu } \neq \mu )$ therefore is $2[1-\Phi (\frac{\sqrt{n}}{2\sigma})]$, which is of the order of $\frac{e^{-\frac{n}{8\sigma ^2}}}{\sqrt{n}}$.

This is a (somewhat faster than) exponential rate. The argument for the case $\mu = 1$ is exactly similar; the expression just needs a very small modification. Obviously, therefore, by the Borel-Cantelli lemma, with probability one, for all large $n, \hat{\mu } = \mu $. This means, the MLE of $\sigma ^2$ is in fact
even strongly consistent, because $\frac{1}{n}\sum_{i = 1}^n (X_i – \mu )^2$ converges a.s. to $\sigma ^2$ by the usual SLLN.

Share

Leave a comment

*

Share

Welcome!

Welcome to the IMS Bulletin website! We are developing the way we communicate news and information more effectively with members. The print Bulletin is still with us (free with IMS membership), and still available as a PDF to download, but in addition, we are placing some of the news, columns and articles on this blog site, which will allow you the opportunity to interact more. We are always keen to hear from IMS members, and encourage you to write articles and reports that other IMS members would find interesting. Contact the IMS Bulletin at bulletin@imstat.org

What is “Open Forum”?

In the Open Forum, any IMS member can propose a topic for discussion. Email your subject and an opening paragraph (to bulletin@imstat.org) and we'll post it to start off the discussion. Other readers can join in the debate by commenting on the post. Search other Open Forum posts by using the Open Forum category link below. Start a discussion today!

About IMS

The Institute of Mathematical Statistics is an international scholarly society devoted to the development and dissemination of the theory and applications of statistics and probability. We have about 4,500 members around the world. Visit IMS at http://imstat.org
Latest Issue