Given that X and Y are random variables show that:
$$E[E[X \mid Y]] = E[X]$$
I was thinking that I could use the definition of expected value (the summation one) to solve this, but when I tried I hit a wall. Thanks.
$\endgroup$2 Answers
$\begingroup$This is the Law of Total Expectation. The proof is as follows:
$$ \begin{align} E[E[X|Y]] &= E \left[ \sum_{x} x \cdot P(X = x | Y) \right] \\ &= \sum_y \left[ \sum_{x} x \cdot P(X = x | Y = y) \right] P(Y = y)\\ &= \sum_x x \sum_y P(X = x | Y = y) \cdot P(Y = y) \\ &= \sum_x x \sum_y P(X = x \, \text{and} \, Y = y) \\ &= \sum_x x \cdot P(X = x)\\ &= E[X] \end{align} $$
The non-discrete is a bit tougher I think. The Wikipedia link should help you out here.
$\endgroup$ $\begingroup$The continuous case is similar. Let us specify the Law of Total Expectation (also called Tower Property) more precisely:
$$E_Y(E_X[X|Y])=E_X[X]$$
where $E_Y$ is the expectation w.r.t. $Y$ and $E_X$ w.r.t. $X$. Let $S_X=supp(X)$ and $S_Y=supp(Y)$$$E_Y(E_X[X|Y])=\int_{S_Y} E_X[X|Y]\; f(y)\; dy$$$$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; \frac{f(x,y)}{f(y)}dx\; f(y)\; dy$$$f(y)$ is constant w.r.t. $X$. Thus,$$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; f(x,y)\;dx\; \frac{f(y)}{f(y)}\; dy$$$$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; f(x,y)\;dx\; dy$$By Fubini's theorem$$E_Y(E_X[X|Y])=\int_{S_X}\int_{S_Y}x\; f(x,y)\;dy\; dx$$Now $X$ is constant w.r.t. $Y$.$$E_Y(E_X[X|Y])=\int_{S_X}x\; \int_{S_Y} f(x,y)\;dy\; dx$$
The inner integral is the marginalisation from $f(x,y)$ to $f(x)$. Thus,$$E_Y(E_X[X|Y])=\int_{S_X}x\; f(x)\; dx$$This is just the definition of the expectation. Thus,$$E_Y(E_X[X|Y])=E_X[X]$$
I hope this helps. This proof also exists in a measure theoretic context which is more general. The Wikipedia article is quite good.
$\endgroup$ 0
