evaluation - Expectation of a family of random variables

I did some search here, but seems that no one asked about this.

I want to define a family of independent and identically distributed random variables, $x_1,...,x_n$, and then calculates the expected value of some expressions like $\sum_{i,j=1}^nx_ix_j$. The result would be some function depending on $n$. Is there a way to do this?

Answer

Here is a general solution for any distribution whose moments exist ...

Notation Define the power sum $s_r$:

$$s_r=\sum _{i=1}^n X_i^r$$

The Problem

Let $\left(X_1,\ldots,X_n\right)$ denote $n$ iid random variables. This is the same problem as drawing a random sample of size $n$ from a population random variable $X$. The problem is to find:

$$E\Big(\sum_{i,j=1}^n X_i X_j\Big) = E\Big [\Big (\sum_{i=1}^n X_i\Big)^2\Big ] = E\Big [s_1^2\Big]$$

This is a problem known as finding moments of moments: they can be very difficult to solve by hand, but quite easy to solve with the help of a computer algebra system, for any arbitrary symmetric power sum. In this instance, we seek the expectation of $s_1^2$ ... i.e. the 1st raw moment of $s_1^2$ ... so the solution (expressed ToRaw moments of the population) is:

where RawMomentToRaw is a function from the mathStatica package for Mathematica, and where $\acute{\mu }_1$ and $\acute{\mu }_2$ denote the 1st and 2nd raw moments of random variable $X$, whatever its distribution (assuming they exist). All done.

More detail

There is an extensive discussion of moments of moments in Chapter 7 of our book:

• Rose and Smith, "Mathematical Statistics with Mathematica", Springer, NY

A free download of the chapter is available here:

http://www.mathstatica.com/book/Rose_and_Smith_2002edition_Chapter7.pdf

---

Example 1: The Normal Distribution

If $X \sim N(\mu, \sigma^2)$, then: $$\acute{\mu }_1 = E[X] = \mu \quad \text{ and } \quad \acute{\mu }_2 = E[X^2] = \mu^2 + \sigma^2$$

Substituting in $\acute{\mu }_1$ and $\acute{\mu }_2$ in Out[1] yields the solution: $$E\Big(\sum_{i,j=1}^n X_i X_j\Big) = n \left(n \mu ^2 + \sigma ^2\right)$$

Simple check: The Normal case with $n = 3$

In the case of $n = 3$, the joint pdf of $(X_1, X_2, X_3)$ is say $f(x_1, x_2, x_3)$:

The sum of products we are interested in is:

and the desired expectation is:

which matches perfectly the general $n$-Normal solution derived above, but with $n = 3$.

---

Example 2: The Uniform Distribution

If $X \sim Uniform(a,b)$ (as considered in both other answers), then: $$\acute{\mu }_1 = E[X] = \frac{a+b}{2} \quad \text{ and } \quad \acute{\mu }_2 = E[X^2] = \frac{1}{3} \left(a^2+a b+b^2\right)$$

Substituting in $\acute{\mu }_1$ and $\acute{\mu }_2$ in Out[1] yields the solution: $$E\Big(\sum_{i,j=1}^n X_i X_j\Big) =\frac{1}{3} n \left(a^2+a b+b^2\right)+\frac{1}{4} (n-1) n (a+b)^2$$

Again, this is different to the other answers posted - and much more complicated. Again, it is easy to perform a quick check:

Simple check: The Uniform case with $n = 3$

In the case of $n = 3$, the joint pdf of $(X_1, X_2, X_3)$ is say $g(x_1, x_2, x_3)$:

and the desired expectation is:

which matches perfectly our general $n$-Uniform solution derived above, with $n = 3$.