Expectations
Definitions
- Expected value: $E[X] = \int x f(x) dx$
- Variance (spread): $\text{Var}(X) = E[(X-E[X])^2]$
- Covariance (relationship): $\text{Cov}(X, Y) = E[(X - E[X])(Y - E[Y])] $
- Correlation: $\text{Corr}(X, Y) = \text{Cov}(X, Y)/(\sigma_x \sigma_y) $
Properties
Rule of the lazy statistician. Let $Y=r(X)$. Then \(E[Y] = E[r(X)] = \int r(x) f_{x}(x) dx.\)
For random variables $X_1, X_2, …, X_n$ and constants $a_i$,
- Sums: $E[\sum a_i X_i] = \sum a_i E[X_i]$
- Products: (if $X_i$ independent) $ E[\prod X_i] = \prod E[X_i] $
- Sum of variances: (if $X_i$ independent) $ Var(\sum X_i) = \sum Var(X_i)$
- $Var(X) = E[X^2] + E[X]^2$
- $Var(aX+b) = a^2 Var(X)$
- $Cov(X,Y) = E[XY] - E[X]E[Y]$
- Sum of variances: (general case) $ Var(\sum X_i) = \sum Var(X_i) + 2 \sum \sum_{i < j} Cov(X_i, X_j)$
Conditionals
- $E[X \mid Y=y] = \int x f_{X \mid Y}(x \mid y) dx$
- Note that while $E[X]$ is a number, $E[X \mid Y]$ is a random variable whose value is $E[X \mid Y=y]$ when $Y=y$.
- Iterated expectations: $ E[E[X \mid Y]] = E[X] $
- $Var(X \mid Y=y) = \int (x - E[X \mid Y=y])^2 f_{X \mid Y}(x \mid y) dx $
- $Var(X) = E[Var(X \mid Y)] + Var(E[X \mid Y])$