Random Variables and Expectation

I. Sample Statistics

X = (n x p) data matrix with n rows of subjects and p measurements.

J = (n x n) matrix of all ones.

Sample mean matrix:

M = (JX)/n

m_jk = (x_1k + x_2k + … + x_nk)/n = m_k

Sample Covariance Matrix:

S = (X-M)'(X-M)/(n-1)

s_jk = [(x_1j - m_j)(x_1k - m_k) + (x_2j - m_j)(x_2k - m_k)+

…+ (x_nj - m_j)(x_nk - m_k)]/(n-1)

Sample Correlation Matrix:

s_k = Sqrt(s_kk)

D = Diag[ s₁ , s₂, …, s_p ]

R = D^-1SD^-1

r_jk = s_jk / s_j s_k

Mean and Variance of a Linear Transformation

Define X as a n x p data matrix , n = no. obs, p = no. var's

Construct a new set of variables

Y = XB

M_x = JX/n

S_x = (X-M_x)'(X-M_x)/(n-1)

Then

M_y = JY/n = J(XB)/n = (JX/n)B = M_xB

S_y = (Y-M_y)'(Y-M_y)/(n-1)

= (XB - M_xB)'(XB - M_xB)/(n-1)

= [(X-M)B]'[(X-M_x)B]/(n-1)

= B'(X-M_x)'(X-M_x)B/(n-1)

= B'S_xB

II. Expectation

X is a random variable that can be assigned one of the values

x₁ , x₂, …x_j , …x_M

Pr[X = x_j ] = probability that r.v. X takes on the value x_j .

Mean of X:

E[X] = x₁Pr[X=x₁] + x₂Pr[X=x₂] +

… + x_MPr[X=x_M]

Two rules of expectation for random variables X and Y :

E[c X] = cE[X]

E[ X + Y ] = E[X] + E[Y]

Random Vectors X and Y.

X' =[X₁ X₂ … X_p ] = (1 x p) row vector of p random variables

Y' =[Y₁ Y₂ … Y_q ] = (1 x q) row vector of q random variables

E[X'] = [ E[X₁] E[X₂] … E[X_p] ] = E[X]'

Cov(X,Y) = E[ (X-E[X])(Y - E[Y])' ]

Two rules of expectation for random vectors X and Y:

E[C Y] = C E[Y]

Cov[ BX, CY] = BCov(X,Y)C'

Note: Same two rules apply to sample means and covariances.