Multivariate Normal Distribution

I. Definition of Multivariate Normal Density

X is a (p x 1) normally distributed random vector

with mean E[X] and Covariance matrix C(X,X).

Denoted X ~ Normal ( E[X], Cov(X,X) )

The probability density for X, denoted f(X), is defined by

G² = (X-E[X])'Cov(X,X)^-1(X-E[X])

c = (2pi)]^p/2Det[Cov(X,X)]^1/2

f(X) = exp[ -G² / 2 ] / c

Alternative Expression for G²

Note: If a is a m x 1 vector and b is n x 1 vector, and X is a m x n matrix

a'Xb = Tr[ (ba')X ]

G² = Tr[(X-E[X]) (X-E[X])'Cov(X,X)^-1]

Also note Tr[A+B] = Tr[A]+Tr[B]

Thus for N independent observations:

G² = N× Tr{S (X-E[X]) (X-E[X])'/N} × Cov(X,X)^-1}

II. Distribution of a linear combination of a Multivariate Random Vector

X is multivariate normal, B is a fixed set of coefficients

Y = BX

Y is also multivariate normal

mean: E[Y] = B E[X]

Cov: C(Y,Y) = B Cov(X,X) B'

Proofs

III. Rotatation of Axes to orthogonal coordinates

Cov(X,X) = VDV', with V'V = VV' = I, and D = Diag[ d₁, …, d_p]

Define Z = D^-1/2V'(X-E[X]),

Then E[Z] = 0, Cov(Z,Z) = I.

f(Z) = exp[ -(Z'Z)/2 ] / (2pi)^p/2

G² = Z' Z and G² is chi - square distributed with df = p.

The contour of constant density is determined by the points

X = E[X] + (VD^1/2)Z, such that Z'Z = G².

IV. Multivariate Central Limit Theorem

X₁ X₂ … X_n are statistically independent random vectors

from a common distribution

with mean E[X] and covariance Cov(X,X).

Define M = (X₁ + …+X_n)/n

Sqrt( n)[ M - E(X) ] is approximately Normal( 0, Cov(X,X) )

n(M-E[X])'Cov(X,X)^-1(M-E[X]) is approximately Chi - Square with df = p.