Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Calculate Covariance Between Multinomial Categories by Hand

Let (X1,,Xk)Multinomial(n,p1,,pk)(X_1, \ldots, X_k)\sim\text{Multinomial}(n, p_1, \ldots, p_k). I’ll show how to calculate Cov(Xi,Xj),ijCov(X_i, X_j), i\ne j. The main purpose of doing this is to illustrate the little trick of reducing complexity of problems by decomposing a complicated random variable into simple ones.

Let (Yi1,,Yik)iidMultinomial(1,p1,,pk),i1(Y_{i1}, \ldots, Y_{ik})\overset{iid}{\sim}\text{Multinomial}(1, p_1, \ldots, p_k), i\ge1, then

Xj=i=1nYij,1jk.X_j = \sum_{i=1}^n Y_{ij}, 1\le j \le k.

So,

Cov(Xi,Xj)=Cov(l=1nYli,m=1nYmj)=l=1nm=1nCov(Yli,Ymj)=m=1nCov(Ymi,Ymj)=nCov(Y1i,Y1j)=n(EY1iY1jEY1iEY1j)=npipj\begin{align} Cov(X_i,X_j) &= Cov(\sum_{l=1}^n Y_{li}, \sum_{m=1}^n Y_{mj})\nonumber\newline &= \sum_{l=1}^n\sum_{m=1}^n Cov(Y_{li}, Y_{mj})\nonumber\newline &= \sum_{m=1}^n Cov(Y_{mi}, Y_{mj})\nonumber\newline &= nCov(Y_{1i}, Y_{1j}) = n(EY_{1i}Y_{1j} - EY_{1i}EY_{1j})\nonumber\newline &= -np_ip_j\nonumber\newline \end{align}