As a recent field transferee from chemist to data scientist, I find myself wading through more matrix multiplication than I'm used to. I did some linear algebra way back, but I struggle with identifying 'which way' (i.e. inner or outer product) a given matrix multiplication is going.
I feel like there's some sort of convention with vector multiplication where the vector $\mathbf X$ is treated as a column vector, and $\mathbf X^T \mathbf X$ and $\mathbf X\mathbf X^T$ are the inner and outer product of $\mathbf X$ respectively. Is this true in all cases? Are there any tricks or mnemonics to help me keep track of the product of a string of such multiplications?
EDIT: There's a comment stating that it is common to assume that vectors are treated as column matrices. How common? Does it vary between disciplines? How likely (given that i'm looking through Wikipedia and stackexchange, not centuries-old manuscripts) am I to encounter the converse scenario?
$\endgroup$ 31 Answer
$\begingroup$From experience, column vectors aren't capitalized. Notes from wiki.
$X^{T}X $ and $XX^{T} $ denote the covariance matrix of $X \in \mathbb{C}^{m \times n}$
where as,
$ x^{t}x = \langle x, x \rangle = \| x\| $ is in the inner product and the outer product is given by $ x \otimes x = xx^{t} = \begin{bmatrix} x_{1} \\ x_{2} \\ x_{3} \end{bmatrix} \begin{bmatrix} x_{1} & x_{2} & x_{3} \end{bmatrix} = \begin{bmatrix} x_{1}x_{1} & x_{1}x_{2} & x_{1}x_{3} \\ x_{2}x_{1} & x_{2}x_{2} & x_{2}x_{3}\\ x_{3}x_{1} & x_{3}x_{2} & x_{3}x_{3}\end{bmatrix} $
Of course the outer product is for larger vectors as well i.e.
$$ u = (u_{1},u_{2}, \cdots,u_{m})\\ v= (v_{1},v_{2}, \cdots,v_{n}) $$
$$ u \otimes v = A = \begin{bmatrix} u_{1}v_{1} & u_{1}v_{2} & \cdots & u_{1}v_{n} \\ u_{2}v_{1} & u_{2}v_{2} & \cdots & u_{2}v_{n} \\ \vdots & \vdots & \ddots & \vdots \\ u_{m}v_{1} & u_{m}v_{2} & \cdots & u_{m}v_{n} \end{bmatrix} $$
$\endgroup$
