讨论内积与外积之前，首先定义两个基本列向量：
u = [ u 1 u 2 u 3 ] , v = [ v 1 v 2 v 3 ] \mathbf{u} = \left[ \begin{matrix} u_1 \\ u_2 \\ u_3 \end{matrix} \right] ,\quad \mathbf{v} = \left[ \begin{matrix} v_1 \\ v_2 \\ v_3 \end{matrix} \right]u=⎣⎡u1u2u3⎦⎤,v=⎣⎡v1v2v3⎦⎤
- Inner product 内积
u T v = [ u 1 , u 2 , u 3 ] [ v 1 v 2 v 3 ] = u 1 v 1 + u 2 v 2 + u 3 v 3 \mathbf{u}^T\mathbf{v} =\left[ \begin{matrix} u_1, u_2, u_3 \end{matrix} \right] \left[ \begin{matrix} v_1 \\ v_2 \\ v_3 \end{matrix} \right] =u_1v_1+u_2v_2+u_3v_3uTv=[u1,u2,u3]⎣⎡v1v2v3⎦⎤=u1v1+u2v2+u3v3
内积与范数紧密相关。
KaTeX parse error: Undefined control sequence: \norm at position 2: \̲n̲o̲r̲m̲{\mathbf{u}}=(\…
- 向量外积
u v T = [ u 1 u 2 u 3 ] [ v 1 , v 2 , v 3 ] = [ u 1 v 1 u 1 v 2 u 1 v 3 u 2 v 1 u 2 v 2 u 2 v 3 u 3 v 1 u 3 v 2 u 3 v 3 ] \mathbf{u}\mathbf{v}^T =\left[ \begin{matrix} u_1\\ u_2\\u_3 \end{matrix} \right] \left[ \begin{matrix} v_1,v_2,v_3 \end{matrix} \right] =\left[ \begin{matrix} u_1v_1 & u_1v_2 & u_1v_3 \\ u_2v_1 & u_2v_2 & u_2v_3 \\ u_3v_1 & u_3v_2 & u_3v_3 \end{matrix} \right]uvT=⎣⎡u1u2u3⎦⎤[v1,v2,v3]=⎣⎡u1v1u2v1u3v1u1v2u2v2u3v2u1v3u2v3u3v3⎦⎤
介绍完向量之间的inner & outer product 之后，我们来讨论矩阵的内积和外积。同样地，首先定义两个基本矩阵。
U = [ u 11 u 12 u 13 u 21 u 22 u 23 u 31 u 32 u 33 ] , V = [ v 11 v 12 v 13 v 21 v 22 v 23 v 31 v 32 v 33 ] \mathbf{U} = \left[ \begin{matrix} u_{11} & u_{12} & u_{13} \\ u_{21} & u_{22} & u_{23} \\ u_{31} & u_{32} & u_{33} \end{matrix} \right], \quad \mathbf{V} = \left[ \begin{matrix} v_{11} & v_{12} & v_{13} \\ v_{21} & v_{22} & v_{23} \\ v_{31} & v_{32} & v_{33} \end{matrix} \right]U=⎣⎡u11u21u31u12u22u32u13u23u33⎦⎤,V=⎣⎡v11v21v31v12v22v32v13v23v33⎦⎤
矩阵内积
n p . i n n e r ( U , V ) = n p . i n n e r ( V , U ) T = [ u 11 v 11 + u 12 v 12 + u 13 v 13 u 11 v 21 + u 12 v 22 + u 13 v 23 u 11 v 31 + u 12 v 31 + u 13 v 33 u 21 v 11 + u 22 v 12 + u 23 v 13 u 21 v 21 + u 22 v 22 + u 23 v 23 u 21 v 31 + u 22 v 32 + u 23 v 33 u 31 v 11 + u 32 v 12 + u 33 v 13 u 31 v 21 + u 32 v 22 + u 33 v 23 u 31 v 31 + u 32 v 32 + u 33 v 33 ] \begin{aligned} np.inner(U, V) = np.inner(V, U)^T\\ =\left[\begin{matrix} u_{11}v_{11}+u_{12}v_{12}+u_{13}v_{13} & u_{11}v_{21}+u_{12}v_{22}+u_{13}v_{23} & u_{11}v_{31}+u_{12}v_{31}+u_{13}v_{33} \\ u_{21}v_{11}+u_{22}v_{12}+u_{23}v_{13} & u_{21}v_{21}+u_{22}v_{22}+u_{23}v_{23} & u_{21}v_{31}+u_{22}v_{32}+u_{23}v_{33} \\ u_{31}v_{11}+u_{32}v_{12}+u_{33}v_{13} & u_{31}v_{21}+u_{32}v_{22}+u_{33}v_{23} & u_{31}v_{31}+u_{32}v_{32}+u_{33}v_{33} \end{matrix}\right] \end{aligned}np.inner(U,V)=np.inner(V,U)T=⎣⎡u11v11+u12v12+u13v13u21v11+u22v12+u23v13u31v11+u32v12+u33v13u11v21+u12v22+u13v23u21v21+u22v22+u23v23u31v21+u32v22+u33v23u11v31+u12v31+u13v33u21v31+u22v32+u23v33u31v31+u32v32+u33v33⎦⎤example
n p . i n n e r ( U , V ) = [ 1 2 3 4 ] [ 5 6 7 8 ] = [ 17 23 39 53 ] \begin{aligned} np.inner(U, V) &=\left[\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix}\right] \left[\begin{matrix} 5 & 6 \\ 7 & 8 \end{matrix}\right]\\ &=\left[\begin{matrix} 17 & 23 \\ 39 & 53 \end{matrix}\right] \end{aligned}np.inner(U,V)=[1324][5768]=[17392353]矩阵外积
矩阵外积就是矩阵的升维运算，将两个矩阵拉成向量形式，再按向量的外积运算。如mxn维矩阵与pxq维矩阵的外积维度为(mn)x(pq). 两个2x2的矩阵外积维度为(2x2)x(2x2)=4x4.
例如
n p . o u t e r ( U , V ) = [ 1 2 3 4 ] [ 5 6 7 8 ] = [ 1 2 3 4 ] [ 5 6 7 8 ] = [ 5 6 7 8 10 12 14 16 15 16 21 24 20 24 28 32 ] \begin{aligned} np.outer(U, V) &=\left[\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix}\right] \left[\begin{matrix} 5 & 6 \\ 7 & 8 \end{matrix}\right]\\ &=\left[\begin{matrix} 1 \\ 2 \\ 3 \\ 4 \end{matrix}\right] \left[\begin{matrix} 5 & 6 & 7 & 8 \end{matrix}\right]\\ &=\left[\begin{matrix} 5 & 6 & 7 & 8 \\ 10 & 12 & 14 & 16 \\ 15 & 16 & 21 & 24 \\ 20 & 24 & 28 & 32 \\ \end{matrix}\right] \end{aligned}np.outer(U,V)=[1324][5768]=⎣⎢⎢⎡1234⎦⎥⎥⎤[5678]=⎣⎢⎢⎡5101520612162471421288162432⎦⎥⎥⎤
注意：n p . o u t e r ( b , a ) = n p . o u t e r ( a , b ) T np.outer(b,a) = np.outer(a,b)^Tnp.outer(b,a)=np.outer(a,b)T
- 补充1-点乘 dot product
点乘表示普通的矩阵惩罚 - example
n p . i n n e r ( U , V ) = [ 1 2 3 4 ] [ 5 6 7 8 ] = [ 19 22 43 50 ] \begin{aligned} np.inner(U, V) &=\left[\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix}\right] \left[\begin{matrix} 5 & 6 \\ 7 & 8 \end{matrix}\right]\\ &=\left[\begin{matrix} 19 & 22 \\ 43 & 50 \end{matrix}\right] \end{aligned}np.inner(U,V)=[1324][5768]=[19432250] - 补充2-元素（乘积）Hadamard product / Element-wise multiplication / Element-wise product /Point-wise product
元素乘积与矩阵加法类似，使用对应元素相乘即可。因此要求两个乘数矩阵的维度一致。 - example
n p . m u l t i p l y ( U , V ) = [ 1 2 3 4 ] [ 5 6 7 8 ] = [ 5 12 21 32 ] \begin{aligned} np.multiply(U, V) &=\left[\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix}\right] \left[\begin{matrix} 5 & 6 \\ 7 & 8 \end{matrix}\right]\\ &=\left[\begin{matrix} 5 & 12 \\ 21 & 32 \end{matrix}\right] \end{aligned}np.multiply(U,V)=[1324][5768]=[5211232] - 笔者注：在深度学习图像处理领域使用元素乘积的频率远高于点乘。卷积运算就是先求元素乘积再对元素乘积的结果求和。