什么是Hessian矩阵和Jacobian矩阵

Jacobian矩阵可以看作是函数一阶导数的推广

假如一个函数$f:\mathbb{R}^n\rightarrow \mathbb{R}^m$，那$f$的Jacobian矩阵就是

$$J=\begin{pmatrix}\frac{\partial f_1}{\partial x_1} & \cdots & \frac{\partial f_1}{\partial x_n}\\ \vdots & \ddots & \vdots \\ \frac{\partial f_m}{\partial x_1} & \cdots & \frac{\partial f_m}{\partial x_n} \end{pmatrix}$$

Hessian矩阵可以看作是函数二阶矩阵的推广

假如一个函数$f:\mathbb{R}^n\rightarrow \mathbb{R}$，那$f$的Hessian矩阵就是

$$H=\left[\frac{\partial^2 f}{\partial x_i x_j}\right]_{i,j}=\begin{pmatrix}\frac{\partial^2 f}{\partial x_1\partial x_1} & \frac{\partial^2 f}{\partial x_1\partial x_2} & \cdots & \frac{\partial^2 f}{\partial x_1x_n}\\ \frac{\partial^2 f}{\partial x_2\partial x_1} & \frac{\partial^2 f}{\partial x_2\partial x_2}  & \cdots & \frac{\partial^2 f}{\partial x_2x_n}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial x_n x_1} &  \frac{\partial^2 f}{\partial x_n x_2} & \cdots & \frac{\partial^2 f}{\partial x_n x_n} \end{pmatrix}$$