笔记-最优控制理论1

最优控制问题：在满足系统方程的约束条件下，在容许控制域中确定一个最优控制律，使得系统状态从已知初态转移到要求的目标集，并使性能指标达到极值。

1）建立被控系统的状态方程
2）确定边界条件
3）选定性能指标
4）确定控制律的容许范围
5）按一定方法计算最优控制

庞特里亚金极小值原理、经典变分法

终端状态和终端时间分 2 × 3 2 \times 3 2×3类：

终端时间固定，终端状态固定、自由、约束。
– t f t_f tf给定， x ( t f ) \boldsymbol{x}(t_f) x(tf)固定，即 x ( t f ) = x t f \boldsymbol{x}(t_f) = \boldsymbol{x}_{t_f} x(tf)=xtf
– t f t_f tf给定， x ( t f ) \boldsymbol{x}(t_f) x(tf)自由，就是无约束
– t f t_f tf给定， x ( t f ) \boldsymbol{x}(t_f) x(tf)受约束，即 G [ x ( t f ) , t f ] = 0 \boldsymbol{G} \left[ \boldsymbol{x}(t_f), t_f \right] = 0 G[x(tf),tf]=0
终端时间自由，终端状态固定、自由、约束。
– t f t_f tf自由， x ( t f ) \boldsymbol{x}(t_f) x(tf)固定，即 x ( t f ) = x t f \boldsymbol{x}(t_f) = \boldsymbol{x}_{t_f} x(tf)=xtf
– t f t_f tf自由， x ( t f ) \boldsymbol{x}(t_f) x(tf)自由，就是无约束
– t f t_f tf自由， x ( t f ) \boldsymbol{x}(t_f) x(tf)受约束，即 G [ x ( t f ) , t f ] = 0 \boldsymbol{G} \left[ \boldsymbol{x}(t_f), t_f \right] = 0 G[x(tf),tf]=0

性能指标分3类：

\quad – 波尔扎(Bolza)型(综合指标)： J = ψ ( x ( t f ) , t f ) + ∫ t 0 t f L [ x ( t ) , u ( t ) , t ] d t J = \psi \left( \boldsymbol{x}(t_f), t_f \right) + \int_{t_0}^{t_f} {L \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] {\rm{d}} t} J=ψ(x(tf),tf)+∫t0tfL[x(t),u(t),t]dt

\quad – 迈耶尔(Mayer)型(终端指标)： J = ψ ( x ( t f ) , t f ) J = \psi \left( \boldsymbol{x}(t_f), t_f \right) J=ψ(x(tf),tf)

\quad – 拉格朗日(Lagrange)型(积分指标)： J = ∫ t 0 t f L [ x ( t ) , u ( t ) , t ] d t J = \int_{t_0}^{t_f} {L \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] {\rm{d}} t} J=∫t0tfL[x(t),u(t),t]dt

经典变分法

考虑动态系统：
x ˙ = f [ x ( t ) , u ( t ) , t ] (1) \dot{\boldsymbol{x}} = \boldsymbol{f} \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] \tag{1} x˙=f[x(t),u(t),t](1)

性能指标为：
J = ψ ( x ( t f ) , t f ) + ∫ t 0 t f L [ x ( t ) , u ( t ) , t ] d t (2) J = \psi \left( \boldsymbol{x}(t_f), t_f \right) + \int_{t_0}^{t_f} {L \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] {\rm{d}} t} \tag{2} J=ψ(x(tf),tf)+∫t0tfL[x(t),u(t),t]dt(2)

求最优控制 u ∗ ( t ) \boldsymbol{u}^*(t) u∗(t)和满足状态方程的极值轨线 x ∗ ( t ) \boldsymbol{x}^*(t) x∗(t)，使性能指标取极值。

1）终端时刻固定，终端状态自由（ t f t_f tf给定， x ( t f ) \boldsymbol{x}(t_f) x(tf)自由）

将状态方程写成等式约束方程的形式：
f [ x ( t ) , u ( t ) , t ] − x ˙ = 0 (3) \boldsymbol{f} \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] - \dot{\boldsymbol{x}} = 0 \tag{3} f[x(t),u(t),t]−x˙=0(3)

引入拉格朗日乘子 λ ( t ) \boldsymbol{\lambda}(t) λ(t)，又称为伴随变量、协态或共轭状态，增广泛函为：
J E = ψ ( x ( t f ) , t f ) + ∫ t 0 t f { L [ x ( t ) , u ( t ) , t ] + λ T ( t ) [ f [ x ( t ) , u ( t ) , t ] − x ˙ ( t ) ] } d t (4) J_E = \psi \left( \boldsymbol{x}(t_f), t_f \right) + \int_{t_0}^{t_f} { \left \{ L \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] + \boldsymbol{\lambda}^T(t) \left[ \boldsymbol{f} \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] - \dot{\boldsymbol{x}}(t) \right] \right \} {\rm{d}} t} \tag{4} JE=ψ(x(tf),tf)+∫t0tf{L[x(t),u(t),t]+λT(t)[f[x(t),u(t),t]−x˙(t)]}dt(4)

这样就将有约束的泛函 J J J极值问题转换为无约束的增广泛函 J E J_E JE极值问题。

引入哈密顿（Hamilton）函数：
H ( x ( t ) , u ( t ) , λ ( t ) , t ) = L [ x ( t ) , u ( t ) , t ] + λ T ( t ) f [ x ( t ) , u ( t ) , t ] (5) H(\boldsymbol{x}(t), \boldsymbol{u}(t), \boldsymbol{\lambda}(t), t) = L \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] + \boldsymbol{\lambda}^T(t) \boldsymbol{f} \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] \tag{5} H(x(t),u(t),λ(t),t)=L[x(t),u(t),t]+λT(t)f[x(t),u(t),t](5)

那么式 ( 4 ) (4) (4)可以表示为：
J E = ψ ( x ( t f ) , t f ) + ∫ t 0 t f [ H ( x ( t ) , u ( t ) , λ ( t ) , t ) − λ T ( t ) x ˙ ( t ) ] d t (6) J_E = \psi \left( \boldsymbol{x}(t_f), t_f \right) + \int_{t_0}^{t_f} { \left[ H(\boldsymbol{x}(t), \boldsymbol{u}(t), \boldsymbol{\lambda}(t), t) - \boldsymbol{\lambda}^T(t) \dot{\boldsymbol{x}}(t) \right] {\rm{d}} t} \tag{6} JE=ψ(x(tf),tf)+∫t0tf[H(x(t),u(t),λ(t),t)−λT(t)x˙(t)]dt(6)

注释1：没有给出各向量变量的维度，性能指标泛函是标量。

对式 ( 6 ) (6) (6)积分括号里的第二项 λ T ( t ) x ˙ ( t ) \boldsymbol{\lambda}^T(t) \dot{\boldsymbol{x}}(t) λT(t)x˙(t)做分部积分，可得：
J E = ψ ( x ( t f ) , t f ) − λ T ( t f ) x ( t f ) + λ T ( t 0 ) x ( t 0 ) + ∫ t 0 t f [ H ( x ( t ) , u ( t ) , λ ( t ) , t ) − λ ˙ T ( t ) x ( t ) ] d t (7) \begin{aligned} J_E &= \psi \left( \boldsymbol{x}(t_f), t_f \right) - \boldsymbol{\lambda}^T(t_f) \boldsymbol{x}(t_f) + \boldsymbol{\lambda}^T(t_0) \boldsymbol{x}(t_0) \\ &+ \int_{t_0}^{t_f} { \left[ H(\boldsymbol{x}(t), \boldsymbol{u}(t), \boldsymbol{\lambda}(t), t) - \dot{\boldsymbol{\lambda}}^T(t) \boldsymbol{x}(t) \right] {\rm{d}} t} \end{aligned} \tag{7} JE=ψ(x(tf),tf)−λT(tf)x(tf)+λT(t0)x(t0)+∫t0tf[H(x(t),u(t),λ(t),t)−λ˙T(t)x(t)]dt(7)

设 x ( t ) , u ( t ) \boldsymbol{x}(t), \boldsymbol{u}(t) x(t),u(t)相对于最优值 x ∗ ( t ) , u ∗ ( t ) \boldsymbol{x}^*(t), \boldsymbol{u}^*(t) x∗(t),u∗(t)的变分分别为 δ x ( t ) , δ u ( t ) \delta \boldsymbol{x}(t), \delta \boldsymbol{u}(t) δx(t),δu(t)，由于终端状态自由，还要考虑变分 δ x ( t f ) \delta \boldsymbol{x}(t_f) δx(tf)，则由这些变分引起的泛函变分为：
δ J E = δ x T ( t f ) ∂ ψ ∂ x ( t f ) − δ x T ( t f ) λ ( t f ) + ∫ t 0 t f [ δ x T ( ∂ H ∂ x + λ ˙ ) + δ u T ∂ H ∂ u ] d t (8) \begin{aligned} \delta{J_E} &= \delta \boldsymbol{x}^T (t_f) \frac{\partial{\psi}}{\partial{\boldsymbol{x}(t_f)}} - \delta \boldsymbol{x}^T(t_f) \boldsymbol{\lambda}(t_f) + \int_{t_0}^{t_f} { \left[ \delta \boldsymbol{x}^T \left( \frac{\partial{H}}{\partial{\boldsymbol{x}}} + \dot{\boldsymbol{\lambda}} \right) + \delta \boldsymbol{u}^T \frac{\partial{H}}{\partial{\boldsymbol{u}}} \right] {\rm{d}} t} \end{aligned} \tag{8} δJE=δxT(tf)∂x(tf)∂ψ−δxT(tf)λ(tf)+∫t0tf[δxT(∂x∂H+λ˙)+δuT∂u∂H]dt(8)

J E J_E JE为极值的必要条件是：对任意的 δ x , δ u , δ x ( t f ) \delta{\boldsymbol{x}}, \delta{\boldsymbol{u}}, \delta{\boldsymbol{x}(t_f)} δx,δu,δx(tf)，变分 δ J E = 0 \delta{J_E} = 0 δJE=0。

综述所述，可得：

状态方程： x ˙ = ∂ H ∂ λ = f [ x ( t ) , u ( t ) , t ] \dot{\boldsymbol{x}} = \frac{\partial{H}}{\partial{\boldsymbol{\lambda}}} = \boldsymbol{f} \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] x˙=∂λ∂H=f[x(t),u(t),t]

协态方程： λ ˙ = − ∂ H ∂ x \dot{\boldsymbol{\lambda}} = - \frac{\partial{H}}{\partial{\boldsymbol{x}}} λ˙=−∂x∂H

控制方程： ∂ H ∂ u = 0 \frac{\partial{H}}{\partial{\boldsymbol{u}}} = 0 ∂u∂H=0

横截条件： λ ( t f ) = ∂ ψ ∂ x ( t f ) \boldsymbol{\lambda}(t_f) = \frac{\partial{\psi}}{\partial{\boldsymbol{x}(t_f)}} λ(tf)=∂x(tf)∂ψ

横截条件表示协态终端所需满足的条件。当终端状态固定时， δ x ( t f ) = 0 \delta{\boldsymbol{x}(t_f)} = 0 δx(tf)=0，就不需要横截条件了。

注释2：状态方程+协态方程=正则方程

只知道初值 x ( t 0 ) \boldsymbol{x}(t_0) x(t0)和由横截条件确定的协态终端值 λ ( t f ) \boldsymbol{\lambda}(t_f) λ(tf)，即两点边值问题，一般很难求解。这是因为 λ ( t 0 ) \boldsymbol{\lambda}(t_0) λ(t0)未知，如果假定一个 λ ( t 0 ) \boldsymbol{\lambda}(t_0) λ(t0)，正向积分方程组，则在 t = t f t = t_f t=tf时的 λ \boldsymbol{\lambda} λ一般与给定的 λ ( t f ) \boldsymbol{\lambda}(t_f) λ(tf)不同，可反复修改 λ ( t 0 ) \boldsymbol{\lambda}(t_0) λ(t0)的值，直至 λ ( t f ) \boldsymbol{\lambda}(t_f) λ(tf)与给定值的差可以忽略不计为止。

2）终端时刻自由，终端状态受约束（ t f t_f tf自由， x ( t f ) \boldsymbol{x}(t_f) x(tf)属于一个约束集）

设终端状态满足约束方程：
G [ x ( t f ) , t f ] = 0 (9) \boldsymbol{G} \left[ \boldsymbol{x}(t_f), t_f \right] = 0 \tag{9} G[x(tf),tf]=0(9)

其中， G [ x ( t f ) , t f ] = [ G 1 ( x ( t f ) , t f ) , ⋯ , G q ( x ( t f ) , t f ) ] T \boldsymbol{G} \left[ \boldsymbol{x}(t_f), t_f \right] = \left[ G_1 \left( \boldsymbol{x}(t_f), t_f \right), \cdots , G_q \left( \boldsymbol{x}(t_f), t_f \right) \right]^T G[x(tf),tf]=[G1(x(tf),tf),⋯,Gq(x(tf),tf)]T

性能指标为：
J = ψ ( x ( t f ) , t f ) + ∫ t 0 t f L [ x ( t ) , u ( t ) , t ] d t J = \psi \left( \boldsymbol{x}(t_f), t_f \right) + \int_{t_0}^{t_f} {L \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] {\rm{d}} t} J=ψ(x(tf),tf)+∫t0tfL[x(t),u(t),t]dt

与上面例子类似，引入n维拉格朗日乘子向量 λ \boldsymbol{\lambda} λ和q维拉格朗日乘子向量 v \boldsymbol{v} v，做增广泛函：
J E = ψ ( x ( t f ) , t f ) + v T G ( x ( t f ) , t f ) + ∫ t 0 t f { L [ x ( t ) , u ( t ) , t ] + λ T ( t ) [ f [ x ( t ) , u ( t ) , t ] − x ˙ ( t ) ] } d t (10) J_E = \psi \left( \boldsymbol{x}(t_f), t_f \right) + \boldsymbol{v}^T \boldsymbol{G} \left( \boldsymbol{x}(t_f), t_f \right) \\ +\int_{t_0}^{t_f} { \left \{ L \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] + \boldsymbol{\lambda}^T(t) \left[ \boldsymbol{f} \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] - \dot{\boldsymbol{x}}(t) \right] \right \} {\rm{d}} t} \tag{10} JE=ψ(x(tf),tf)+vTG(x(tf),tf)+∫t0tf{L[x(t),u(t),t]+λT(t)[f[x(t),u(t),t]−x˙(t)]}dt(10)

引入哈密顿函数：
H ( x ( t ) , u ( t ) , λ ( t ) , t ) = L [ x ( t ) , u ( t ) , t ] + λ T ( t ) f [ x ( t ) , u ( t ) , t ] (11) H(\boldsymbol{x}(t), \boldsymbol{u}(t), \boldsymbol{\lambda}(t), t) = L \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] + \boldsymbol{\lambda}^T(t) \boldsymbol{f} \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] \tag{11} H(x(t),u(t),λ(t),t)=L[x(t),u(t),t]+λT(t)f[x(t),u(t),t](11)

令
Θ ( x ( t f ) , t f ) = ψ ( x ( t f ) , t f ) + v T G ( x ( t f ) , t f ) (12) \boldsymbol{\Theta} (\boldsymbol{x}(t_f), t_f) = \psi \left( \boldsymbol{x}(t_f), t_f \right) + \boldsymbol{v}^T \boldsymbol{G} \left( \boldsymbol{x}(t_f), t_f \right) \tag{12} Θ(x(tf),tf)=ψ(x(tf),tf)+vTG(x(tf),tf)(12)

式 ( 10 ) (10) (10)可以表示为：
J E = Θ ( x ( t f ) , t f ) + ∫ t 0 t f [ H ( x ( t ) , u ( t ) , λ ( t ) , t ) − λ T ( t ) x ˙ ( t ) ] d t (13) J_E = \boldsymbol{\Theta} (\boldsymbol{x}(t_f), t_f) + \int_{t_0}^{t_f} { \left[ H(\boldsymbol{x}(t), \boldsymbol{u}(t), \boldsymbol{\lambda}(t), t) - \boldsymbol{\lambda}^T(t) \dot{\boldsymbol{x}}(t) \right] {\rm{d}} t} \tag{13} JE=Θ(x(tf),tf)+∫t0tf[H(x(t),u(t),λ(t),t)−λT(t)x˙(t)]dt(13)

与 t f t_f tf固定时不同的地方在于，现在 δ J E \delta J_E δJE由 δ x ( t ) , δ u ( t ) , δ x ( t f ) , δ t f \delta \boldsymbol{x}(t), \delta \boldsymbol{u}(t), \delta \boldsymbol{x}(t_f), \delta{t_f} δx(t),δu(t),δx(tf),δtf所引起。这里， δ t f \delta{t_f} δtf不再为0，并且：
t f = t f ∗ + δ t f δ x ( t f ) = x ( t f ) − x ∗ ( t f ∗ ) = δ x ( t f ∗ ) + [ x ( t f ∗ + δ t f ) − x ( t f ∗ ) ] ≈ δ x ( t f ∗ ) + x ˙ ( t f ∗ ) δ t f \begin{aligned} t_f &= t_f^* + \delta{t_f} \\ \delta \boldsymbol{x} (t_f) &= \boldsymbol{x} (t_f) - \boldsymbol{x}^* (t_f^*) = \delta \boldsymbol{x} (t_f^*) + \left[ \boldsymbol{x} (t_f^* + \delta t_f) - \boldsymbol{x} (t_f^*) \right] \\ &\approx \delta \boldsymbol{x} (t_f^*) + \dot{\boldsymbol{x}} (t_f^*) \delta t_f \end{aligned} tfδx(tf)=tf∗+δtf=x(tf)−x∗(tf∗)=δx(tf∗)+[x(tf∗+δtf)−x(tf∗)]≈δx(tf∗)+x˙(tf∗)δtf

计算 J E J_E JE的变分（只计算至一阶小量）：
Δ J E = Θ ∗ ( x ( t f ) + δ x ( t f ) , t f + δ t f ) + ∫ t 0 t f ∗ + δ t f [ H ( x + δ x , u + δ u , λ , t ) − λ T ( x ˙ + δ x ˙ ) ] ∗ d t − Θ ( x ( t f ) , t f ) ∗ − ∫ t 0 t f ∗ [ H ( x , u , λ , t ) − λ T x ˙ ] ∗ d t \Delta J_E = \boldsymbol{\Theta}^* (\boldsymbol{x}(t_f) + \delta \boldsymbol{x}(t_f), t_f + \delta t_f) + \int_{t_0}^{t_f^* + \delta t_f} { \left[ H(\boldsymbol{x} + \delta \boldsymbol{x}, \boldsymbol{u} + \delta \boldsymbol{u}, \boldsymbol{\lambda}, t) - \boldsymbol{\lambda}^T(\dot{\boldsymbol{x}} + \delta \dot{\boldsymbol{x}}) \right]^* {\rm{d}} t} \\ - \boldsymbol{\Theta} \left( \boldsymbol{x}(t_f), t_f \right)^* - \int_{t_0}^{t_f^*} { \left[ H(\boldsymbol{x}, \boldsymbol{u}, \boldsymbol{\lambda}, t) - \boldsymbol{\lambda}^T \dot{\boldsymbol{x}} \right]^* {\rm{d}} t} ΔJE=Θ∗(x(tf)+δx(tf),tf+δtf)+∫t0tf∗+δtf[H(x+δx,u+δu,λ,t)−λT(x˙+δx˙)]∗dt−Θ(x(tf),tf)∗−∫t0tf∗[H(x,u,λ,t)−λTx˙]∗dt

将上式线性化：
δ J E = [ ∂ Θ ∂ x ( t f ) ] ∗ δ x ( t f ) + [ ∂ Θ ∂ t f ] ∗ δ t f + ∫ t 0 t f ∗ [ ( ∂ H ∂ u ) T δ x + ( ∂ H ∂ u ) T δ u − λ T δ x ˙ ] ∗ d t + ∫ t f ∗ t f ∗ + δ t f [ H ( x + δ x , u + δ u , λ , t ) − λ T ( x ˙ + δ x ˙ ) ] ∗ d t (14) \delta J_E = \left[ \frac{\partial{\boldsymbol{\Theta}}}{\partial{\boldsymbol{x}}(t_f)} \right]^* \delta \boldsymbol{x}(t_f) + \left[ \frac{\partial{\boldsymbol{\Theta}}}{\partial{t_f}} \right]^* \delta t_f + \int_{t_0}^{t_f^*} \left[ \left( \frac{\partial H}{\partial \boldsymbol{u}} \right)^T \delta \boldsymbol{x} + \left( \frac{\partial H}{\partial \boldsymbol{u}} \right)^T \delta \boldsymbol{u} - \boldsymbol{\lambda}^T \delta \dot{\boldsymbol{x}} \right]^* \text{d} t \\ +\int_{t_f^*}^{t_f^* + \delta t_f} { \left[ H(\boldsymbol{x} + \delta \boldsymbol{x}, \boldsymbol{u} + \delta \boldsymbol{u}, \boldsymbol{\lambda}, t) - \boldsymbol{\lambda}^T(\dot{\boldsymbol{x}} + \delta \dot{\boldsymbol{x}}) \right]^* {\rm{d}} t} \tag{14} δJE=[∂x(tf)∂Θ]∗δx(tf)+[∂tf∂Θ]∗δtf+∫t0tf∗[(∂u∂H)Tδx+(∂u∂H)Tδu−λTδx˙]∗dt+∫tf∗tf∗+δtf[H(x+δx,u+δu,λ,t)−λT(x˙+δx˙)]∗dt(14)

针对式 ( 14 ) (14) (14)，对于 ∫ t 0 t f ∗ [ ( ∂ H ∂ u ) T δ x + ( ∂ H ∂ u ) T δ u − λ T δ x ˙ ] ∗ d t \int_{t_0}^{t_f^*} \left[ \left( \frac{\partial H}{\partial \boldsymbol{u}} \right)^T \delta \boldsymbol{x} + \left( \frac{\partial H}{\partial \boldsymbol{u}} \right)^T \delta \boldsymbol{u} - \boldsymbol{\lambda}^T \delta \dot{\boldsymbol{x}} \right]^* \text{d} t ∫t0tf∗[(∂u∂H)Tδx+(∂u∂H)Tδu−λTδx˙]∗dt这一项，采用分部积分法，可化为：
∫ t 0 t f ∗ [ ( ∂ H ∂ u ) T δ x + ( ∂ H ∂ u ) T δ u − λ T δ x ˙ ] ∗ d t ⇓ ∫ t 0 t f ∗ [ ( ∂ H ∂ u + λ ˙ ) T δ x + ( ∂ H ∂ u ) T δ u ] ∗ d t − λ T ( t f ∗ ) δ x ( t f ∗ ) \int_{t_0}^{t_f^*} \left[ \left( \frac{\partial H}{\partial \boldsymbol{u}} \right)^T \delta \boldsymbol{x} + \left( \frac{\partial H}{\partial \boldsymbol{u}} \right)^T \delta \boldsymbol{u} - \boldsymbol{\lambda}^T \delta \dot{\boldsymbol{x}} \right]^* \text{d} t \\ \Downarrow \\ \int_{t_0}^{t_f^*} \left[ \left( \frac{\partial H}{\partial \boldsymbol{u}} + \dot{\boldsymbol{\lambda}} \right)^T \delta \boldsymbol{x} + \left( \frac{\partial H}{\partial \boldsymbol{u}} \right)^T \delta \boldsymbol{u} \right]^* \text{d} t - \boldsymbol{\lambda}^T (t_f^*) \delta \boldsymbol{x} (t_f^*) ∫t0tf∗[(∂u∂H)Tδx+(∂u∂H)Tδu−λTδx˙]∗dt⇓∫t0tf∗[(∂u∂H+λ˙)Tδx+(∂u∂H)Tδu]∗dt−λT(tf∗)δx(tf∗)

对于 ∫ t f ∗ t f ∗ + δ t f [ H ( x + δ x , u + δ u , λ , t ) − λ T ( x ˙ + δ x ˙ ) ] ∗ d t \int_{t_f^*}^{t_f^* + \delta t_f} { \left[ H(\boldsymbol{x} + \delta \boldsymbol{x}, \boldsymbol{u} + \delta \boldsymbol{u}, \boldsymbol{\lambda}, t) - \boldsymbol{\lambda}^T(\dot{\boldsymbol{x}} + \delta \dot{\boldsymbol{x}}) \right]^* {\rm{d}} t} ∫tf∗tf∗+δtf[H(x+δx,u+δu,λ,t)−λT(x˙+δx˙)]∗dt这一项，忽略二阶小量，可化为：
∫ t f ∗ t f ∗ + δ t f [ H ( x + δ x , u + δ u , λ , t ) − λ T ( x ˙ + δ x ˙ ) ] ∗ d t ⇓ 一阶展开 ≈ ∫ t f ∗ t f ∗ + δ t f [ H ( x , u , λ , t ) + ( ∂ H ∂ x ) T δ x + ( ∂ H ∂ u ) T δ u − λ T x ˙ − λ T δ x ˙ ] ∗ d t ⇓ 忽略小量 ≈ H ∗ ( x , u , λ , t ) δ t f − λ T ( t f ∗ ) x ˙ ( t f ∗ ) δ t f ⇓ 合并整理 = H ∗ δ t f − λ T ( t f ∗ ) [ δ x ( t f ) − δ x ( t f ∗ ) ] \int_{t_f^*}^{t_f^* + \delta t_f} { \left[ H(\boldsymbol{x} + \delta \boldsymbol{x}, \boldsymbol{u} + \delta \boldsymbol{u}, \boldsymbol{\lambda}, t) - \boldsymbol{\lambda}^T(\dot{\boldsymbol{x}} + \delta \dot{\boldsymbol{x}}) \right]^* {\rm{d}} t} \\ \Downarrow \text{一阶展开} \\ \approx \int_{t_f^*}^{t_f^* + \delta t_f} { \left[ H(\boldsymbol{x}, \boldsymbol{u}, \boldsymbol{\lambda}, t) + \left( \frac{\partial H}{\partial \boldsymbol{x}} \right)^T \delta \boldsymbol{x} + \left( \frac{\partial H}{\partial \boldsymbol{u}} \right)^T \delta \boldsymbol{u} - \boldsymbol{\lambda}^T \dot{\boldsymbol{x}} - \boldsymbol{\lambda}^T \delta \dot{\boldsymbol{x}} \right]^* {\rm{d}} t} \\ \Downarrow \text{忽略小量} \\ \approx H^* (\boldsymbol{x}, \boldsymbol{u},\boldsymbol{\lambda}, t) \delta t_f - \boldsymbol{\lambda}^T (t_f^*) \dot{\boldsymbol{x}}(t_f^*) \delta t_f \\ \Downarrow \text{合并整理} \\ = H^* \delta t_f - \boldsymbol{\lambda}^T (t_f^*) \left[ \delta \boldsymbol{x}(t_f) - \delta \boldsymbol{x}(t_f^*) \right] ∫tf∗tf∗+δtf[H(x+δx,u+δu,λ,t)−λT(x˙+δx˙)]∗dt⇓一阶展开≈∫tf∗tf∗+δtf[H(x,u,λ,t)+(∂x∂H)Tδx+(∂u∂H)Tδu−λTx˙−λTδx˙]∗dt⇓忽略小量≈H∗(x,u,λ,t)δtf−λT(tf∗)x˙(tf∗)δtf⇓合并整理=H∗δtf−λT(tf∗)[δx(tf)−δx(tf∗)]

因此，式 ( 14 ) (14) (14)可以化为：
δ J E = [ ∂ Θ ∂ x ( t f ) ] ∗ δ x ( t f ) − λ T ( t f ∗ ) δ x ( t f ) + [ ∂ Θ ∂ t f ] ∗ δ t f + H ∗ δ t f + ∫ t 0 t f ∗ [ ( ∂ H ∂ x + λ ˙ ) T δ x + ( ∂ H ∂ u ) T δ u ] ∗ d t (15) \delta J_E = \left[ \frac{\partial{\boldsymbol{\Theta}}}{\partial{\boldsymbol{x}}(t_f)} \right]^* \delta \boldsymbol{x}(t_f) - \boldsymbol{\lambda}^T (t_f^*) \delta \boldsymbol{x} (t_f) + \left[ \frac{\partial{\boldsymbol{\Theta}}}{\partial{t_f}} \right]^* \delta t_f + H^* \delta t_f \\ + \int_{t_0}^{t_f^*} \left[ \left( \frac{\partial H}{\partial \boldsymbol{x}} + \dot{\boldsymbol{\lambda}} \right)^T \delta \boldsymbol{x} + \left( \frac{\partial H}{\partial \boldsymbol{u}} \right)^T \delta \boldsymbol{u} \right]^* \text{d} t \tag{15} δJE=[∂x(tf)∂Θ]∗δx(tf)−λT(tf∗)δx(tf)+[∂tf∂Θ]∗δtf+H∗δtf+∫t0tf∗[(∂x∂H+λ˙)Tδx+(∂u∂H)Tδu]∗dt(15)

J E J_E JE取极值的必要条件是：对任意的 δ x , δ u , δ x ( t f ) , δ t f \delta{\boldsymbol{x}}, \delta{\boldsymbol{u}}, \delta{\boldsymbol{x}(t_f)}, \delta t_f δx,δu,δx(tf),δtf，变分 δ J E = 0 \delta{J_E} = 0 δJE=0，也就是说：

状态方程： x ˙ = ∂ H ∂ λ = f [ x ( t ) , u ( t ) , t ] \dot{\boldsymbol{x}} = \frac{\partial{H}}{\partial{\boldsymbol{\lambda}}} = \boldsymbol{f} \left[ \boldsymbol{x}(t), \boldsymbol{u}(t), t \right] x˙=∂λ∂H=f[x(t),u(t),t]

协态方程： λ ˙ = − ∂ H ∂ x \dot{\boldsymbol{\lambda}} = - \frac{\partial{H}}{\partial{\boldsymbol{x}}} λ˙=−∂x∂H

控制方程： ∂ H ∂ u = 0 \frac{\partial{H}}{\partial{\boldsymbol{u}}} = 0 ∂u∂H=0

横截条件： { λ ( t f ) = ∂ Θ ∂ x ( t f ) = ∂ ψ ∂ x ( t f ) + ∂ G T ∂ x ( t f ) v H ( t f ) = − ∂ Θ ∂ t f = − ∂ ψ ∂ t f − ∂ G T ∂ t f v \left \{ \begin{aligned} \boldsymbol{\lambda}(t_f) &= \frac{\partial \boldsymbol{\Theta}}{\partial{\boldsymbol{x}(t_f)}} = \frac{\partial{\psi}}{\partial{\boldsymbol{x}(t_f)}} + \frac{\partial \boldsymbol{G}^T}{\partial{\boldsymbol{x}(t_f)}} \boldsymbol{v} \\ H(t_f) &= - \frac{\partial \boldsymbol{\Theta}}{\partial{t_f}} = - \frac{\partial{\psi}}{\partial{t_f}} - \frac{\partial \boldsymbol{G}^T}{\partial{t_f}} \boldsymbol{v} \end{aligned} \right. ⎩⎪⎪⎪⎨⎪⎪⎪⎧λ(tf)H(tf)=∂x(tf)∂Θ=∂x(tf)∂ψ+∂x(tf)∂GTv=−∂tf∂Θ=−∂tf∂ψ−∂tf∂GTv

终端时刻自由时，多了一个方程 H ( t f ) = − ∂ Θ ∂ t f = − ∂ ψ ∂ t f − ∂ G T ∂ t f v H(t_f) = - \frac{\partial \boldsymbol{\Theta}}{\partial{t_f}} = - \frac{\partial{\psi}}{\partial{t_f}} - \frac{\partial \boldsymbol{G}^T}{\partial{t_f}} \boldsymbol{v} H(tf)=−∂tf∂Θ=−∂tf∂ψ−∂tf∂GTv，用于求出最优终端时间。

注释3：哈密顿函数和拉格朗日函数的区别在于，拉格朗日函数多了一项 x ˙ \dot{\boldsymbol{x}} x˙，暂时可以这么理解。

3）终端时间自由/固定，终端状态自由/固定

。。。。。。见下，自行推导。

参考文献

罗亚中, 张进, 杨震. 航天系统优化方法自编讲义-第四章.
最优控制原理课件.