ADAM : A METHOD FOR STOCHASTIC OPTIMIZATION

文章目录

概
主要内容
- 算法
- 选择合适的参数
- 一些别的优化算法
- AdaMax
- 理论
代码

Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[J]. arXiv: Learning, 2014.

@article{kingma2014adam:,
title={Adam: A Method for Stochastic Optimization},
author={Kingma, Diederik P and Ba, Jimmy},
journal={arXiv: Learning},
year={2014}}

概

鼎鼎大名.

主要内容

用f(θ)f(\theta)f(θ)表示目标函数, 随机最优通常需要最小化E(f(θ))\mathbb{E}(f(\theta))E(f(θ)), 但是因为每一次我们都取的是一个小批次, 故实际上我们处理的是f1(θ),…,fT(θ)f_1(\theta),\ldots, f_T(\theta)f1(θ),…,fT(θ). 用gt=∇θft(θ)g_t=\nabla_{\theta}f_t(\theta)gt=∇θft(θ)表示第ttt步对应的梯度.

Adam 方法分别估计梯度E(gt)\mathbb{E}(g_t)E(gt)的一阶矩和二阶矩(Adam: adaptive moment estimation 名字的由来).

算法

注意: 下面的算法中关于向量的运算都是逐项(element-wise)的运算.

选择合适的参数

首先, 分析为什么会有
m^t←mt/(1−β2t),v^t←vt/(1−β2t).(A.1)\tag{A.1} \hat{m}_t \leftarrow m_t / (1-\beta_2^t), \\ \hat{v}_t \leftarrow v_t / (1-\beta_2^t). m^t←mt/(1−β2t),v^t←vt/(1−β2t).(A.1)

可以用归纳法证明
mt=(1−β1)∑i=1tβ1t−i⋅givt=(1−β2)∑i=1tβ2t−i⋅gi2.(A.2)\tag{A.2} m_t = (1-\beta_1) \sum_{i=1}^t \beta_1^{t-i} \cdot g_i \\ v_t = (1-\beta_2) \sum_{i=1}^t \beta_2^{t-i} \cdot g_i^2. mt=(1−β1)i=1∑tβ1t−i⋅givt=(1−β2)i=1∑tβ2t−i⋅gi2.(A.2)
倘若分布稳定: E[gt]=E[g],E[gt2]=E[g2]\mathbb{E}[g_t]=\mathbb{E}[g],\mathbb{E}[g_t^2]=\mathbb{E}[g^2]E[gt]=E[g],E[gt2]=E[g2], 则
E[mt]=E[g]⋅(1−β1t)E[vt]=E[g2]⋅(1−β2t).(A.3)\tag{A.3} \mathbb{E}[m_t]=\mathbb{E}[g] \cdot(1-\beta_1^t) \\ \mathbb{E}[v_t]= \mathbb{E}[g^2] \cdot (1- \beta_2^t). E[mt]=E[g]⋅(1−β1t)E[vt]=E[g2]⋅(1−β2t).(A.3)
这就是为什么会有(A.1)这一步.

Adam提出时的一个很大的应用场景就是dropout(正对梯度是稀疏的情况), 这是往往需要我们取较大的β2\beta_2β2(可理解为抵消随机因素).

既然E[g]/E[g2]≤1\mathbb{E}[g]/\sqrt{\mathbb{E}[g^2]}\le 1E[g]/E[g2]≤1, 我们可以把步长α\alphaα理解为一个信赖域(既然∣Δt∣<≈a|\Delta_t| \frac{<}{\approx} a∣Δt∣≈<a).

另外一个很重要的性质是, 比如函数扩大(或缩小)ccc倍cfcfcf, 此时梯度相应为cgcgcg, 我们所对应的
c⋅m^tc2⋅v^t=m^tv^t,\frac{c \cdot \hat{m}_t}{\sqrt{c^2 \cdot \hat{v}_t}}= \frac{\hat{m}_t}{\sqrt{\hat{v}_t}}, c2⋅v^tc⋅m^t=v^tm^t,
并无变化.

一些别的优化算法

AdaGrad:

θt+1=θt−α⋅1∑i=1tgt2+ϵgt.\theta_{t+1} = \theta_t -\alpha \cdot \frac{1}{\sqrt{\sum_{i=1}^tg_t^2}+\epsilon} g_t. θt+1=θt−α⋅∑i=1tgt2+ϵ1gt.

RMSprop:

vt=β2vt−1+(1−β2)gt2θt+1=θt−α⋅1vt+ϵgt.v_t = \beta_2 v_{t-1} + (1-\beta_2) g_t^2 \\ \theta_{t+1} = \theta_t -\alpha \cdot \frac{1}{\sqrt{v_t+\epsilon}}g_t. vt=β2vt−1+(1−β2)gt2θt+1=θt−α⋅vt+ϵ1gt.

AdaDelta:

vt=β2vt−1+(1−β2)gt2θt+1=θt−α⋅mt−1+ϵvt+ϵgtmt=β1mt−1+(1−β1)[θt+1−θt]2.v_t = \beta_2 v_{t-1} + (1-\beta_2) g_t^2 \\ \theta_{t+1} = \theta_t -\alpha \cdot \frac{\sqrt{m_{t-1}+\epsilon}}{\sqrt{v_t+\epsilon}}g_t \\ m_t = \beta_1 m_{t-1}+(1-\beta_1)[\theta_{t+1}-\theta_t]^2. vt=β2vt−1+(1−β2)gt2θt+1=θt−α⋅vt+ϵmt−1+ϵgtmt=β1mt−1+(1−β1)[θt+1−θt]2.

注: 均为逐项

AdaMax

本文还提出了另外一种算法

理论

不想谈了, 感觉证明有好多错误.

代码

import numpy as npclass Adam:def __init__(self, instance, alpha=0.001, beta1=0.9, beta2=0.999,epsilon=1e-8, beta_decay=1., alpha_decay=False):""" the Adam using numpy:param instance: the theta in paper, should have the grad method to call the gradsand the zero_grad method for clearing the grads:param alpha: the same as the paper default:0.001:param beta1: the same as the paper default:0.9:param beta2: the same as the paper default:0.999:param epsilon: the same as the paper default:1e-8:param beta_decay::param alpha_decay: default False, if True, we will set alpha = alpha / sqrt(t)"""self.instance = instanceself.alpha = alphaself.beta1 = beta1self.beta2 = beta2self.epsilon = epsilonself.beta_decay = beta_decayself.alpha_decay = alpha_decayself.initialize_paras()def initialize_paras(self):self.m = 0.self.v = 0.self.timestep = 0def update_paras(self):grads = self.instance.gradself.beta1 *= self.beta_decayself.beta2 *= self.beta_decayself.m = self.beta1 * self.m + (1 - self.beta1) * gradsself.v = self.beta2 * self.v + (1 - self.beta2) * grads ** 2self.timestep += 1if self.alpha_decay:return self.alpha / np.sqrt(self.timestep)return self.alphadef zero_grad(self):self.instance.zero_grad()def step(self):alpha = self.update_paras()betat1 = 1 - self.beta1 ** self.timestepbetat2 = 1 - self.beta2 ** self.timesteptemp = alpha * np.sqrt(betat2) / betat1self.instance.parameters -= temp * self.m / (np.sqrt(self.v) + self.epsilon)class PPP:def __init__(self, parameters, grad_func):self.parameters = parametersself.zero_grad()self.grad_func = grad_funcdef zero_grad(self):self.grad = np.zeros_like(self.parameters)def calc_grad(self):self.grad += self.grad_func(self.parameters)def f(x):return x[0] ** 2 + 5 * x[1] ** 2def grad(x):return np.array([2 * x[0], 100 * x[1]])if __name__ == "__main__":x = np.array([10., 10.])x = PPP(x, grad)xs = []ys = []optim = Adam(x, alpha=0.4)for i in range(100):xs.append(x.parameters.copy())y = f(x.parameters)ys.append(y)optim.zero_grad()x.calc_grad()optim.step()xs = np.array(xs)ys = np.array(ys)import matplotlib.pyplot as pltfig, (ax0, ax1)= plt.subplots(1, 2)ax0.plot(xs[:, 0], xs[:, 1])ax0.scatter(xs[:, 0], xs[:, 1])ax0.set(title="trajectory", xlabel="x", ylabel="y")ax1.plot(np.arange(len(ys)), ys)ax1.set(title="loss-iterations", xlabel="iterations", ylabel="loss")plt.show()