SVM优化方法--SMO优化算法（Sequential minimal optimization）

Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines

SMO算法由Microsoft Research的John C. Platt在1998年提出，并成为最快的二次规划优化算法，特别针对线性SVM和数据稀疏时性能更优。

This paper proposes a new algorithm for training support vector machines: Sequential Minimal Optimization, or SMO. Training a support vector machine requires the solution of a very large quadratic programming (QP) optimization problem. SMO breaks this large QP problem into a series of smallest possible QP problems. These small QP problems are solved analytically, which avoids using a time-consuming numerical QP optimization as an inner loop. The amount of memory required for SMO is linear in the training set size, which allows SMO to handle very large training sets. Because matrix computation is avoided, SMO scales somewhere between linear and quadratic in the training set size for various test problems, while the standard chunking SVM algorithm scales somewhere between linear and cubic in the training set size. SMO’s computation time is dominated by
SVM evaluation, hence SMO is fastest for linear SVMs and sparse data sets. On realworld sparse data sets, SMO can be more than 1000 times faster than the chunking algorithm.

Heuristics for Choosing Which Multipliers To Optimize
As long as SMO always optimizes and alters two Lagrange multipliers at every step and at least one of the Lagrange multipliers violated the KKT conditions before the step, then each step will decrease the objective function according to Osuna’s theorem [16]. Convergence is thus guaranteed. In order to speed convergence, SMO uses heuristics to choose which two Lagrange
multipliers to jointly optimize.

SVM优化方法--SMO优化算法（Sequential minimal optimization）相关推荐

SMO优化算法（Sequential minimal optimization）
转载地址:http://www.cnblogs.com/jerrylead/archive/2011/03/18/1988419.html 11 SMO优化算法(Sequential minimal ...
SMO（Sequential Minimal Optimization）算法的数学原理
SMO算法的数学原理前言 SVM问题回顾 SMO算法两个朗格朗日乘子的解析解选择哪两个乘子需要被优化的启发式方法计算阈值线性SVM的一个优化代码细节前言紧跟着SVM数学原理之后,又拿出 ...
SMO算法详细推导（Sequential Minimal Optimization）
本文针对一般性的"软判断的核函数的对偶问题的SVM",形如下式: 上式问题所在:当采样点 xix_ixi 选取50000个点时,则基于核函数变量Θ(xi,xj)\bm{\Thet ...
花书+吴恩达深度学习（七）优化方法之基本算法（Momentum, Nesterov, AdaGrad, RMSProp, Adam）
目录 0. 前言 1. 指数加权平均(exponentially weighted averages) 2. Momentum 动量 3. Nesterov 动量 4. AdaGrad 5. RMSP ...
Algorithm：机械优化设计的数学模型简介、常用优化方法、优化计算工具简介之详细攻略
Algorithm:机械优化设计的数学模型简介.常用优化方法.优化计算工具简介之详细攻略目录机械设计中基于算法模型的机械优化设计 1.优化设计的数学模型
Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines 论文研读
摘要本文提出了一种用于训练支持向量机的新算法:序列最小优化算法(SMO).训练支持向量机需要解决非常大的二次规划(QP)优化问题.SMO 将这个大的 QP 问题分解为一系列最小的 QP 问题.这些小 ...
45.JVM调优策略、常见问题：内存泄漏(年老代堆空间被占满、持久代被占满、堆栈溢出、线程堆栈满、系统内存被占满)优化方法：优化目标、优化GC步骤、优化总结；案例分析（公司系统参数、网上给的配置参数）
45.JVM调优策略 45.1.常见问题 45.1.1.内存泄漏 45.1.1.1.年老代堆空间被占满 45.1.1.2.持久代被占满 45.1.1.3.堆栈溢出 45.1.1.4.线程堆栈满 45. ...
【支持向量机SVM】关于SMO优化算法的笔记
看了几篇SVM的相关文章,还是没理解透彻...B站关于SMO算法的讲解很清楚,做了一些笔记,看看能不能写成程序...(不知道为什么图片是横着的)
mysql优化方法_mysql优化方案总结
u Mysql数据库的优化技术对mysql优化时一个综合性的技术,主要包括 a: 表的设计合理化(符合3NF) b: 添加适当索引(index) [四种: 普通索引.主键索引.唯一索引u ...

SVM优化方法--SMO优化算法（Sequential minimal optimization）

SVM优化方法--SMO优化算法（Sequential minimal optimization）相关推荐

最新文章

热门文章