一、概述

AutoML在近年来的各类机器学习和Kaggle比赛中层出不穷，明显是机器学习的一个趋势，自动化机器学习提供了方法和流程，使机器学习可供非机器学习专家使用，以提高机器学习的效率并加速机器学习的研究。

FLAML是今年由微软主推的一个全新的高效轻量级自动化机器学习框架。

FLAML 是一个轻量级的 Python 库，可自动、高效且经济地找到准确的机器学习模型。它使用户不必为每个学习者选择学习者和超参数。

对于分类和回归等常见的机器学习任务，它可以快速找到计算资源较少的用户提供数据的质量模型。它支持经典机器学习模型和深度神经网络。

它很容易定制或扩展。用户可以从一个平滑的范围内找到他们想要的可定制性：最小定制（计算资源预算）、中等定制（例如，scikit 风格的学习器、搜索空间和度量）或完全定制（任意训练和评估代码）。

它支持快速自动调优，能够处理复杂的约束/指导/提前停止。FLAML 由 Microsoft Research 发明的一种新的、具有成本效益的超参数优化和学习器选择方法提供支持。

另外FLAML 还具有来自Visual Studio 2022中的ML.NET Model Builder的.NET 实现。

二、安装和使用简介

1、安装

pip install flaml

2、使用简介

号称使用三行三码就可以进行训练，代码风格与scikit-learn一致。

from flaml import AutoML
automl = AutoML()
automl.fit(X_train, y_train, task="classification")

也可以使用指定使用XGBoost、LightGBM、随机森林等的快速超参数调整工具或自定义学习器。

automl.fit(X_train, y_train, task="classification", estimator_list=["lgbm"])

详情见官方github。

https://github.com/microsoft/FLAMLhttps://github.com/microsoft/FLAML

三、使用案例

这里基于kaggle的2022年3月的表格游乐场比赛，进行测试。

from flaml import AutoML
from sklearn import svm
from sklearn.model_selection import train_test_split
import numpy as np
import pandas as pd
import pickle
from sklearn.linear_model import LinearRegression
from sklearn.linear_model import Ridge
from sklearn.svm import SVR
from sklearn.ensemble import StackingRegressor
from sklearn.ensemble import RandomForestRegressor,ExtraTreesRegressor
from sklearn.experimental import enable_hist_gradient_boosting
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.linear_model import LassoCV
from sklearn.linear_model import RidgeCV# 读取训练和测试数据
train = pd.read_csv('child/train_action.csv', index_col='row_id')
test = pd.read_csv('child/test_action.csv', index_col='row_id')# 进行一些基础的特征工程
def feature_engineering(data):data['time'] = pd.to_datetime(data['time'])data['month'] = data['time'].dt.monthdata['weekday'] = data['time'].dt.weekdaydata['hour'] = data['time'].dt.hourdata['minute'] = data['time'].dt.minutedata['is_month_start'] = data['time'].dt.is_month_start.astype('int')data['is_month_end'] = data['time'].dt.is_month_end.astype('int')data['hour+minute'] = data['time'].dt.hour * 60 + data['time'].dt.minutedata['is_weekend'] = (data['time'].dt.dayofweek > 4).astype('int')data['is_afternoon'] = (data['time'].dt.hour > 12).astype('int')data['x+y'] = data['x'].astype('str') + data['y'].astype('str')data['x+y+direction'] = data['x'].astype('str') + data['y'].astype('str') + data['direction'].astype('str')data['hour+direction'] = data['hour'].astype('str') + data['direction'].astype('str')data['hour+x+y'] = data['hour'].astype('str') + data['x'].astype('str') + data['y'].astype('str')data['hour+direction+x'] = data['hour'].astype('str') + data['direction'].astype('str') + data['x'].astype('str')data['hour+direction+y'] = data['hour'].astype('str') + data['direction'].astype('str') + data['y'].astype('str')data['hour+direction+x+y'] = data['hour'].astype('str') + data['direction'].astype('str') + data['x'].astype('str') + data['y'].astype('str')data['hour+x'] = data['hour'].astype('str') + data['x'].astype('str')data['hour+y'] = data['hour'].astype('str') + data['y'].astype('str')data = data.drop(['time'], axis=1)return data# 改变类型以减少内存使用
def reduce_mem_usage(df):start_mem = df.memory_usage().sum() / 1024 ** 2print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))for col in df.columns:col_type = df[col].dtypeif col_type != object and col!= 'time':c_min = df[col].min()c_max = df[col].max()if str(col_type)[:3] == 'int':if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:df[col] = df[col].astype(np.int8)elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:df[col] = df[col].astype(np.int16)elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:df[col] = df[col].astype(np.int32)elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:df[col] = df[col].astype(np.int64)else:if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:df[col] = df[col].astype(np.float16)elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:df[col] = df[col].astype(np.float32)else:df[col] = df[col].astype(np.float64)else:df[col] = df[col].astype('category')end_mem = df.memory_usage().sum() / 1024 ** 2print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))return df# 组织数据
train = reduce_mem_usage(train)
test = reduce_mem_usage(test)for data in [train, test]:data = feature_engineering(data)train = reduce_mem_usage(train)
test = reduce_mem_usage(test)for data in [train, test]:data = feature_engineering(data)y = train['congestion']
del train['time']
#del train['time1']
del train['congestion']del test['time']
#del test['time1']
del test['pre']# 声明AutoML并进行训练
automl = AutoML()
automl.fit(train, y, task="regression") #, time_budget=-1，#这个参数是只搜索时间，-1为不限制# 保存模型
with open("automl_v2.pkl", "wb") as f:pickle.dump(automl, f, pickle.HIGHEST_PROTOCOL)# 加载模型并预测
with open("automl_v2.pkl", "rb") as f:automl = pickle.load(f)
predictions = automl.predict(test)# 保存到csv
preds = []
for pred in predictions:preds.append(pred)res = pd.DataFrame()
res['congestion'] = preds
res.to_csv("faml_v2.csv")

如果使用automl.fit(train, y, task="regression")，全部默认参数进行训练，一分钟就训练完成，把结果提交到kaggle得到了5.688分，虽然排名不咋地，但是就速度和结果同单独的比如随机森林等等还是非常可以的。

下面修改参数automl.fit(train, y, task="regression", time_budget=600)，表示会训练10分钟，-1为不限制时间，不建议为-1，否则运行的时间就非常长了。下面是10分钟后输出结果如下(太长了，保留了一小部分)，可以看出进行了若干种方法的搜索。

[flaml.automl: 03-16 16:47:29] {2068} INFO - task = regression
[flaml.automl: 03-16 16:47:29] {2070} INFO - Data split method: uniform
[flaml.automl: 03-16 16:47:29] {2074} INFO - Evaluation method: holdout
[flaml.automl: 03-16 16:47:29] {2155} INFO - Minimizing error metric: 1-r2
[flaml.automl: 03-16 16:47:30] {2248} INFO - List of ML learners in AutoML Run: ['lgbm', 'rf', 'catboost', 'xgboost', 'extra_tree', 'xgb_limitdepth']
[flaml.automl: 03-16 16:47:30] {2501} INFO - iteration 0, current learner lgbm
[flaml.automl: 03-16 16:47:30] {2617} INFO - Estimated sufficient time budget=50419s. Estimated necessary time budget=431s.
[flaml.automl: 03-16 16:47:30] {2669} INFO - at 3.5s,   estimator lgbm's best error=0.7083,   best estimator lgbm's best error=0.7083
[flaml.automl: 03-16 16:47:30] {2501} INFO - iteration 1, current learner lgbm
[flaml.automl: 03-16 16:47:30] {2669} INFO - at 3.5s,   estimator lgbm's best error=0.7083,   best estimator lgbm's best error=0.7083
[flaml.automl: 03-16 16:47:30] {2501} INFO - iteration 2, current learner lgbm
[flaml.automl: 03-16 16:47:30] {2669} INFO - at 3.6s,   estimator lgbm's best error=0.4913,   best estimator lgbm's best error=0.4913
[flaml.automl: 03-16 16:47:30] {2501} INFO - iteration 3, current learner xgboost

......

......

......
[flaml.automl: 03-16 16:55:37] {2669} INFO - at 490.4s,   estimator lgbm's best error=0.2688,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:55:37] {2501} INFO - iteration 140, current learner extra_tree
[flaml.automl: 03-16 16:55:37] {2669} INFO - at 490.9s,   estimator extra_tree's best error=0.3131,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:55:37] {2501} INFO - iteration 141, current learner xgboost
[flaml.automl: 03-16 16:55:42] {2669} INFO - at 496.0s,   estimator xgboost's best error=0.2914,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:55:42] {2501} INFO - iteration 142, current learner extra_tree
[flaml.automl: 03-16 16:55:43] {2669} INFO - at 496.6s,   estimator extra_tree's best error=0.3131,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:55:43] {2501} INFO - iteration 143, current learner extra_tree
[flaml.automl: 03-16 16:55:44] {2669} INFO - at 497.3s,   estimator extra_tree's best error=0.3113,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:55:44] {2501} INFO - iteration 144, current learner extra_tree
[flaml.automl: 03-16 16:55:44] {2669} INFO - at 497.8s,   estimator extra_tree's best error=0.3113,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:55:44] {2501} INFO - iteration 145, current learner extra_tree
[flaml.automl: 03-16 16:55:45] {2669} INFO - at 498.5s,   estimator extra_tree's best error=0.3043,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:55:45] {2501} INFO - iteration 146, current learner extra_tree
[flaml.automl: 03-16 16:55:46] {2669} INFO - at 499.2s,   estimator extra_tree's best error=0.3043,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:55:46] {2501} INFO - iteration 147, current learner lgbm
[flaml.automl: 03-16 16:55:49] {2669} INFO - at 502.8s,   estimator lgbm's best error=0.2688,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:55:49] {2501} INFO - iteration 148, current learner lgbm
[flaml.automl: 03-16 16:56:47] {2669} INFO - at 561.0s,   estimator lgbm's best error=0.2688,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:56:47] {2501} INFO - iteration 149, current learner lgbm
[flaml.automl: 03-16 16:56:51] {2669} INFO - at 564.2s,   estimator lgbm's best error=0.2688,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:56:51] {2501} INFO - iteration 150, current learner lgbm
[flaml.automl: 03-16 16:57:12] {2669} INFO - at 586.0s,   estimator lgbm's best error=0.2688,   best estimator lgbm's best error=0.2688
[flaml.automl: 03-16 16:57:12] {2501} INFO - iteration 151, current learner lgbm
[flaml.automl: 03-16 16:57:25] {2669} INFO - at 598.9s,   estimator lgbm's best error=0.2673,   best estimator lgbm's best error=0.2673
[flaml.automl: 03-16 16:57:25] {2501} INFO - iteration 152, current learner extra_tree
[flaml.automl: 03-16 16:57:25] {2669} INFO - at 599.0s,   estimator extra_tree's best error=0.3043,   best estimator lgbm's best error=0.2673
[flaml.automl: 03-16 16:57:25] {2501} INFO - iteration 153, current learner extra_tree
[flaml.automl: 03-16 16:57:26] {2669} INFO - at 599.2s,   estimator extra_tree's best error=0.3043,   best estimator lgbm's best error=0.2673
[flaml.automl: 03-16 16:57:26] {2501} INFO - iteration 154, current learner extra_tree
[flaml.automl: 03-16 16:57:26] {2669} INFO - at 599.3s,   estimator extra_tree's best error=0.3043,   best estimator lgbm's best error=0.2673
[flaml.automl: 03-16 16:57:26] {2501} INFO - iteration 155, current learner rf
[flaml.automl: 03-16 16:57:26] {2669} INFO - at 599.5s,   estimator rf's best error=0.3408,   best estimator lgbm's best error=0.2673
[flaml.automl: 03-16 16:57:39] {2895} INFO - retrain lgbm for 12.7s
[flaml.automl: 03-16 16:57:39] {2900} INFO - retrained model: LGBMRegressor(colsample_bytree=0.7603311183328791,
learning_rate=0.07676228628554725, max_bin=1023,
min_child_samples=12, n_estimators=233, num_leaves=634,
reg_alpha=0.17131266959954505, reg_lambda=0.0009765625,
verbose=-1)
[flaml.automl: 03-16 16:57:39] {2277} INFO - fit succeeded
[flaml.automl: 03-16 16:57:39] {2279} INFO - Time taken to find the best model: 598.8882813453674
[flaml.automl: 03-16 16:57:39] {2293} WARNING - Time taken to find the best model is 100% of the provided time budget and not all estimators' hyperparameter search converged. Consider increasing the time budget.

训练完成之后，会给出当前最佳结果的模型。

LGBMRegressor(colsample_bytree=0.7603311183328791,
learning_rate=0.07676228628554725, max_bin=1023,
min_child_samples=12, n_estimators=233, num_leaves=634,
reg_alpha=0.17131266959954505, reg_lambda=0.0009765625,
verbose=-1)

这次把结果提交到kaggle得到了5.304分，有所提高。

机器学习笔记 - AutoML框架FLAML初体验相关推荐

RPC 框架 Kitex 初体验 (虚拟机环境)
目录前提安装代码生成工具获取示例代码运行示例代码引用附加资料注脚前提按照 HTTP 框架 Hertz 初体验已经完成相关环境配置安装代码生成工具安装 kitex go inst ...
python爬虫框架排行榜-Python爬虫框架--pyspider初体验
之前接触scrapy本来是想也许scrapy能够让我的爬虫更快,但是也许是我没有掌握scrapy的要领,所以爬虫运行起来并没有我想象的那么快,看这篇文章就是之前使用scrapy的写得爬虫.然后昨天我又 ...
云原生 CI/CD 框架 Tekton 初体验
Tekton 是一款功能非常强大而灵活的 CI/CD 开源的云原生框架.Tekton 的前身是 Knative 项目的 build-pipeline 项目,这个项目是为了给 build 模块增加 pi ...
微博开源框架Motan初体验
前两天,我在开源中国的微信公众号看到新浪微博的轻量Rpc框架--Motan开源了.上网查了下,才得知这个Motan来头不小,支撑着新浪微博的千亿调用,曾经在2014年的春晚中有着千亿次的调用,对抗了春 ...
【语义分割 - DeepLab学习笔记】之DeepLab初体验
目录前言一.DeepLab简述二.DeepLab怎么使用 1.配置 2.代码解析前言最近在接触一个项目,要用到语义分割,如果大家有什么问题,都可以跟我沟通交流.除了通过博客交流外,欢迎你加入 ...
物联网学习笔记——京东云擎初体验部署REST服务
0.前言持续关注物联网,持续关注REST.CoAP.websocket在物联网中的应用.前些时间尝试使用PHP Slim框架搭建REST应用,但是REST服务部署在自己的电脑上,只能在局域网 ...
嵌入式编程笔记之六--设备树初体验
设备树的起源设备树(Device Tree)是一种描述硬件资源的数据结构,它由 uboot 传递给 Linux 内核,被内核解析,内核根据设备树中的硬件描述信息加载利用相应驱动资源.在引入设备树之前 ...
主流实时流处理计算框架Flink初体验
2022 年 5 月 1 日百思不得小赵
Net Core平台灵活简单的日志记录框架NLog+Mysql组合初体验
Net Core平台灵活简单的日志记录框架NLog初体验前几天分享的"[Net Core集成Exceptionless分布式日志功能以及全局异常过滤][https://www.cnblog ...
微软 AutoML 框架之 FLAML | 环境配置、安装、分类/回归测试
一.引言 FLAML(A Fast and Lightweight AutoML Library),是由微软主推的一个全新的高效轻量级自动化机器学习框架. 论文 arXiv 地址 | FLAML: A ...

机器学习笔记 - AutoML框架FLAML初体验