LBO验证与LBO_full验证的区别

LBO_full验证的意思是:

除了留出一份作为特定的验证集以外,

其余所有数据丢入训练中,每次训练都使用除了初始设定的验证集以外的所有数据进行训练,然后取平均

NUMBER_OF_MODELS=3

代码[1]如下:

def make_predictions2(train_df,test_df,features_columns, target, lgb_params, NFOLDS=2):SEED=42P = test_df[features_columns]#设定测试集print('#'*20)print('LBO full set training...') ## We need Divide Train Set by Time blocks## Convert TransactionDT to Months## And use last month as Validation ## to find best roundtrain_df['DT_M'] = train_df['TransactionDT'].apply(lambda x: (START_DATE + datetime.timedelta(seconds = x)))train_df['DT_M'] = (train_df['DT_M'].dt.year-2017)*12 + train_df['DT_M'].dt.month main_train_set = train_df[train_df['DT_M']<(train_df['DT_M'].max())].reset_index(drop=True)#前面几个月数据集作为训练集validation_set = train_df[train_df['DT_M']==train_df['DT_M'].max()].reset_index(drop=True) #最后一个月数据集作为验证集#训练集裸数据和类别标签分开X,y = main_train_set[features_columns], main_train_set[TARGET]#五个月的数据作为训练#验证集裸数据和类别标签分开v_X, v_y = validation_set[features_columns], validation_set[TARGET]#最后一个月的数据作为验证#---------------------下面是验证,获取最佳current_iteration()------------------------------------------------------------for current_model in range(3):print('Model:',current_model+1)SEED += 1seed_everything(SEED)corrected_lgb_params = lgb_params.copy()corrected_lgb_params['seed'] = SEEDtrain_data = lgb.Dataset(X, label=y)valid_data = lgb.Dataset(v_X, label=v_y)  estimator = lgb.train(corrected_lgb_params,train_data,valid_sets = [train_data, valid_data],verbose_eval = 1000,)estimators_bestround.append(estimator.current_iteration())#下面根据#---------------------下面是测试以及输出预测结果------------------------------------------------------------corrected_lgb_params = lgb_params.copy()corrected_lgb_params['n_estimators'] = int(np.mean(estimators_bestround))corrected_lgb_params['early_stopping_rounds'] = Noneprint('#'*10)print('Mean Best round:', corrected_lgb_params['n_estimators'])# 所有训练数据X,y = train_df[features_columns], train_df[TARGET]# 测试数据P = test_df[features_columns]RESULTS['lbo_full'] = 0NUMBER_OF_MODELS = 3for current_model in range(NUMBER_OF_MODELS):print('Model:',current_model+1)SEED += 1seed_everything(SEED)    train_data = lgb.Dataset(X, label=y)estimator = lgb.train(corrected_lgb_params,train_data)RESULTS['prediction'] += estimator.predict(P)/NUMBER_OF_MODELSreturn RESULTS

LBO验证的意思是:

除了留出一份作为特定的验证集以外(这里与上面相同),其余进行交叉验证(N_SPLITS=3)

print('#'*20)
print('LBO training...') ## We need Divide Train Set by Time blocks
## Convert TransactionDT to Months
## And use last month as Validation
train_df['DT_M'] = train_df['TransactionDT'].apply(lambda x: (START_DATE + datetime.timedelta(seconds = x)))
train_df['DT_M'] = (train_df['DT_M'].dt.year-2017)*12 + train_df['DT_M'].dt.month main_train_set = train_df[train_df['DT_M']<(train_df['DT_M'].max())].reset_index(drop=True)
validation_set = train_df[train_df['DT_M']==train_df['DT_M'].max()].reset_index(drop=True)## We will use oof kfold to find "best round"
folds = KFold(n_splits=N_SPLITS, shuffle=True, random_state=SEED)# Main Data
X,y = main_train_set[features_columns], main_train_set[TARGET]# Validation Data
v_X, v_y = validation_set[features_columns], validation_set[TARGET]estimators_bestround = []
for fold_, (trn_idx, val_idx) in enumerate(folds.split(X, y)):print('Fold:',fold_+1)tr_x, tr_y = X.iloc[trn_idx,:], y[trn_idx]    train_data = lgb.Dataset(tr_x, label=tr_y)valid_data = lgb.Dataset(v_X, label=v_y)  estimator = lgb.train(lgb_params,train_data,valid_sets = [train_data, valid_data],verbose_eval = 1000,)estimators_bestround.append(estimator.current_iteration())## Now we have "mean Best round" and we can train model on full set
corrected_lgb_params = lgb_params.copy()
corrected_lgb_params['n_estimators'] = int(np.mean(estimators_bestround))
corrected_lgb_params['early_stopping_rounds'] = None
print('#'*10)
print('Mean Best round:', corrected_lgb_params['n_estimators'])# Main Data
X,y = train_df[features_columns], train_df[TARGET]# Test Data
P = test_df[features_columns]
RESULTS['lbo'] = 0for fold_, (trn_idx, val_idx) in enumerate(folds.split(X, y)):print('Fold:',fold_+1)tr_x, tr_y = X.iloc[trn_idx,:], y[trn_idx]train_data = lgb.Dataset(tr_x, label=tr_y)estimator = lgb.train(corrected_lgb_params,train_data)RESULTS['lbo'] += estimator.predict(P)/N_SPLITS#这里的P其实是验证集print('AUC score', metrics.roc_auc_score(RESULTS[TARGET], RESULTS['lbo']))
print('#'*20)

注意事项：

根据[2],不要在特征工程阶段，把验证集也包括进来，因为这会导致leakage.

Reference:

[1]https://www.kaggle.com/kyakovlev/ieee-cv-options

[2]https://www.kaggle.com/c/ieee-fraud-detection/discussion/107728#latest-627879

LBO验证与LBO_full验证的区别相关推荐

Spring Validation验证框架参数验证 @Validated 和 @Valid 的区别
参数验证 @Validated 和 @Valid 的区别 Spring Validation验证框架对参数的验证机制提供了@Validated(Spring's JSR-303 规范,是标准 JSR- ...
验证和确认的区别_验证与确认之间的区别
验证和确认的区别 Today we are going to cover the topic difference between Verification and Validation. 今天,我们 ...
Windows身份验证和混合验证的区别
两个验证方式的不同主要集中在信任连接和非信任连接. windows 身份验证相对于混合模式更加安全,使用本连接模式时候,sql不判断sa密码,而仅根据用户的windows权限来进行身份验证,我们称为& ...
【转】ASP.NET验证控件详解(非空验证，比较验证，范围验证，正则表达式，自定义验证)...
[转]ASP.NET验证控件详解(非空验证,比较验证,范围验证,正则表达式,自定义验证) ASP.NET验证控件详解现在ASP.NET,你不但可以轻松的实现对用户输入的验证,而且,还可以选择验证在服 ...
【Block-Level Verification】芯片开发通识_验证目标_ 验证语言_ 验证职业前景 _挑战和瓶颈_验证周期_功能描述文档_验证计划_回归测试_硅后测试_逃逸分析...
SystemVerilog验证通识 1. 芯片开发概述不同于通用电路,专用集成电路为了专门解决或者优化相关工程问题,例如专用算法的电路实现,如芯片里加入人工智能处理单元,为CPU\GPU减负,目的是 ...
机器学习中为什么需要验证集，验证集与测试集的区别是什么？
目录 1.为什么需要验证集 2.验证集与测试集的区别 3.如何使用验证集 4.为什么验证集和测试集都需要? 参考链接 1.为什么需要验证集在机器学习中,训练模型时需要调节模型的参数来提高模型的性能, ...
交叉验证方法汇总【附代码】（留一法、K折交叉验证、分层交叉验证、对抗验证、时间序列交叉验证）
目录交叉验证是什么? 留一验证(LOOCV,Leave one out cross validation ) LOOCC代码验证集方法验证集方法代码 K折交叉验证(k-fold cross va ...
9、 Struts2验证(声明式验证、自定义验证器)
1. 什么是Struts2 验证器一个健壮的 web 应用程序必须确保用户输入是合法.有效的. Struts2 的输入验证基于 XWork Validation Framework 的声明式验证: ...
计算机系统验证课件,回顾性验证及再验证简介(ppt 42页)
回顾性验证及再验证简介目录: 一. 验证-生命周期二. 验证的方法及适用条件三. 验证状态的维持-生命周期法四. 变更控制五. 变更控制六. 回顾性验证七. 再验证八. 工艺用水九. ...

LBO验证与LBO_full验证的区别

LBO验证与LBO_full验证的区别相关推荐

最新文章

热门文章