[elasticsearch笔记]Managing the index lifecycle

索引生命周期

note
demo

note

索引的生命周期
- hot
- warm
- cold
- delete
如果 index.lifecycyle.indexing_complete=true, 索引不会通过 ilm policy rollover
应用 ilm policy 到已有index中有两个主要问题：
- 不能有 rollover 相关设置，因为rollover是针对新建的index的
- 避免一设置就进行 hot、warm、cold、delete相关policy，因为每个阶段的 min_age 都是比对索引的原始创建时间。通过修改 min_age 规避这个问题。min_age的比对时间是上个阶段的起始时间，还是索引的创建时间？？？
rollover 新建的索引不会继承旧索引的policy，不建议在建立索引的时候设置包含rollover的ilm policy。因为rollover后的索引不会应用ilm policy。采用定义模板的方式使用 ilm policy
索引生命周期中优先级越高，系统启动时，会越早恢复

demo

#
# hot: 数据量达到50G或者7天后 rollover。
# warm: 进入 host 30天后，进入 warm，同时设置 replica为2，强制 merge，shrink。
# cold: 进入 warm 60天后，进入cold阶段，分配到cold节点。
# delete: 进入 cold 90天后，删除索引。
#
PUT _ilm/policy/full_policy
{"policy": {"phases": {"hot": {"actions": {"rollover": {"max_age": "7d","max_size": "50G","max_docs": 100000000}}},"warm": {"min_age": "30d","actions": {"forcemerge": {"max_num_segments": 1},"shrink": {"number_of_shards": 1},"allocate": {"number_of_replicas": 2}}},"cold": {"min_age": "60d","actions": {"allocate": {"require": {"type": "cold"}}}},"delete": {"min_age": "90d","actions": {"delete": {}}}}}
}#
# rollover 需要指定 index.lifecycle.rollover_alias
#
PUT _template/datastream_template
{"index_patterns": ["datastream-*"],                 "settings": {"number_of_shards": 1,"number_of_replicas": 1,"index.lifecycle.name": "full_policy",      "index.lifecycle.rollover_alias": "datastream"    }
}PUT datastream-000001
{"aliases": {"datastream": {"is_write_index": true}}
}#
# rollover 新建的索引不会继承旧索引的policy
# 不建议在建立索引的时候设置包含rollover的ilm policy。因为rollover后的索引不会应用ilm policy。
# 采用定义模板的方式使用 ilm policy
#
PUT test-index
{"settings": {"number_of_shards": 1,"number_of_replicas": 1,"index.lifecycle.name": "full_policy"}
}GET datastream-*/_ilm/explainPUT _ilm/policy/my_policy
{"policy": {"phases": {"warm": {"actions": {"allocate" : {"include" : {"box_type": "hot,warm"}}}}}}
}PUT _ilm/policy/my_policy
{"policy": {"phases": {"warm": {"actions": {"allocate" : {"number_of_replicas": 1,"require" : {"box_type": "cold"}}}}}}
}PUT _ilm/policy/my_policy
{"policy": {"phases": {"cold": {"actions": {"freeze" : { }}}}}
}PUT _ilm/policy/my_policy
{"policy": {"phases": {"warm": {"actions": {"readonly" : { }}}}}
}PUT my_index
{"settings": {"index.lifecycle.name": "my_policy","index.lifecycle.rollover_alias": "my_data"},"aliases": {"my_data": {"is_write_index": true}}
}# 更新 index 的 ilm policy
PUT my_index/_settings
{"lifecycle.name": "my_other_policy"
}#
# 优先级越高，系统启动时，会越早恢复
#
PUT _ilm/policy/my_policy
{"policy": {"phases": {"warm": {"actions": {"set_priority" : {"priority": 50}}}}}
}PUT _ilm/policy/my_policy
{"policy": {"phases": {"hot": {"actions": {"unfollow" : {}}}}}
}#
# update policy
#
PUT _ilm/policy/policy_change_demo
{"policy": {"phases": {"hot": {"actions": {"rollover": {"max_size": "25GB"}}},"delete": {"min_age": "30d","actions": {"delete": {}}}}}
}PUT _ilm/policy/policy_change_demo
{"policy": {"phases": {"hot": {"actions": {"rollover": {"max_size": "25GB"}}},"delete": {"min_age": "10d", "actions": {"delete": {}}}}}
}GET _ilm/status
POST _ilm/stop
POST _ilm/start

[elasticsearch笔记]Managing the index lifecycle相关推荐

Elasticsearch Index Lifecycle Management (ILM)
0 - 前言 Elastic Stack 6.6版本的发布,带来了很多新功能,比如: Index Lifecycle Management Frozen Index Geoshape based on ...
【elasticsearch】elasticsearch 7 index.lifecycle.rollover_alias does not point to index
文章目录 1.场景1 2.场景再现2 3.源码解析 1.场景1 一环境出现这种问题. illegal_argument_exception: index.lifecycle.rollover_alia ...
狂神Elasticsearch笔记
ElasticSearch笔记我们要讲解什么? SQL : like %狂神说% ,如果是的大数据,就十分慢!索引! ElasticSearch:搜索! (百度.github. 淘宝电商! ) 1. ...
Elasticsearch笔记（三）基础知识
Elasticsearch笔记(二)安装与使用理论知识索引.文档.节点.分片索引.文档偏向开发人员节点.分片偏向运维人员文档(Document) 简介 Elasticsearch是面向文档的 ...
ILM(Index Lifecycle Management)
目录概念概述 ILM触发行为 Rollover Shrink Force merge Freeze Delete 索引生命周期阶段概念阶段执行索引生命周期行为 Allocate Delete ...
【Kibana】索引生命周期策略错误illegal_argument_exception: index.lifecycle.rollover_alias does not point to index
一.背景在使用ELK进行日志收集分析时,我们经常会将每天的日志生成一个索引,类似somta-log-2021-02-25,在以前的版本是需要我们写脚本或者服务来删除不想要的索引的,但是在新的ES里面 ...
Elasticsearch最佳实践之Index与Shard设计
Index与Shard,这两个概念在<Elasticsearch最佳实践之核心概念与原理>一文有详细的介绍,分别对应了Elasticsearch的两种数据组织方式:逻辑组织和物理组织. ...
Elasticsearch笔记-es基础版——一看就会
文章目录 1.基础 2.语法 3.基本查询 4.高级查询 1.基础 elasticsearch是一个分布式的全文搜索引擎. 基于Lucene.具有restful的api接口.分布式,高横向扩展能力. ...
Elasticsearch笔记基础入门
并不能保证通过这四篇文章让你掌握ES,但是!我会用大白话串讲ES的一些概念.和花哨的玩法.起码可以把你对Elasticsearch的陌生度降到最低,等有一天你自己业务需要使用ES时,会因为提前读了ES ...

[elasticsearch笔记]Managing the index lifecycle

索引生命周期

note

demo

[elasticsearch笔记]Managing the index lifecycle相关推荐

最新文章

热门文章