08.remote_cluster配置

文章目录

1. 设置remote cluster
2. cross cluster search (跨集群的搜索)
- 1. 设置remote cluster
- 2. 从remote cluster 进行搜索
- 3. 跳过不可用的remote cluster
3. remote cluster search的两种模式

集群可以配置关联远程的集群。从而实现跨集群的一些搜索等操作。

1. 设置remote cluster

可以通过cluster setting api进行动态设置，也可以在 elasticsearch.yml文件中进行设置

cluster:remote:cluster_one: seeds: 127.0.0.1:9300transport.ping_schedule: 30s cluster_two:seeds: 127.0.0.1:9301transport.compress: true skip_unavailable: true

seeds是配置的对应的remote cluster的一个node,
transport.ping_schedule 设置了tcp层面的活性监听
skip_unavailable 设置为true的话，当这个remote cluster不可用的时候，就会忽略，默认是false，当对应的remote cluster不可用的话，则会报错。

PUT _cluster/settings
{"persistent": {"cluster": {"remote": {"cluster_one": {"seeds": ["127.0.0.1:9300"],"transport.ping_schedule": "30s"},"cluster_two": {"seeds": ["127.0.0.1:9301"],"transport.compress": true,"skip_unavailable": true},"cluster_three": {"seeds": ["127.0.0.1:9302"]}}}}
}

在使用api进行动态设置的时候每次都要把seeds带上，如果想要delete一个remote cluster，则把对应的seeds设置为null即可。

2. cross cluster search (跨集群的搜索)

在进行跨集群的搜索之前，首先我们要有一个remote cluster

1. 设置remote cluster

PUT _cluster/settings
{"persistent": {"cluster": {"remote": {"cluster_one": {"seeds": ["127.0.0.1:9300"]},"cluster_two": {"seeds": ["127.0.0.1:9301"]},"cluster_three": {"seeds": ["127.0.0.1:9302"]}}}}
}

2. 从remote cluster 进行搜索

假如我们想搜索cluster_one 中的twitter index,我们可以这样搜索

GET /cluster_one:twitter/_search
{"query": {"match": {"user": "kimchy"}}
}

对应的返回是这个样子

{"took": 150,"timed_out": false,"_shards": {"total": 1,"successful": 1,"failed": 0,"skipped": 0},"_clusters": {"total": 1,"successful": 1,"skipped": 0},"hits": {"total" : {"value": 1,"relation": "eq"},"max_score": 1,"hits": [{"_index": "cluster_one:twitter", "_type": "_doc","_id": "0","_score": 1,"_source": {"user": "kimchy","date": "2009-11-15T14:12:12","message": "trying out Elasticsearch","likes": 0}}]}
}

如果想从当前cluster,还有多个remote cluster搜索，可以这样做

GET /twitter,cluster_one:twitter,cluster_two:twitter/_search
{"query": {"match": {"user": "kimchy"}}
}

对应的返回会是


{"took": 150,"timed_out": false,"num_reduce_phases": 4,"_shards": {"total": 3,"successful": 3,"failed": 0,"skipped": 0},"_clusters": {"total": 3,"successful": 3,"skipped": 0},"hits": {"total" : {"value": 3,"relation": "eq"},"max_score": 1,"hits": [{"_index": "twitter", "_type": "_doc","_id": "0","_score": 2,"_source": {"user": "kimchy","date": "2009-11-15T14:12:12","message": "trying out Elasticsearch","likes": 0}},{"_index": "cluster_one:twitter", "_type": "_doc","_id": "0","_score": 1,"_source": {"user": "kimchy","date": "2009-11-15T14:12:12","message": "trying out Elasticsearch","likes": 0}},{"_index": "cluster_two:twitter", "_type": "_doc","_id": "0","_score": 1,"_source": {"user": "kimchy","date": "2009-11-15T14:12:12","message": "trying out Elasticsearch","likes": 0}}]}
}

3. 跳过不可用的remote cluster

我们可以设置跳过不可用的remote cluster,就是使用上面说的skip_unavailable

PUT _cluster/settings
{"persistent": {"cluster.remote.cluster_two.skip_unavailable": true}
}

3. remote cluster search的两种模式

有两种模式来进行remote cluster search，这两种是基于不同的网络情况来做的选择
1.最小化网络通信次数
这种情况也是默认的情况，当前es的协调节点针对一次请求只访问对应的remote cluster一次，拉取回来所有的的数据，减少了网络交互的次数，减少了网络延迟。

2.非最小化网络通信次数
当前es的协调节点对应一次search会请求remote cluster两次，一次是获取remote cluster的shard信息，第二次是直接请求remote的对应的shard进行数据获取。
感觉这种方式对于网络比较好的集群来说，效果更好，减少了一次数据聚合的过程，直接在当前cluster的协调节点对数据进行组装即可。