ElasticSearch 6.8.1动态更新同义词(适配elasticsearch-analysis-dynamic-synonym)
动态更新同义词插件适配
非热更新的同义词配置
elasticsearch-analysis-dynamic-synonym 插件可以动态更新同义词,但github上只有5.1.1的版本了,不适配es6.8.1,通过修改源码适配6.8.1。
首先从github上下载5.1.1的源码和插件压缩包
github地址
下载后解压源码包,并导入idea
POM文件修改
修改如下两个版本号
源码修改
由于修改了导入的elasticsearch版本,源码有些地方会报错,修改DynamicSynonymFilter下的getLogger,并同时修改其它类里的getLogger报错的地方:
// public static Logger logger = ESLoggerFactory.getLogger("dynamic-synonym");// 6.8.1的es去掉了ELLoggerFactory,修改为Loggerspublic static Logger logger = Loggers.getLogger(String.class,"dynamic-synonym");
修改LocalSynonymFile下的getReader,新增一个newBufferedReader方法(6.8.1没有这个方法,从5.1.1的源码里把newBufferedReader方法给copy出来)
/*** Opens the given url for reading returning a {@code BufferedReader} that may be* used to read text from the URL in an efficient manner. Bytes from the* file are decoded into characters using the specified charset.*/// copy from elasticsearch-5.1.1.jar source codepublic static BufferedReader newBufferedReader(URL url, Charset cs) throws IOException {CharsetDecoder decoder = cs.newDecoder();Reader reader = new InputStreamReader(url.openStream(), decoder);return new BufferedReader(reader);}public Reader getReader() {Reader reader = null;BufferedReader br = null;try {reader = newBufferedReader(synonymFilePath.toUri().toURL(), Charsets.UTF_8);/*br = new BufferedReader(new InputStreamReader(synonymFileURL.openStream(), Charsets.UTF_8));StringBuffer sb = new StringBuffer("");String line = null;while ((line = br.readLine()) != null) {logger.info("reload local synonym: {}", line);sb.append(line).append(System.getProperty("line.separator"));}reader = new FastStringReader(sb.toString());*/} catch (IOException e) {logger.error("get local synonym reader {} error!", e, location);throw new IllegalArgumentException("IOException while reading local synonyms file", e);} finally {try {if (br != null) {br.close();}} catch (IOException e) {e.printStackTrace();}}return reader;}
修改DynamicSynonymPlugin里的createComponents,这个方法是从第三方的适配6.6.2的jar包里找到的,放进来也能用
/* @Overridepublic Collection<Object> createComponents(Client client,ClusterService clusterService,ThreadPool threadPool,ResourceWatcherService resourceWatcherService,ScriptService scriptService,SearchRequestParsers searchRequestParsers) {Collection<Object> components = new ArrayList<>();components.add(pluginComponent);return components;}*/// copy from es-as-dy-syn-6.62 of third partypublic Collection<Object> createComponents(Client client,ClusterService clusterService,ThreadPool threadPool,ResourceWatcherService resourceWatcherService,ScriptService scriptService,NamedXContentRegistry xContentRegistry,Environment environment,NodeEnvironment nodeEnvironment,NamedWriteableRegistry namedWriteableRegistry) {Collection<Object> components = new ArrayList();components.add(this.pluginComponent);return components;}
总之就是有报错的就去解决掉,一般报错的都是6.8.1的es的jar包里少了啥,从5.1.1的es里的jar包里把它找过来就行了
打包解压到es服务器
maven install打成jar包,放进从github上下载的插件包解压出来的文件夹(替换掉5.1.1的插件jar包),并修改其中的properties文件(这个文件应该会被es服务器读取,修改版本号,注释掉jvm\site\isolate否则会报错)
# Elasticsearch plugin descriptor file
# This file must exist as 'plugin-descriptor.properties' at
# the root directory of all plugins.
#
# A plugin can be 'site', 'jvm', or both.
#
### example site plugin for "foo":
#
# foo.zip <-- zip file for the plugin, with this structure:
# _site/ <-- the contents that will be served
# plugin-descriptor.properties <-- example contents below:
#
# site=true
# description=My cool plugin
# version=1.0
#
### example jvm plugin for "foo"
#
# foo.zip <-- zip file for the plugin, with this structure:
# <arbitrary name1>.jar <-- classes, resources, dependencies
# <arbitrary nameN>.jar <-- any number of jars
# plugin-descriptor.properties <-- example contents below:
#
# jvm=true
# classname=foo.bar.BazPlugin
# description=My cool plugin
# version=2.0.0-rc1
# elasticsearch.version=2.0
# java.version=1.7
#
### mandatory elements for all plugins:
#
# 'description': simple summary of the plugin
description=Analysis-plugin for synonym
#
# 'version': plugin's version
version=6.8.1
#
# 'name': the plugin name
name=analysis-dynamic-synonym### mandatory elements for site plugins:
#
# 'site': set to true to indicate contents of the _site/
# directory in the root of the plugin should be served.
#site=${elasticsearch.plugin.site}
#
### mandatory elements for jvm plugins :
#
# 'jvm': true if the 'classname' class should be loaded
# from jar files in the root directory of the plugin.
# Note that only jar files in the root directory are
# added to the classpath for the plugin! If you need
# other resources, package them into a resources jar.
#jvm=true
#
# 'classname': the name of the class to load, fully-qualified.
classname=com.bellszhu.elasticsearch.plugin.DynamicSynonymPlugin
#
# 'java.version' version of java the code is built against
# use the system property java.specification.version
# version string must be a sequence of nonnegative decimal integers
# separated by "."'s and may have leading zeros
java.version=1.8
#
# 'elasticsearch.version' version of elasticsearch compiled against
# You will have to release a new version of the plugin for each new
# elasticsearch release. This version is checked when the plugin
# is loaded so Elasticsearch will refuse to start in the presence of
# plugins with the incorrect elasticsearch.version.
elasticsearch.version=6.8.1
#
### deprecated elements for jvm plugins :
#
# 'isolated': true if the plugin should have its own classloader.
# passing false is deprecated, and only intended to support plugins
# that have hard dependencies against each other. If this is
# not specified, then the plugin is isolated by default.
#isolated=${elasticsearch.plugin.isolated}
#
打包上传至服务器es目录下的plugins目录下新建的dynamic-synonym目录下解压,重启es服务器即可加载插件成功
修改es服务器的java.policy文件
如果需要远程获取同义词文件,则需要修改这个文件,否则会由于java的什么安全机制,在获取文件时报错
找到jre/lib/security 下的java.policy文件
grant中添加一行
permission java.net.SocketPermission “*”, “connect,resolve”;
之后就是和github上的步骤一样就可以了。
还有一些要注意的地方:
官方示例中的这个需要改成ik_smart或者ik_max_word(whitespace是按空格分词的)
synonym就是自定义的分析器,在建立mapping时使用这个自定义分析器
同义词文件格式:
- A=>B
C=>D
即B是A的同义词,D是C的同义词(好像是单向联想的) - A,B
C,D
即AB CD互为同义词(双向联想)
建议用2,因为1实测有bug如
A=>B
B=>C
但实际搜索时 A只能搜出A,搜不出B,C;B能搜出B,C,C也能搜出B,C
ElasticSearch 6.8.1动态更新同义词(适配elasticsearch-analysis-dynamic-synonym)相关推荐
- ElasticSearch探索之路(五)集群与分片:选举、动态更新、近实时搜索、事务日志、段合并
文章目录 集群内部原理 集群与节点 分片 选举 分片内部原理 索引不变性 动态更新索引 近实时搜索 事务日志 段合并 集群内部原理 集群与节点 一个运行中的Elasticsearch实例称为一个节点, ...
- ES elasticsearch-analysis-dynamic-synonym连接数据库动态更新synonym近义词
前言 在很多搜索场景中,我们希望能够搜索出搜索词相关的目标,同时也希望能搜索出其近义词相关的目标.例如在商品搜索中,搜索"瓠瓜",也希望能够搜索出"西葫芦",但 ...
- ElasticSearch搜索引擎详解-持续更新中
ElasticSearch搜索引擎详解 1. ElasticSearch概述 1.1 elasticsearch是什么 1.2 全文搜索引擎 1.3 elasticsearch and solr 1. ...
- 【Spark】SparkStreaming-流处理-规则动态更新-解决方案
SparkStreaming-流处理-规则动态更新-解决方案 image2017-10-27_11-10-53.png (1067×738)elasticsearch-headElasticsearc ...
- 动态更新阿里云DDNS解析记录的IPv6地址,随时随地用域名远程访问自己的电脑【如何远程访问家里的电脑】
远程访问电脑 日志 简介 要求 1. 获取两台电脑 2.IPv6网络 2.1检查光猫 2.2检查路由器 2.3配置电脑防火墙 2.3.1添加ICMPv6协议 2.3.2配置SMB协议 2.4配置远程桌 ...
- ES学习(五)同义词分词器dynamic synonym for ElasticSearch
dynamic synonym for ElasticSearch elasticsearch动态同义词插件是添加一个同义词过滤器在给定间隔(默认60秒)来重新加载同义词文件(本地文件或远程文件). ...
- Flyme 9.2系统更新:适配魅族17、18、18s系列
2022年4月28日,魅族官方发布消息称,Flyme 9.2再次更新.适配魅族17系列.魅族18及18s系列,今日内完成全量推送. 据了解,Flyme 9.2新增应用相册图标,应用照片轻松分类:重构图 ...
- Spark/Flink广播实现作业配置动态更新
点击上方"zhisheng",选择"设为星标" 后台回复"ffa"可以查看 Flink 资料 前言 在实时计算作业中,往往需要动态改变一些配 ...
- 动态更新 AGS Cache
作者:Flyingis 提升ArcGIS Server访问速度最佳的方式是Cache,将所有图层切片保存在服务器,客户端请求时直接访问cache好的图片,这里分为两种情况,一是所有图层都做cache, ...
最新文章
- mysql2tb_MySQL2
- python基础指令-python的一些基本命令
- SVM+HOG:从完全不包含人体的图片中随机剪裁出64*128大小的用于人体检测的负样本
- 每日一笑 | 在俄罗斯人眼里,没有什么是胶带解决不了的
- python向dict里添加_Python有条件地向Dict添加键
- linux内核同步机制相关收集
- .net 中debug 微软内部代码
- led投影仪能换大功率灯吗_LED大功率洗墙灯怎么防水
- Linux---线程安全
- bzoj 1878: [SDOI2009]HH的项链 ——树状数组+ 差分
- 请不要滥用SharedPreference
- .net mvc身份证验证
- 正则表达式regex(入门使用)
- steam游戏文件夹在哪儿?
- debezium集成Oralce攻略(上)
- Android -- 小功能 仿美图秀秀(美颜相机)马赛克功能
- Win8.1 安装nltk及nltk_data数据
- Google GMS Crash 优化方案
- CentOS Netcat 用法
- 微信H5支付----报undened index openid