ElasticSearch(搜索服务器)-第二天
bulk
bulk 脚本
Bulk 批量操作是将文档的增删改查一些列操作,通过一次请求全都做完。减少网络传输次数。
语法:
示例:
POST _bulk
{"delete":{"_index":"person","_id":4}}
{"create":{"_index":"person","_id":8}}
{"name" : "李四8","age" : 44,"address" : "北京海淀"}
{"update":{"_index":"person","_id":2}}
{"doc":{"name" : "李四儿"}}
bulk javaApi
测试代码添加
/*** 批量操作 bulk* @throws IOException*/@Testpublic void testBulk() throws IOException {// POST _bulk// {"delete":{"_index":"person","_id":4}}// {"create":{"_index":"person","_id":8}}// {"name" : "李四8","age" : 44,"address" : "北京海淀"}// {"update":{"_index":"person","_id":2}}// {"doc":{"name" : "李四儿"}}//1创建请求BulkRequest bulkRequest=new BulkRequest();//删除DeleteRequest deleteRequest=new DeleteRequest("person","4");bulkRequest.add(deleteRequest);// 添加IndexRequest indexRequest = new IndexRequest("person").type("_doc").id("8");Map<String, Object> sourceMap = new HashMap<>();sourceMap.put("name", "李四4");sourceMap.put("age", 44);sourceMap.put("address", "北京海淀");indexRequest.source(sourceMap);bulkRequest.add(indexRequest);// 修改Map<String, Object> sourceMap2 = new HashMap<>();sourceMap2.put("name", "李四4");UpdateRequest updateRequest=new UpdateRequest("person","2");updateRequest.doc(sourceMap2);bulkRequest.add(updateRequest);//2执行操作BulkResponse bulkResponse = client.bulk(bulkRequest, RequestOptions.DEFAULT);//3获取结果RestStatus status = bulkResponse.status();System.out.println(status);}
导入数据
导入数据-数据准备
4.1需求
将数据库中Goods表的数据导入到ElasticSearch中
4.2实现步骤
创建goods索引
查询Goods表数据
批量添加到ElasticSearch中
4.3准备工作
1mysql导入数据,模拟业务逻辑。
2es创建索引
PUT goods
{"mappings": {"properties": {"title": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},"price": { "type": "double"},"createTime": {"type": "date"},"categoryName": { "type": "keyword"},"brandName": { "type": "keyword"},"spec": { "type": "object"},"saleNum": { "type": "integer"}, "stock": { "type": "integer"}}}
}
- title:商品标题
- price:商品价格
- createTime:创建时间
- categoryName:分类名称。如:家电,手机
- brandName:品牌名称。如:华为,小米
- spec: 商品规格。如: spec:{“屏幕尺寸”,“5寸”,“内存大小”,“128G”}
- saleNum:销量
- stock:库存量
3添加一条数据测试
POST goods/_doc/1
{"title":"小米手机","price":1000,"createTime":"2019-12-01","categoryName":"手机","brandName":"小米","saleNum":3000,"stock":10000,"spec":{"网络制式":"移动4G","屏幕尺寸":"4.5"}
}
导入数据-代码实现
代码
1导入依赖
<!--mybatis--><dependency><groupId>org.mybatis.spring.boot</groupId><artifactId>mybatis-spring-boot-starter</artifactId><version>2.1.0</version></dependency><!--mysql驱动--><dependency><groupId>mysql</groupId><artifactId>mysql-connector-java</artifactId></dependency>
2配置文件增加配置
# datasource
spring:datasource:url: jdbc:mysql:///es11?serverTimezone=UTCusername: rootpassword: rootdriver-class-name: com.mysql.cj.jdbc.Driver# mybatis
mybatis:mapper-locations: classpath:mapper/*Mapper.xml # mapper映射文件路径type-aliases-package: com.itheima.es11.domain
3创建实体类
com.itheima.es11.domain
public class Goods {private int id;private String title;private double price;private int stock;private int saleNum;private Date createTime;private String categoryName;private String brandName;private Map spec;@JSONField(serialize = false)//在转换JSON时,忽略该字段private String specStr;//接收数据库的信息 "{}"getter and setter....
}
4创建mapper
com.itheima.es11.mapper
@Repository
@Mapper
public interface GoodsMapper {/*** 查询所有商品*/public List<Goods> findAll();
}
5xml文件
resources下创建mapper/GoodsMapper.xml
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd"><mapper namespace="com.itheima.es11.mapper.GoodsMapper"><select id="findAll" resultType="goods">select`id` ,`title` ,`price` ,`stock` ,`saleNum` ,`createTime` ,`categoryName`,`brandName` ,`spec` as specStrfrom goods</select>
</mapper>
6测试类 测试方法
@Testpublic void importData() throws IOException {//1查询所有商品List<Goods> all = goodsMapper.findAll();// System.out.println(all.size());//2bulk导入es//2.1创建请求BulkRequest bulkRequest=new BulkRequest();for (Goods goods : all) {//将spec转换goods.setSpec(JSON.parseObject(goods.getSpecStr(),Map.class));// 添加IndexRequest indexRequest = new IndexRequest("goods").type("_doc").id(goods.getId()+"");indexRequest.source(JSON.toJSONString(goods),XContentType.JSON);bulkRequest.add(indexRequest);}//2.2执行操作BulkResponse bulkResponse = client.bulk(bulkRequest, RequestOptions.DEFAULT);//2.3获取结果RestStatus status = bulkResponse.status();System.out.println(status);}
如遇到如下错:
解决方案:
PUT /goods/_settings
{"settings": {"index.mapping.total_fields.limit": 2000}
}
导入数据 详解-了解
有能力同学看一眼即可。
关注点:es的mapping设置为object,数据给为“{}”,es解析不了。
各种搜索-重点
matchAll 脚本
语法:
示例:
GET goods/_search
{"query": {"match_all": {}}
}
注意:
1 get带请求体。
2默认返回10条。
3缓存策略。
4返回字段解析。
- took:本次操作花费的时间,单位为毫秒。
- timed_out:请求是否超时
- _shards:说明本次操作共搜索了哪些分片
- hits:搜索命中的记录
- hits.total : 符合条件的文档总数 hits.hits :匹配度较高的前N个文档
- hits.max_score:文档匹配得分,这里为最高分
- _score:每个文档都有一个匹配度得分,按照降序排列。
- _source:显示了文档的原始内容。
分页查询
GET goods/_search
{"query": {"match_all": {}},"from": 0,"size": 20
}
matchAll javaApi
/*** MathchAll*/@Testpublic void testMathchAll() throws IOException {// GET goods/_search// {// "query": {// "match_all": {}// },// "from": 0,// "size": 20// }//1创建请求SearchRequest searchRequest = new SearchRequest("goods");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//查询条件searchSourceBuilder.query(QueryBuilders.matchAllQuery());//分页条件searchSourceBuilder.from(0);searchSourceBuilder.size(20);searchRequest.source(searchSourceBuilder);//2执行操作SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);//3获取结果TimeValue took = searchResponse.getTook();SearchHits hits = searchResponse.getHits();List<Goods> list = new ArrayList<>();for (SearchHit hit : hits) {//获取json字符串格式的数据String sourceAsString = hit.getSourceAsString();//转为java对象Goods goods = JSON.parseObject(sourceAsString, Goods.class);list.add(goods);}//遍历list展现数据for (Goods goods : list) {System.out.println(goods);}}
term 词条查询
不会对查询条件进行分词。
脚本语法:
GET goods/_search
{"query": {"term": {"brandName": {"value": "小米"}}}
}
java代码:
/*** termQuery*/@Testpublic void testTermQuery() throws IOException {// GET goods/_search// {// "query": {// "term": {// "brandName": {// "value": "小米"// }// }// }// }//1创建请求SearchRequest searchRequest = new SearchRequest("goods");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//查询条件searchSourceBuilder.query(QueryBuilders.termQuery("brandName","小米"));searchRequest.source(searchSourceBuilder);//2执行操作SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);//3获取结果TimeValue took = searchResponse.getTook();SearchHits hits = searchResponse.getHits();List<Goods> list = new ArrayList<>();for (SearchHit hit : hits) {//获取json字符串格式的数据String sourceAsString = hit.getSourceAsString();//转为java对象Goods goods = JSON.parseObject(sourceAsString, Goods.class);list.add(goods);}//遍历list展现数据for (Goods goods : list) {System.out.println(goods);}}
matchQuery
流程:
- 会对查询条件进行分词。
- 然后将分词后的查询条件和词条进行等值匹配
- 默认取并集(OR)
脚本:
GET goods/_search
{"query": {"match": {"title": "华为手机"}}
}
GET goods/_search
{"query": {"match": {"title": {"query": "小米手机","operator": "or"}}}
}
体会 or 和and的不同
javaApi:
其他不变,只看变动
searchSourceBuilder.query(QueryBuilders.matchQuery("title","华为手机"));
searchSourceBuilder.query(QueryBuilders.matchQuery("title","华为手机").operator(Operator.OR));
模糊查询 脚本
现象:
GET goods/_search
{"query": {"match": {"title": "象"}}
}
查不到
继续查询:
GET goods/_search
{"query": {"match": {"title": "象牙白"}}
}
可以查到,想想为什么?倒排索引表中对应“白”没有关联文档。
引出模糊查询:
wildcard查询:会对查询条件进行分词。还可以使用通配符?(任意单个字符)和* (0个或多个字符)
GET goods/_search {"query": {"wildcard": {"title": {"value": "象??"}}} }
regexp查询:正则查询
GET goods/_search
{"query": {"regexp": {"title": "\\w+(.)*"}}
}
字母开头数据都查出来了
- prefix查询:前缀查询
GET goods/_search
{"query": {"prefix": {"brandName": {"value": "三"}}}
}
模糊查询 javaApi
测试类
- wildcard查询
searchSourceBuilder.query(QueryBuilders.wildcardQuery("title", "象**"));
- regexp查询
searchSourceBuilder.query(QueryBuilders.regexpQuery("title","\\w+(.)*"));
- prefix查询
searchSourceBuilder.query(QueryBuilders.prefixQuery("brandName","三"));
范围查询
range 范围查询:查找指定字段在指定范围内包含值。
脚本:
GET goods/_search
{"query": {"range": {"price": {"gte": 2000,"lte": 3000}}}
}
代码:
searchSourceBuilder.query(QueryBuilders.rangeQuery("price").gte(2000).lte(3000));
排序:
脚本:
GET goods/_search
{"query": {"range": {"price": {"gte": 2000,"lte": 3000}}},"sort": [{"price": {"order": "desc"}}]
}
代码:
//排序
searchSourceBuilder.sort("price", SortOrder.DESC);
queryString
流程:
- 会对查询条件进行分词。
- 然后将分词后的查询条件和词条进行等值匹配
- 默认取并集(OR)
- 可以指定多个查询字段
脚本:
GET goods/_search
{"query": {"query_string": {"fields": ["title","categoryName","brandName"],"query": "华为手机"}}
}
GET goods/_search
{"query": {"simple_query_string": {"fields": ["title","categoryName","brandName"],"query": "华为 AND 手机"}}
}
不支持连接符,按照“华为”“AND”“手机” 三个词查询。
代码:
searchSourceBuilder.query(QueryBuilders.queryStringQuery("华为手机").field("title").field("categoryName" ).field("brandName").defaultOperator(Operator.OR));
searchSourceBuilder.query(QueryBuilders.simpleQueryStringQuery("华为手机").field("title").field("categoryName" ).field("brandName"));
布尔查询 脚本
1概念:
对多个查询条件连接
2链接方式:
- must(and):条件必须成立
- must_not(not):条件必须不成立
- should(or):条件可以成立
- filter:条件必须成立,性能比must高。不会计算得分。 range 1000-2000,term 华为
3脚本:
GET goods/_search
{"query": {"bool": {"must": [{"term": {"brandName": {"value": "华为"}}},{"match": {"title": "电信"}}],"must_not": [{"term": {"brandName": {"value": "小米"}} }],"should": [{"term": {"brandName": {"value": "小米"}} }],"filter": {"match": {"title": "白"} }}}
}
布尔查询 javaApi
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("brandName", "华为");MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("title", "电信");boolQueryBuilder.must(termQueryBuilder);boolQueryBuilder.must(matchQueryBuilder);TermQueryBuilder termQueryBuilder1 = QueryBuilders.termQuery("brandName", "小米");boolQueryBuilder.mustNot(termQueryBuilder1);boolQueryBuilder.should(termQueryBuilder1);MatchQueryBuilder matchQueryBuilder1 = QueryBuilders.matchQuery("title", "白");boolQueryBuilder.filter(matchQueryBuilder1);searchSourceBuilder.query(boolQueryBuilder);
聚合查询 脚本
指标聚合:相当于MySQL的聚合函数。max、min、avg、sum等
脚本:
GET goods/_search
{"query": {"match": {"title": "华为"}},"aggs": {"max_price": {"max": {"field": "price"}}}
}
桶聚合:相当于MySQL的group by 操作。不要对text类型的数据进行分组,会失败。
脚本:
GET goods/_search
{"query": {"match": {"title": "手机"}},"aggs": {"group_by_brands": {"terms": {"field": "brandName","size": 10}}}
}
聚合查询 javaApi
/*** aggsQuery*/@Testpublic void testAggsQuery() throws IOException {// GET goods/_search// {// "query": {// "match": {// "title": "手机"// }// },// "aggs": {// "group_by_brands": {// "terms": {// "field": "brandName",// "size": 10// }// }// }// }//1创建请求SearchRequest searchRequest = new SearchRequest("goods");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//查询条件searchSourceBuilder.query(QueryBuilders.matchQuery("title","手机"));// 聚合TermsAggregationBuilder termsAggregationBuilder = AggregationBuilders.terms("group_by_brands").field("brandName").size(100);searchSourceBuilder.aggregation(termsAggregationBuilder);//排序searchSourceBuilder.sort("price", SortOrder.DESC);searchRequest.source(searchSourceBuilder);//2执行操作SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);//3获取结果SearchHits hits = searchResponse.getHits();List<Goods> list = new ArrayList<>();for (SearchHit hit : hits) {//获取json字符串格式的数据String sourceAsString = hit.getSourceAsString();//转为java对象Goods goods = JSON.parseObject(sourceAsString, Goods.class);list.add(goods);}//遍历list展现数据// for (Goods goods : list) {// System.out.println(goods);// }//获取聚合结果Aggregations aggregations = searchResponse.getAggregations();Map<String, Aggregation> asMap = aggregations.getAsMap();Terms group_by_brands = (Terms) asMap.get("group_by_brands");List<? extends Terms.Bucket> buckets = group_by_brands.getBuckets();for (Terms.Bucket bucket : buckets) {String keyAsString = bucket.getKeyAsString();System.out.println("keyAsString:"+keyAsString);long docCount = bucket.getDocCount();System.out.println("docCount:"+docCount);System.out.println("===============================");}}
高亮 脚本
高亮三要素:
- 高亮字段
- 前缀
- 后缀
脚本:
GET goods/_search
{"query": {"match": {"title": "手机"}},"highlight": {"fields": {"title": {"pre_tags":"<font color='red'>","post_tags": "</font>"}}}
}
注意观察结果。
高亮 javaApi
/*** highlightQuery*/@Testpublic void testHighlightQuery() throws IOException {// GET goods/_search// {// "query": {// "match": {// "title": "手机"// }// },// "highlight": {// "fields": {// "title": {// "pre_tags":"<font color='red'>",// "post_tags": "</font>"// }// }// }// }//1创建请求SearchRequest searchRequest = new SearchRequest("goods");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//查询条件searchSourceBuilder.query(QueryBuilders.matchQuery("title","手机"));//高亮HighlightBuilder highlightBuilder = new HighlightBuilder();highlightBuilder.field("title");highlightBuilder.preTags("<font color='red'>");highlightBuilder.postTags("</font>");searchSourceBuilder.highlighter(highlightBuilder);searchRequest.source(searchSourceBuilder);//2执行操作SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);//3获取结果SearchHits hits = searchResponse.getHits();List<Goods> list = new ArrayList<>();for (SearchHit hit : hits) {//获取json字符串格式的数据String sourceAsString = hit.getSourceAsString();//转为java对象Goods goods = JSON.parseObject(sourceAsString, Goods.class);//获取高亮Map<String, HighlightField> highlightFields = hit.getHighlightFields();HighlightField highlightField = highlightFields.get("title");Text[] fragments = highlightField.getFragments();//替换titlegoods.setTitle(fragments[0].toString());list.add(goods);}//遍历list展现数据for (Goods goods : list) {System.out.println(goods);}}
重建索引
需求:
随着业务需求的变更,索引的结构可能发生改变。
ElasticSearch的索引一旦创建,只允许添加字段,不允许改变字段。因为改变字段,需要重建倒排索引,影响内部缓存结构,性能太低。
那么此时,就需要重建一个新的索引,并将原有索引的数据导入到新索引中。
!!!索引名必须小写
步骤:
1建立索引
PUT student_v1
{"mappings": {"properties": {"birthday":{"type": "date"}}}
}
2插入数据
PUT student_v1/_doc/1
{"birthday":"1992-01-07"
}
3修改映射?失败
PUT student_v1
{"mappings": {"properties": {"birthday":{"type": "text"}}}
}
4建立另外索引
PUT student_v2
{"mappings": {"properties": {"birthday":{"type": "text"}}}
}
5转移数据
POST _reindex
{"source": {"index": "student_v1"},"dest": {"index": "student_v2"}
}
6v2插入新数据
PUT student_v2/_doc/2
{"birthday":"1992年01月07日"
}
7索引别名
7.1删除v1
DELETE student_v1
7.2v2起别名为v1
POST student_v2/_alias/student_v1
7.3查询数据
GET student_v1/_doc/1
7.4查看索引信息
GET student_v1
#1 运维 创建 user_v1
PUT user_v1
{"mappings": {"properties": {"name":{"type": "text"}}}
}
#2 起别名
POST user_v1/_alias/user
#3 开发 user 增删改查
PUT /user/_doc/1
{"name":"张三"
}
#需求 改字段类型#4 创建user_v2
PUT user_v2
{"mappings": {"properties": {"name":{"type": "keyword"}}}
}
#5 数据转移:旧索引名 新索引名 bulkAPI。
POST _reindex
{"source": {"index": "user_v1"},"dest": {"index": "user_v2"}
}
#6 user_v2别名设置成user
POST user_v2/_alias/user
#7 删除user_v1
DELETE user_v1
ElasticSearch(搜索服务器)-第二天相关推荐
- ElasticSearch(搜索服务器)-第一天
1为什么使用es ElasticSearch 搜索服务器.简称es. 初识es 搜索时数据库的问题 2.1是什么 搜索服务器 软件 2.1.1互联网搜索 https://www.baidu.com/ ...
- java es api jar包_Elasticsearch 搜索服务器 Java API 使用详解
ElasticSearch 是一个基于 Lucene 的搜索服务器.它提供了一个分布式多用户能力的全文搜索引擎,基于 RESTful web 接口.Elasticsearch 是用 Java 开发的, ...
- ElasticSearch搜索详细讲解与操作
全文检索基础 全文检索流程 流程: #mermaid-svg-7Eg2qFEl06PIEAxZ {font-family:"trebuchet ms",verdana,arial, ...
- JAVA商城项目(微服务框架)——第11天 elasticsearch搜索
0.学习目标 独立安装Elasticsearch 会使用Rest的API操作索引 会使用Rest的API查询数据 会使用Rest的API聚合数据 掌握Spring Data Elasticsearch ...
- 蝴蝶影视服务器响应异常,elasticsearch的服务器响应异常及解决策略(转)
详述: 1 _riverStatus Import_fail 问题描述: 发现有个索引的数据同步不完整,在 http://192.168.1.17:9200/_plugin/head/ 在browse ...
- 上个厕所的功夫弄清楚了ES搜索服务器
1.概念 ElasticSearch是一个基于Lucene的搜索服务器 是一个分布式.高扩展.高实时的搜索与数据分析引擎 基于RESTful web接口 Elasticsearch是用Java语言开发 ...
- 分布式搜索elasticsearch搜索功能【深入】
elasticsearch搜索功能[深入] 分布式搜索elasticsearch搜索功能[深入] 1.数据聚合 1.1 聚合的种类 1.2 DSL实现聚合 1.2.1 Bucket聚合 1.2.2 M ...
- Elasticsearch 搜索条件与聚合结果再进行过滤的多重聚合查询-过滤桶的使用(六)
Elasticsearch 搜索条件与聚合结果再进行过滤的多重聚合查询-过滤桶的使用(六) 实现的需求:数据分析的大范围是福特的车子,而且还需要上个月这个牌子的汽车的平均售价 需求分析: Rest-A ...
- Elasticsearch搜索结果处理
CSDN话题挑战赛第1期 活动详情地址:https://marketing.csdn.net/p/bb5081d88a77db8d6ef45bb7b6ef3d7f 参赛话题:Java学习记录 话题描述 ...
最新文章
- AOP配置开发入门案例
- Learning React Native笔记
- Linux常用的基本命令head、tail、tar、grep、date、cal(二)
- Innodb存储引擎的特性(1).
- apache.camel_Apache Camel 2.20发布–新增功能
- mysql数据库的备份和恢复的常用方法_MySQL数据库备份与恢复方法(转)
- Spring配置 context:component-scan/ mvc:annotation-driven /
- FFMpeg写MP4文件例子分析
- 2.kafka 安装
- 《编码:隐匿在计算机软硬件背后的语言》TXT
- 空间计量经济学(2)---广义空间自回归与空间误差模型
- 用户sa登录失败的解决办法
- 2013年IT TOP100
- 公考复盘(一)——第三季第四季
- 操作系统在计算机科学发展中的重要性,浅谈计算机科学中的系统论与辩证法
- 如何在网上获取视频地址(.mp4)
- GEE|时间序列分析(四)
- 调用支付宝第三方接口(沙箱环境) Spring Boot+Maven
- 2020年社招面试题
- Eclipse+Java+Swing+Mysql实现学生信息管理系统
热门文章
- 深入了解计算机的知识,电脑入门基础知识之深入理解计算机系统
- diff和patch 打补丁
- 一张纸厚度是多少毫米_一张纸的厚度是多少?
- 计算机现代教育三大特征,现代教育技术试题和答案
- EC11、EC16、ECxx旋转编码器按钮软件滤波程序滤除干扰杂波51单片机C程序
- 2022.01.19翻译Ancient Berland Circus
- 信息技术用计算机画画教学设计,2015小学四级信息技术用计算机画画教学设计.doc...
- 02.图像分割中将掩码转换为坐标点的方法(多目标)
- 2016年上半年信息系统项目管理师真题之上午题小虎趣味解答第26-30题
- python中文字体下载_Python在Matplotlib图中显示中文字体的操作方法