Elasticsearch（二）

Clasticsearch（二）

DSL查询语法

文档

文档：https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html

常见查询类型包括：

• 查询所有：查询出所有数据，一般测试用。如：match_all

• 全文检索查询：利用分词器对用户输入的内容分词，然后去倒排索引库中匹配。如

• • match_query
  • multi_match_query

• 精确查询：根据精确词条查找数据，一般是查找keyword、数值、日期、boolean等字符。如：

• • ids
  • range
  • term

• 地理查询：根据经纬度查询。例如：

• • geo_distance
  • geo_bounding_box

• 复合查询：复合查询可以将上述各种查询条件组合起来，合并查询条件。如

• • bool
  • function_score

基本语法

查询DSL基本语法

GET /indexName/_search
{"query":{"查询类型":{"查询条件":"条件值"}}
}

全文检索查询

GET /indexName/_search
{"query": {"match": {"FIELD": "TEXT"}}
}

精确查询

常用：

• term查询

GET /hotel/_search
{"query": {"term": {"city": {"value": "上海"}}}
}

• range查询

GET /hotel/_search
{"query": {"range": {"price": {"gte": 100,"lte": 300}}}
}

地理查询

常见场景包括：

• 携程：搜索我附近的酒店
• 滴滴：搜索我附近的出租车
• 微信：搜索我附近的人

根据经纬度查询：

• geo_bounding_box：查询geo_point值落在某个矩形范围的所有文档

• geo_distance：查询到指定中心小于某个距离值得所有文档

复合查询

复合查询：复合查询可以将其它简单查询组合起来，实现复杂得搜索逻辑，例如：

• function score：算分函数查询，可以控制文档相关性算分，控制文档排名

相关性算分

Function Score Query

案例：给“如家”这个品牌得酒店排名靠前一些

Boolean Query

布尔查询是一个或多个子句得组合。子查询得组合方式有：

• must：必须匹配每个子查询，类似“与”
• should：选择性匹配子查询，类似“或”
• must_not：必须不匹配，不参与算分，类似“非”
• filter：必须匹配，不参与算分

案例：利用bool查询实现功能

GET /hotel/_search
{"query": {"bool": {"must": [{"match": {"name": "如家"}}],"must_not": [{"range": {"price": {"gt": 400}}}],"filter": [{"geo_distance": {"distance": "10km","location": {"lat": 31.21,"lon": 121.5}}}]}}
}

搜索结果处理

排序

elasticsearch支持搜索结果排序，默认是根据相关度算分来排序。可以排序类型有：keyword类型、数值类型、地理坐标类型、日期类型等。

分页

elasticsearch默认情况下只返回top10得数据。如果要查询更多数据就需要修改分页参数。

elasticsearch中通过修改from、size参数来控制要返回得分页结果：

深度分页问题

ES是分布式的，所以会面临深度分页问题。

深度分页解决方案

针对深度分页，ES提供了两种解决方案：

• search after：分页时需要排序，原理是从上一次的排序值开始，查询下一页数据。官方推荐使用的方式。
• scroll：原理将排序数据形成快照，保存在内存。官方已经不推荐使用。

高亮

RestClient查询文档

快速入门

    @Testvoid testInit() throws IOException {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSLrequest.source().query(QueryBuilders.matchAllQuery());// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);System.out.println(response);}

解析JSON

    @Testvoid testInit() throws IOException {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSLrequest.source().query(QueryBuilders.matchAllQuery());// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.解析响应SearchHits hits = response.getHits();// 4.1.获取总条数long total = hits.getTotalHits().value;System.out.println(total);// 4.2.遍历文档数组for (SearchHit hit : hits.getHits()) {// 4.3.获取json对象String json = hit.getSourceAsString();// 4.4.将json对象变成java对象HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);System.out.println(hotelDoc);}}

全文检索查询

精确查询

复合查询-boolean query

    @Testvoid testBool() throws IOException {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSLBoolQueryBuilder boolQuery = QueryBuilders.boolQuery();boolQuery.must(QueryBuilders.termQuery("city","上海"));boolQuery.filter(QueryBuilders.rangeQuery("price").lte(250));request.source().query(boolQuery);// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);handleResponse(response);}/*** 解析响应* @param response*/private static void handleResponse(SearchResponse response) {// 4.解析响应SearchHits hits = response.getHits();// 4.1.获取总条数long total = hits.getTotalHits().value;System.out.println(total);// 4.2.遍历文档数组for (SearchHit hit : hits.getHits()) {// 4.3.获取json对象String json = hit.getSourceAsString();// 4.4.将json对象变成java对象HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);System.out.println(hotelDoc);}}

排序和分页

    @Testvoid testPageAndSort() throws IOException {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSLrequest.source().query(QueryBuilders.matchAllQuery());// 排序request.source().sort("price", SortOrder.ASC);request.source().from(1).size(5);// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);handleResponse(response);}

高亮

    @Testvoid testHighlight() throws IOException {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSLrequest.source().query(QueryBuilders.matchQuery("all","如家"));request.source().highlighter(new HighlightBuilder().field("name").requireFieldMatch(false));// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);handleResponse(response);}

高亮的结果解析

/*** 解析响应* @param response*/private static void handleResponse(SearchResponse response) {// 4.解析响应SearchHits hits = response.getHits();// 4.1.获取总条数long total = hits.getTotalHits().value;System.out.println(total);// 4.2.遍历文档数组for (SearchHit hit : hits.getHits()) {// 4.3.获取json对象String json = hit.getSourceAsString();// 4.4.将json对象变成java对象HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);// 获取高亮结果Map<String, HighlightField> highlightFields = hit.getHighlightFields();if (!CollectionUtils.isEmpty(highlightFields)) {// 根据字段名获取高亮结果HighlightField highlightField = highlightFields.get("name");if (highlightField != null) {// 获取高亮值String name = highlightField.getFragments()[0].string();hotelDoc.setName(name);}}System.out.println(hotelDoc);}}

demo案例

实现搜索功能

1.定义实体类，接受前端请求

RequestParams

@Data
public class RequestParams {private String key;private Integer page;private Integer size;private String sortBy;
}

PageResult

@Data
public class PageResult {private Long total;private List<HotelDoc> hotels;
}

2.定义controller接口，接受页面请求，调用IHotelService的search方法

    @PostMapping("/list")public PageResult search(@RequestBody RequestParams params) {return hotelService.search(params);}

3.定义IHotelService中的search方法，利用match查询实现根据关键字搜索酒店信息

 @Overridepublic PageResult search(RequestParams params) {try {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSLString key = params.getKey();if (StringUtils.isBlank(key)) {request.source().query(QueryBuilders.matchAllQuery());} else {request.source().query(QueryBuilders.matchQuery("all", key));}// 分页int page = params.getPage();int size = params.getSize();request.source().from((page - 1) * size).size(size);// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);return handleResponse(response);} catch (IOException e) {throw new RuntimeException(e);}}/*** 解析响应** @param response*/private static PageResult handleResponse(SearchResponse response) {// 4.解析响应SearchHits hits = response.getHits();// 4.1.获取总条数long total = hits.getTotalHits().value;// 4.2.遍历文档数组List<HotelDoc> hotels = new ArrayList<>();for (SearchHit hit : hits.getHits()) {// 4.3.获取json对象String json = hit.getSourceAsString();// 4.4.将json对象变成java对象HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);hotels.add(hotelDoc);}PageResult pageResult = new PageResult();pageResult.setTotal(total);pageResult.setHotels(hotels);return pageResult;}

添加品牌、城市、星级、价格等过滤功能

1.修改RequestParams类，添加brand、city、starName、minPrice、maxPrice等参数

package cn.itcast.hotel.pojo;import lombok.Data;/*** @author xc* @date 2023/5/11 16:14*/
@Data
public class RequestParams {private String key;private Integer page;private Integer size;private String sortBy;private String city;private String brand;private String starName;private Integer minPrice;private Integer maxPrice;
}

2.修改search方法的实现，在关键字搜索时，如果brand等参数存在，对其做过滤

@Overridepublic PageResult search(RequestParams params) {try {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSL// 构建BooleanQueryBoolQueryBuilder boolQuery = QueryBuilders.boolQuery();// 关键字搜索String key = params.getKey();if (StringUtils.isBlank(key)) {boolQuery.must(QueryBuilders.matchAllQuery());} else {boolQuery.must(QueryBuilders.matchQuery("all", key));}// 城市条件if (params.getCity() != null && !params.getCity().equals("")) {boolQuery.filter(QueryBuilders.termQuery("city",params.getCity()));}// 品牌条件if (params.getBrand() != null && !params.getBrand().equals("")) {boolQuery.filter(QueryBuilders.termQuery("brand",params.getBrand()));}// 星级条件if (params.getStarName() != null && !params.getStarName().equals("")) {boolQuery.filter(QueryBuilders.termQuery("starName",params.getStarName()));}// 价格条件if (params.getMinPrice() != null && params.getMaxPrice() != null) {boolQuery.filter(QueryBuilders.rangeQuery("price").gte(params.getMinPrice()).lte(params.getMaxPrice()));}request.source().query(boolQuery);// 分页int page = params.getPage();int size = params.getSize();request.source().from((page - 1) * size).size(size);// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);return handleResponse(response);} catch (IOException e) {throw new RuntimeException(e);}}

附近的酒店

距离排序

    @Overridepublic PageResult search(RequestParams params) {try {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSL// 构建BooleanQueryBoolQueryBuilder boolQuery = QueryBuilders.boolQuery();// 关键字搜索String key = params.getKey();if (StringUtils.isBlank(key)) {boolQuery.must(QueryBuilders.matchAllQuery());} else {boolQuery.must(QueryBuilders.matchQuery("all", key));}// 城市条件if (params.getCity() != null && !params.getCity().equals("")) {boolQuery.filter(QueryBuilders.termQuery("city",params.getCity()));}// 品牌条件if (params.getBrand() != null && !params.getBrand().equals("")) {boolQuery.filter(QueryBuilders.termQuery("brand",params.getBrand()));}// 星级条件if (params.getStarName() != null && !params.getStarName().equals("")) {boolQuery.filter(QueryBuilders.termQuery("starName",params.getStarName()));}// 价格条件if (params.getMinPrice() != null && params.getMaxPrice() != null) {boolQuery.filter(QueryBuilders.rangeQuery("price").gte(params.getMinPrice()).lte(params.getMaxPrice()));}request.source().query(boolQuery);// 分页int page = params.getPage();int size = params.getSize();request.source().from((page - 1) * size).size(size);// 2.3.排序String location = params.getLocation();if (location != null && !location.equals("")){request.source().sort(SortBuilders.geoDistanceSort("location",new GeoPoint(location)).order(SortOrder.ASC).unit(DistanceUnit.KILOMETERS));}// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);return handleResponse(response);} catch (IOException e) {throw new RuntimeException(e);}}/*** 解析响应** @param response*/private static PageResult handleResponse(SearchResponse response) {// 4.解析响应SearchHits hits = response.getHits();// 4.1.获取总条数long total = hits.getTotalHits().value;// 4.2.遍历文档数组List<HotelDoc> hotels = new ArrayList<>();for (SearchHit hit : hits.getHits()) {// 4.3.获取json对象String json = hit.getSourceAsString();// 4.4.将json对象变成java对象HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);// 获取排序值Object[] sortValues = hit.getSortValues();if (sortValues.length > 0) {Object sortValue = sortValues[0];hotelDoc.setDistance(sortValue);}hotels.add(hotelDoc);}PageResult pageResult = new PageResult();pageResult.setTotal(total);pageResult.setHotels(hotels);return pageResult;}

广告置顶

1.给HotelDoc类添加isAD字段，Boolean类型

package cn.itcast.hotel.pojo;import lombok.Data;
import lombok.NoArgsConstructor;@Data
@NoArgsConstructor
public class HotelDoc {private Long id;private String name;private String address;private Integer price;private Integer score;private String brand;private String city;private String starName;private String business;private String location;private String pic;private Object distance;private Boolean isAD;public HotelDoc(Hotel hotel) {this.id = hotel.getId();this.name = hotel.getName();this.address = hotel.getAddress();this.price = hotel.getPrice();this.score = hotel.getScore();this.brand = hotel.getBrand();this.city = hotel.getCity();this.starName = hotel.getStarName();this.business = hotel.getBusiness();this.location = hotel.getLatitude() + ", " + hotel.getLongitude();this.pic = hotel.getPic();}
}

2.修改search方法，添加function score功能，给isAD值为true的酒店增加权重

    @Overridepublic PageResult search(RequestParams params) {try {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSL// 构建BooleanQueryBoolQueryBuilder boolQuery = QueryBuilders.boolQuery();// 关键字搜索String key = params.getKey();if (StringUtils.isBlank(key)) {boolQuery.must(QueryBuilders.matchAllQuery());} else {boolQuery.must(QueryBuilders.matchQuery("all", key));}// 城市条件if (params.getCity() != null && !params.getCity().equals("")) {boolQuery.filter(QueryBuilders.termQuery("city",params.getCity()));}// 品牌条件if (params.getBrand() != null && !params.getBrand().equals("")) {boolQuery.filter(QueryBuilders.termQuery("brand",params.getBrand()));}// 星级条件if (params.getStarName() != null && !params.getStarName().equals("")) {boolQuery.filter(QueryBuilders.termQuery("starName",params.getStarName()));}// 价格条件if (params.getMinPrice() != null && params.getMaxPrice() != null) {boolQuery.filter(QueryBuilders.rangeQuery("price").gte(params.getMinPrice()).lte(params.getMaxPrice()));}// 2.算分FunctionScoreQueryBuilder functionScoreQueryBuilder = QueryBuilders.functionScoreQuery(boolQuery, new FunctionScoreQueryBuilder.FilterFunctionBuilder[]{new FunctionScoreQueryBuilder.FilterFunctionBuilder(QueryBuilders.termQuery("isAD",true),ScoreFunctionBuilders.weightFactorFunction(10))});request.source().query(functionScoreQueryBuilder);// 分页int page = params.getPage();int size = params.getSize();request.source().from((page - 1) * size).size(size);// 2.3.排序String location = params.getLocation();if (location != null && !location.equals("")){request.source().sort(SortBuilders.geoDistanceSort("location",new GeoPoint(location)).order(SortOrder.ASC).unit(DistanceUnit.KILOMETERS));}// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);return handleResponse(response);} catch (IOException e) {throw new RuntimeException(e);}}int size = params.getSize();request.source().from((page - 1) * size).size(size);// 2.3.排序String location = params.getLocation();if (location != null && !location.equals("")){request.source().sort(SortBuilders.geoDistanceSort("location",new GeoPoint(location)).order(SortOrder.ASC).unit(DistanceUnit.KILOMETERS));}// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);return handleResponse(response);} catch (IOException e) {throw new RuntimeException(e);}}