how to speed up returning unique elasticsearch values?-CodePudding

I have more than 100 index in elastic. I want return all unique values from all indexes.

        String[] instanceNames = getAllIndices().toArray(String[]::new);
    Map<String, Map<String, Set<String>>> results = new HashMap<>();

    for (String element : instanceNames) {

        SearchRequest searchRequest = new SearchRequest();
        searchRequest.indices(element);
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.matchAllQuery);
        searchSourceBuilder.aggregation(AggregationBuilders.terms("DISTINCT_VALUES").field("provider.keyword"));
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = null;
        try {
            searchResponse = getClient().search(searchRequest, RequestOptions.DEFAULT);
        } catch (ElasticsearchException | IOException e) {
            throw new ServiceException(I18n.ELASTIC_SEARCH_ERROR, e);
        }


        List<SearchHit> searchHits = Arrays.asList(searchResponse.getHits().getHits());
        }

instanceNames -> contains all indexes in my elasticsearch. This metod works, but returns data is very slowly.

CodePudding user response：

Ok, I solved problem. I used AggregationBuilders to create 2 aggreagtion. First by index, second by provider.

        SearchRequest searchRequest = new SearchRequest();
    searchRequest.indices("_all");
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(QueryBuilders.matchAllQuery()).size(0);
    searchSourceBuilder.aggregation(AggregationBuilders.terms(aggregationIndexName)
            .field("_index")
            .size(100)
            .subAggregation(AggregationBuilders.terms(aggregationProviderName)
                    .field("provider.keyword")
                    .size(100)));
    searchRequest.source(searchSourceBuilder);