How to improve connection performance when loading large amount of records via elasticsearch?

Hello, Francisco,
 
Thank you for reaching out to us.
 
Our team kindly suggests trying the following:

1. Prefiltering the data:

   One way to decrease the server's response is to prefilter the data and return only the part necessary for the specific report.

2. Using subquery: 

   We would like to mention that the subquery parameter allows setting the server-side filter, which helps decrease the size of the response from the server.
   Please note that for the Elasticsearch data source type, the subquery parameter should be set as a Bool Query Object.

3. Setting more specific reports:

   Flexmonster provides the possibility to set default report settings, specifying how to display the data initially.
   This way, you can provide the desired starting points for the reports (for example, specifying which fields to display, applying an initial filter to the fields, etc.) and, in return, decrease the size of the response from the server.
   We have a section in our documentation with examples on how to configure the report. Please see https://www.flexmonster.com/doc/available-tutorials-report/.

 
 
Please let us know if this helps.
 
Kind regards, 
Vera

Hello, Francisco,
 
Thank you for your response. 
 
Flexmonster first requests all unique entries of a field to display them in the filter pop-up. Then the query for displaying the specified report is initiated, and the required filters are applied. It seems in your case, there are 180,000 unique company names, and this is why all 180,000 are requested at first. 
 
This can be adjusted by specifying the subquery property in the Data Source object.
Here is an example of how subquery could be used to decrease the server's response:

report: {
 "dataSource": {
   "type": "elasticsearch",
   "node": "https://olap.flexmonster.com:9200",
   "index": "fm-product-sales",
   "subquery": {
      "bool": {
         "filter": [
            {
             "terms": {
                "Category.keyword": ["Bikes", "Accessories"],
             }
            },
            {
             "terms": {
                "Color.keyword": ["blue", "red"]
             }
            }
         ]
      }
   },
   "mapping": {...}
  },
  "slice": {...}
}

As a result, only the specified members will be loaded as opposed to loading all of them.
We have prepared a JSFiddle example for illustration: https://jsfiddle.net/flexmonster/fo2kj4hw/.
 
Please let us know if this works fine for you.
 
Kind regards, 
Vera

Hello, Bernard,
 
Thank you for your question. 
 
Yes, subquery is part of the Data Source object.
 
Please let us know if the provided example helps.
 
Kind regards,
Vera