Get Free Trial
Get Free Trial

Understanding Flexmonster restrictions

posted on September 6th, 2018

Open
Andre Pinheiro asked on September 6, 2018

Folks,
 
We have chosen flexmonster mainly because of one of its features that supports loading large amounts of data, but I would like some help in order to explain some problems we are facing when trying to use this feature.
 
We have the following scenario:

  • We are trying to load in flexmonster 500.000 rows that are present in the database
  • The query takes about 1 minute to be executed and send its response to flexmonster
  • Flemonster then starts to process the response (i.e. load data... about 90.000.000 k, according to flexmonster info) and then to "analyze the data". These two steps takes around 5 minutes to process.
  • In chrome, flexmonster crashes. In Microsoft Edge, it runs fine, with the time reported previously

Based on this, i have some questions:

  • In your example (https://www.flexmonster.com/demos/js/one-million-rows/), it takes seconds to load 1 million lines. What can be impacting my project to cause it to take 5 minutes to load 500k rows (and to crash on chrome)? One difference I have spoted is that not all of the cells are filled on your example, but in mine all cells are filled. The ammount of columns are the same in both cases.
  • in your example, where is the data stored? On a database? On a file?

Thanks in advance

4 answers

Public
Ian Sadovy Ian Sadovy ⋅ Flexmonster ⋅ September 7, 2018

Hello Andre,
 
Thank you for the detailed explanation.
 
Answering the question about our example, for demo purposes, random data is generated via code so the loading time is minimal.
 
Since your data is stored in the database, here are our best practices for performance optimization:

  • Update Flexmonster Pivot to the latest version. In version 2.5.0 we have made significant memory optimization.
  • Use Flexmonster Data Compressor to load data from the database. It will decrease the data transfer and analyzing time.
  • Use a streaming approach to return data from the server (there are samples in the documentation). So the data loading and analyzing will be simultaneous (and faster as a result).
  • Use data prefiltering to load only necessary data to the client's browser.
  • Cache compressed responses (for example on the server file system) to avoid frequent queries to the database and save time.
  • Configure GZIP compression on the server. 

 
At Flexmonster we always pay a lot of attention to the performance. But still, loading 500.000 rows of data to the client's browser is a heavy task and requires a significant amount of RAM and processing time.
So, the following article may be useful for you as well: "What Data Source to Choose to Boost Your Pivot’s Performance".
 
Hope you find the above recommendations helpful.
Please let us know if you have any other questions.
 
Regards,
Ian

Public
Andre Pinheiro ⋅ September 7, 2018

Hello Ian,
 
thank you very much for your tips and prompt response. Here are some comments regading your recomendations:
 

  • Update Flexmonster Pivot: We are already using the 2.6.2 version.
  • Use Flexmonster Data Compressor to load data from the database AND use a streaming approach to return data from the server: I have a restriction on my project to implement what the documention recomends, but right now I am trying to overcome it. Certainly it will boost the loading time.
  • Use data prefiltering to load only necessary data to the client’s browser: Will talk to the business analists in order to see if any filter may be applied
  • Cache compressed responses: Going to check the possibles approachs to that.
  • Configure GZIP compression on the server: I am not very familiar to this. Do you have any reading recomendations?

 
Kindest regards,
Andre Pinheiro

Public
Ian Sadovy Ian Sadovy ⋅ Flexmonster ⋅ September 8, 2018

Hello Andre,
 
Thank you for the feedback.
As for the GZIP configuration, it depends on the type of the server you have (i.e. IIS, Apache, etc.).
You can find more in the following article: https://gtmetrix.com/enable-gzip-compression.html
Hope it helps.
 
Regards,
Ian

Public
Andre Pinheiro ⋅ September 10, 2018

Hello Ian,
 
many thanks for all of the suggestions.
 
Kindest regards,
Andre Pinheiro

Please login or Register to Submit Answer