asked on April 25, 2021
Hi,
We are using the flexmonster for one year. We have a requirement of loading a huge data into flexmonster. Currently, we are using a CSV file to load the data. Mainly we take the data from the database which is dynamic. So can you please suggest a way to load the data efficiently as CSV is taking a lot of time also our CSV file is increasing day by day. And there is also a limitation of memory in the client browser. Currently, the size of the file is almost 1.5GB. Also for custom source API or Flexmonster data server If I am not wrong, we have to load the data into Server memory and in such case, we can not fetch the data which is changed(After the data is loaded in memory). So kindly guide a way to implement such a scenario.
Regards,
Vaibhav
⋅ April 27, 2021
Hello, Vaibhav,
Thank you for writing to us.
We would like to mention that both Flexmonster Data Server and the custom data source API are the best approach for handling and analyzing a large data set.
This is because not all the data is loaded into the browser at once. Instead, only the part needed for the current display is loaded. When more data is needed for the representation, for example, when a row is expanded, then a new data query is initiated, and more data is loaded. Moreover, the data is aggregated on the server-side, thus enabling to handle larger data volumes.
The Flexmonster Data Server is the most convenient option in case of development since it is a ready-to-use solution. It allows handling large data sets from different data sources (JSON, CSV, and specific databases).
For Flexmonster Data Server, it is possible to set a data refresh time interval via the DataRefreshTime option. This way, Flexmonster Data Server will refresh the data with the specified rate. For example, you could configure it to refresh the data every several minutes. Please see our configuration guide for more details.
Another option is to implement the custom data source API, which is a protocol developed by our team that the Flexmonster Data Server is built on.
The custom data source API protocol provides a single set of requests and responses for connecting Flexmonster to any data source. This provides full control over how the data is processed, and you could create any desired data processing layer and manage how the data is refreshed.
Please let us know if this helps, and if any additional questions arise.
Looking forward to your response.
Kind regards,
Vera
